Posted Updated 7 minutes read (About 1048 words)

RKE 部署随记

部署 RKE

前期准备：

1
2
3
4
5
6
7
# RKE 二进制
curl -LO "https://github.com/rancher/rke/releases/download/v1.5.12/rke_linux-amd64"

mv rke_linux-amd64 /usr/local/bin/rke && chmod +x /usr/local/bin/rke

# 各节点安装 Docker
curl https://releases.rancher.com/install-docker/20.10.sh | sh

生成配置：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
cat <<EOF > cluster.yml
# 旧版本 RKE 私钥类型不支持 rsa，需要选择 ed25519
ssh_key_path: /root/.ssh/id_ed25519
nodes:
  - address: 172.16.0.106
    hostname_override: RKE-server-0
    internal_address: 172.16.0.106
    user: root
    role:
      - controlplane
      - etcd
      - worker
  - address: 172.16.0.105
    hostname_override: RKE-server-1
    internal_address: 172.16.0.105
    user: root
    role:
      - controlplane
      - etcd
      - worker
  - address: 172.16.0.104
    hostname_override: RKE-server-2
    internal_address: 172.16.0.104
    user: root
    role:
      - controlplane
      - etcd
      - worker
private_registries:
  - url: registry.cn-hangzhou.aliyuncs.com
    is_default: true
kubernetes_version: "v1.20.15-rancher2-2"
network:
  plugin: calico
EOF

安装 RKE：

1
rke up --config cluster.yml

方便后续运维配置：

1
2
3
4
5
6
7
8
9
10
11
docker cp kube-apiserver:usr/local/bin/kubectl /usr/local/bin/kubectl

echo "source <(kubectl completion bash)" >> ~/.bashrc

mkdir ~/.kube

mv kube_config_cluster.yml ~/.kube/config

curl https://rancher-mirror.rancher.cn/helm/get-helm-3.sh | INSTALL_HELM_MIRROR=cn bash -s -- --version v3.17.1

echo "source <(helm completion bash)" >> ~/.bashrc

Custom 类型 RKE：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
docker cp kube-apiserver:usr/local/bin/kubectl /usr/local/bin/kubectl

echo "source <(kubectl completion bash)" >> ~/.bashrc

mkdir ~/.kube

# 参考 KB 生成 Kubeconfig，也可以在 Rancher UI 下载：https://www.suse.com/support/kb/doc/?id=000020018
# Rancher 2.7.14+/Rancher 2.8.5+, RKE 1.4.19+/RKE 1.5.10+
kubectl --kubeconfig $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl/kubecfg-kube-node.yaml get secrets -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r .currentState.certificatesBundle.\"kube-admin\".config | sed -e "/^[[:space:]]*server:/ s_:.*_: \"https://127.0.0.1:6443\"_" > ~/.kube/config

# 如果没有 jq 命令
docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=org.opencontainers.image.source=https://github.com/rancher/hyperkube) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml get secret -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | base64 -d | jq -r .currentState.certificatesBundle.\"kube-admin\".config | sed -e "/^[[:space:]]*server:/ s_:.*_: \"https://127.0.0.1:6443\"_"' > ~/.kube/config

# 更早之前的版本
kubectl --kubeconfig $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl/kubecfg-kube-node.yaml get configmap -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .currentState.certificatesBundle.\"kube-admin\".config | sed -e "/^[[:space:]]*server:/ s_:.*_: \"https://127.0.0.1:6443\"_" > ~/.kube/config

# 如果没有 jq 命令
docker run --rm --net=host -v $(docker inspect kubelet --format '{{ range .Mounts }}{{ if eq .Destination "/etc/kubernetes" }}{{ .Source }}{{ end }}{{ end }}')/ssl:/etc/kubernetes/ssl:ro --entrypoint bash $(docker inspect $(docker images -q --filter=label=org.opencontainers.image.source=https://github.com/rancher/hyperkube.git) --format='{{index .RepoTags 0}}' | tail -1) -c 'kubectl --kubeconfig /etc/kubernetes/ssl/kubecfg-kube-node.yaml get configmap -n kube-system full-cluster-state -o json | jq -r .data.\"full-cluster-state\" | jq -r .currentState.certificatesBundle.\"kube-admin\".config | sed -e "/^[[:space:]]*server:/ s_:.*_: \"https://127.0.0.1:6443\"_"' > ~/.kube/config

curl https://rancher-mirror.rancher.cn/helm/get-helm-3.sh | INSTALL_HELM_MIRROR=cn bash -s -- --version v3.17.1

echo "source <(helm completion bash)" >> ~/.bashrc

常见问题

如果是 CentOS 和 RHEL 系统，默认不允许使用 root 用户进行安装，报错信息如下：

1
WARN[0000] Failed to set up SSH tunneling for host [x.x.x.x]: Can’t retrieve Docker Info ，Failed to dial to /var/run/docker.sock: ssh: rejected: administratively prohibited (open failed)

需要准备其他用户：

1
groupadd rancher && useradd rancher -g rancher && usermod -aG docker rancher

如果出现下面错误，是由于指定的 ssh_key_path 文件对应的主机不正确或对应的用户名不正确，可以检查下节点对应用户的 ~/.ssh/authorized_keys 文件是否正确：

1
WARN[0000] Failed to set up SSH tunneling for host [x.x.x.x]: Can't retrieve Docker Info: error during connect: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info": Unable to access node with address [x.x.x.x:22] using SSH. Please check if you are able to SSH to the node using the specified SSH Private Key and if you have configured the correct SSH username. Error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

如果出现下面错误：

1
WARN[0000] Failed to set up SSH tunneling for host [x.x.x.x]: Can't retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access the service on /var/run/docker.sock. The service might be still starting up. Error: ssh: rejected: connect failed (open failed)

需要在 /etc/ssh/sshd_config 文件中添加以下内容：

1
AllowTcpForwarding yes

清理 iptables 规则

1
2
3
4
5
6
7
iptables -F \
  && iptables -X \
  && iptables -Z \
  && iptables -F -t nat \
  && iptables -X -t nat \
  && iptables -Z -t nat \
  && docker restart kube-proxy

清理节点

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
docker stop $(docker ps -aq)

docker rm -f $(docker ps -qa)

docker volume rm $(docker volume ls -q)

for mount in $(mount | grep tmpfs | grep '/var/lib/kubelet' | awk '{ print $3 }') /var/lib/kubelet /var/lib/rancher;
do
  umount $mount;
done

rm -rf /etc/ceph \
       /etc/cni \
       /etc/kubernetes \
       /opt/cni \
       /opt/rke \
       /run/calico \
       /run/flannel \
       /run/secrets/kubernetes.io \
       /var/lib/calico \
       /var/lib/cni \
       /var/lib/etcd \
       /var/lib/kubelet \
       /var/lib/rancher/rke \
       /var/lib/weave \
       /var/log/containers \
       /var/log/kube-audit \
       /var/log/pods \
       /var/run/calico

reboot

RKE 部署随记

https://warnerchen.github.io/2024/09/05/RKE-部署随记/

Author

Warner Chen

Posted on

2024-09-05

Updated on

2025-09-02

Licensed under

#
You need to set install_url to use ShareThis. Please set it in _config.yml.

Like this article? Support the author with

You forgot to set the business or currency_code for Paypal. Please set it in _config.yml.

Comments

You forgot to set the shortname for Disqus. Please set it in _config.yml.
Follow

Links

Recents

Archives

Tags

ai2
ansible1
calico2
cert-manager1
cilium1
cisco1
container5
containerd1
containers1
docker2
elasticsearch1
elk1
etcd3
fluentd1
harbor1
istio2
jenkins2
jfrog1
kafka1
kubernetes9
linux4
metallb1
mysql1
nexus1
others2
prometheus3
redis2
ssl1
suse37
web1
zookeeper1
面试1
首页1

Subscribe for updates

You need to set client_id and slot_id to show this AD unit. Please set it in _config.yml.

follow.it

×