Kubeadm证书过期时间调整
kubeadm 默认证书为一年，一年过期后，会导致api service不可用，使用过程中会出现：x509: certificate has expired or is not yet valid
如何进行调整，下面给了两个方案，供大家选择
方案一 通过修改kubeadm 调整证书过期时间
修改源代码，调整过期时间
[具体参考]：https://github.com/21ki/kubernetes/blob/master/.github/workflows/go.yml
替换证书
#用新的kubeadm 替换官方的kubeadm
chmod +x kubeadm && \cp -f kubeadm /usr/bin
#备份原有的证书
mv /etc/kubernetes/pki /etc/kubernetes/pki.old
#如果服务器保存得又kubeadm.conf 可直接使用 如果没有可以用以下命令生成
kubeadm config view > kubeadm.yaml
#生成新的证书，kubeadm.yaml 指定你自己服务器上的
kubeadm alpha phase certs all --config ~/kubeadm.yaml
#备份原有的conf文件
mv /etc/kubernetes/*conf /etc/kubernetes/*conf-old
#根据新证书重新生成新的配置文件
kubeadm alpha phase kubeconfig all --config ~/kubeadm.yaml
#替换老的config文件
\cp -f /etc/kubernetes/admin.conf ~/.kube/config
验证
cd /etc/kubernetes/pki
for i in $(ls *.crt); do echo "===== $i ====="; openssl x509 -in $i -text -noout | grep -A 3 'Validity' ; done
#===== apiserver.crt =====
# Validity
# Not Before: Sep 17 01:48:46 2020 GMT
# Not After : Sep 17 01:48:46 2021 GMT
# Subject: CN=kube-apiserver
#===== apiserver-etcd-client.crt =====
# Validity
# Not Before: Sep 17 01:48:48 2020 GMT
# Not After : Sep 17 01:48:49 2021 GMT
# Subject: O=system:masters, CN=kube-apiserver-etcd-client
#===== apiserver-kubelet-client.crt =====
# Validity
# Not Before: Sep 17 01:48:46 2020 GMT
# Not After : Sep 17 01:48:47 2021 GMT
# Subject: O=system:masters, CN=kube-apiserver-kubelet-client
#===== ca.crt =====
# Validity
# Not Before: Sep 17 01:48:46 2020 GMT
# Not After : Sep 15 01:48:46 2030 GMT
# Subject: CN=kubernetes
#===== front-proxy-ca.crt =====
# Validity
# Not Before: Sep 17 01:48:47 2020 GMT
# Not After : Sep 15 01:48:47 2030 GMT
# Subject: CN=front-proxy-ca
#===== front-proxy-client.crt =====
# Validity
# Not Before: Sep 17 01:48:47 2020 GMT
# Not After : Sep 17 01:48:48 2021 GMT
# Subject: CN=front-proxy-client
#
# 批量验证证书
for crt in $(find /etc/kubernetes/pki/ -name "*.crt"); do openssl x509 -in $crt -noout -dates; done
方案二 启用自动轮换kubelet 证书
kubelet证书分为server和client两种， k8s 1.9默认启用了client证书的自动轮换，但server证书自动轮换需要用户开启
增加 kubelet 参数
# 在/etc/systemd/system/kubelet.service.d/10-kubeadm.conf 增加如下参数
Environment="KUBELET_EXTRA_ARGS=--feature-gates=RotateKubeletServerCertificate=true"
增加 controller-manager 参数
# 在/etc/kubernetes/manifests/kube-controller-manager.yaml 添加如下参数
- command:
- kube-controller-manager
- --experimental-cluster-signing-duration=87600h0m0s
- --feature-gates=RotateKubeletServerCertificate=true
- ....
创建 rbac 对象 创建rbac对象，允许节点轮换kubelet server证书：
cat > ca-update.yaml << EOF
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:certificates.k8s.io:certificatesigningrequests:selfnodeserver
rules:
- apiGroups:
- certificates.k8s.io
resources:
- certificatesigningrequests/selfnodeserver
verbs:
- create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: kubeadm:node-autoapprove-certificate-server
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:certificates.k8s.io:certificatesigningrequests:selfnodeserver
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: Group
name: system:nodes
EOF
kubectl create –f ca-update.yaml
如果证书已经过期，如何进行重新签发证书 针对kubeadm 1.13.x 及以上处理 准备kubeadm.conf 配置文件一份
apiVersion: kubeadm.k8s.io/v1beta1
kind: ClusterConfiguration
kubernetesVersion: v1.14.1 #-->这里改成你集群对应的版本
imageRepository: registry.cn-hangzhou.aliyuncs.com/google_containers
#这里使用国内的镜像仓库，否则在重新签发的时候会报错：could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt"
重新签发命令
kubeadm alpha certs renew all --config=/root/kubeadm.conf
运行如上命令会重新生成以下证书
#-- /etc/kubernetes/pki/apiserver.key
#-- /etc/kubernetes/pki/apiserver.crt
#-- /etc/kubernetes/pki/apiserver-etcd-client.key
#-- /etc/kubernetes/pki/apiserver-etcd-client.crt
#-- /etc/kubernetes/pki/apiserver-kubelet-client.key
#-- /etc/kubernetes/pki/apiserver-kubelet-client.crt
#-- /etc/kubernetes/pki/front-proxy-client.key
#-- /etc/kubernetes/pki/front-proxy-client.crt
#-- /etc/kubernetes/pki/etcd/healthcheck-client.key
#-- /etc/kubernetes/pki/etcd/healthcheck-client.crt
#-- /etc/kubernetes/pki/etcd/peer.key
#-- /etc/kubernetes/pki/etcd/peer.crt
#-- /etc/kubernetes/pki/etcd/server.key
#-- /etc/kubernetes/pki/etcd/server.crt
更新/etc/kubernetes/*.conf文件
#备份删除旧的/etc/kubernetes/*.conf文件
mkdir /etc/kubernetes/old-conf
mv /etc/kubernetes/*.conf /etc/kubernetes/old-conf
#生成新的conf文件
kubeadm init phase kubeconfig all --config=/root/kubeadm.conf
#运行如上命令会重新生成以下conf文件
#-- /etc/kubernetes/admin.conf
#-- /etc/kubernetes/controller-manager.conf
#-- /etc/kubernetes/kubelet.conf
#-- /etc/kubernetes/scheduler.conf
完成后重启kube-apiserver,kube-controller,kube-scheduler,etcd这4个容器,最后覆盖config文件
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
docker ps | grep -v pause | grep -E "etcd|scheduler|controller|apiserver" | awk '{print $1}' | awk '{print "docker","restart",$1}' | bash
针对kubeadm 1.13.0（不包含1.13.0） 以下处理 移动证书和配置【注意！必须移动，不然会使用现有的证书，不会重新生成】
cd /etc/kubernetes
mkdir ./pki_bak
mkdir ./pki_bak/etcd
mkdir ./conf_bak
mv pki/apiserver* ./pki_bak/
mv pki/front-proxy-client.* ./pki_bak/
mv pki/etcd/healthcheck-client.* ./pki_bak/etcd/
mv pki/etcd/peer.* ./pki_bak/etcd/
mv pki/etcd/server.* ./pki_bak/etcd/
mv ./admin.conf ./conf_bak/
mv ./kubelet.conf ./conf_bak/
mv ./controller-manager.conf ./conf_bak/
mv ./scheduler.conf ./conf_bak/
创建证书
kubeadm alpha phase certs all --apiserver-advertise-address=${MASTER_API_SERVER_IP} --apiserver-cert-extra-sans=主机内网ip,主机公网ip
运行如上命令会重新生成以下证书
#-- /etc/kubernetes/pki/apiserver.key
#-- /etc/kubernetes/pki/apiserver.crt
#-- /etc/kubernetes/pki/apiserver-etcd-client.key
#-- /etc/kubernetes/pki/apiserver-etcd-client.crt
#-- /etc/kubernetes/pki/apiserver-kubelet-client.key
#-- /etc/kubernetes/pki/apiserver-kubelet-client.crt
#-- /etc/kubernetes/pki/front-proxy-client.key
#-- /etc/kubernetes/pki/front-proxy-client.crt
#-- /etc/kubernetes/pki/etcd/healthcheck-client.key
#-- /etc/kubernetes/pki/etcd/healthcheck-client.crt
#-- /etc/kubernetes/pki/etcd/peer.key
#-- /etc/kubernetes/pki/etcd/peer.crt
#-- /etc/kubernetes/pki/etcd/server.key
#-- /etc/kubernetes/pki/etcd/server.crt
不移动证书会有如下提示
#[certificates] Using the existing apiserver certificate and key.
#[certificates] Using the existing apiserver-kubelet-client certificate and key.
#[certificates] Using the existing front-proxy-client certificate and key.
#[certificates] Using the existing etcd/server certificate and key.
#[certificates] Using the existing etcd/peer certificate and key.
#[certificates] Using the existing etcd/healthcheck-client certificate and key.
#[certificates] Using the existing apiserver-etcd-client certificate and key.
#[certificates] valid certificates and keys now exist in "/etc/kubernetes/pki"
#[certificates] Using the existing sa key.
生成新配置文件
kubeadm alpha phase kubeconfig all --apiserver-advertise-address=${MASTER_API_SERVER_IP}
将新生成的admin配置文件覆盖掉原本的admin文件
mv $HOME/.kube/config $HOME/.kube/config.old
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
sudo chmod 644 $HOME/.kube/config
完成后重启kube-apiserver,kube-controller,kube-scheduler,etcd这4个容器
文章作者 🐳Myki
上次更新 2020-09-21