How do I remove a Control Plane or Etcd node from a Kubernetes cluster managed by Kubespray? This guide walks you through the safe removal of a Kubernetes node serving as a control plane or etcd node using Kubespray. In our previous article, we discussed in detail how to deploy Kubernetes on Ubuntu using Kubespray.
Confirm cluster nodes
Ensure that the Kubeconfig file is properly configured on your Kubespray management node:
mkdir ~/.kube
vim ~/.kube/configYou can copy the contents of /etc/kubernetes/admin.conf from one of the master nodes.
List active nodes in your cluster:
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8smas01 Ready control-plane 18m v1.31.4
k8smas02 Ready control-plane 18m v1.31.4
k8smas03 Ready control-plane 18m v1.31.4As shown in the output, the cluster includes nodes functioning as control plane, etcd, and worker nodes. In a best-practice production setup, these roles should ideally be distributed across separate nodes – especially separating control plane nodes from worker nodes.
Removing a node from kubernetes cluster
It is possible to safely remove cluster nodes from your Kubespray-deployed Kubernetes cluster using the remove-node.yml playbook. This process can proof to be useful for tasks like nodes reconfiguration, decommissioning, or when doing autoscaling.
⚠️ Important Notes:
- ❌ The playbook does not support removing the first control plane or etcd node. These is critical node for cluster operations and must remain active.
- 🔐 If a node is unreachable via SSH, use this option to skip the reset step:
--extra-vars "reset_nodes=false"You can set the following host variable in your inventory if only one of multiple nodes is unreachable.
reset_nodes=falseExample command to remove the node k8smas03:
ansible-playbook -i inventory/cluster1/hosts.yaml remove-node.yml \
-e "node=k8smas03"You can also use --limit:
--limit "kube_control_plane:etcd"Answer yes
Are you sure you want to delete nodes state? Type 'yes' to delete nodes.:
yesSuccessful node removal ansible playbook output:
TASK [network_plugin/cilium : Reset | remove network device cilium_host] *********************************************************************************************************************************************************************************
changed: [k8smas03]
Tuesday 22 April 2025 16:56:50 +0200 (0:00:00.202) 0:04:15.805 *********
TASK [network_plugin/cilium : Reset | check if network device cilium_net is present] *********************************************************************************************************************************************************************
ok: [k8smas03]
Tuesday 22 April 2025 16:56:51 +0200 (0:00:00.197) 0:04:16.002 *********
Tuesday 22 April 2025 16:56:51 +0200 (0:00:00.023) 0:04:16.026 *********
TASK [network_plugin/cilium : Reset | check if network device cilium_vxlan is present] *******************************************************************************************************************************************************************
ok: [k8smas03]
Tuesday 22 April 2025 16:56:51 +0200 (0:00:00.195) 0:04:16.222 *********
TASK [network_plugin/cilium : Reset | remove network device cilium_vxlan] ********************************************************************************************************************************************************************************
changed: [k8smas03]
Tuesday 22 April 2025 16:56:51 +0200 (0:00:00.234) 0:04:16.456 *********
TASK [reset : Gather active network services] ************************************************************************************************************************************************************************************************************
ok: [k8smas03] => (item=NetworkManager)
ok: [k8smas03] => (item=systemd-networkd)
ok: [k8smas03] => (item=networking)
ok: [k8smas03] => (item=network)
Tuesday 22 April 2025 16:56:52 +0200 (0:00:01.285) 0:04:17.741 *********
TASK [reset : Restart active network services] ***********************************************************************************************************************************************************************************************************
changed: [k8smas03] => (item=systemd-networkd)
PLAY [Post node removal] *********************************************************************************************************************************************************************************************************************************
Tuesday 22 April 2025 16:56:53 +0200 (0:00:00.450) 0:04:18.192 *********
Tuesday 22 April 2025 16:56:53 +0200 (0:00:00.026) 0:04:18.218 *********
Tuesday 22 April 2025 16:56:53 +0200 (0:00:00.026) 0:04:18.244 *********
Tuesday 22 April 2025 16:56:53 +0200 (0:00:00.023) 0:04:18.268 *********
Tuesday 22 April 2025 16:56:53 +0200 (0:00:00.025) 0:04:18.293 *********
Tuesday 22 April 2025 16:56:53 +0200 (0:00:00.031) 0:04:18.325 *********
TASK [remove-node/post-remove : Remove-node | Delete node] ***********************************************************************************************************************************************************************************************
changed: [k8smas03 -> k8smas01(172.35.1.58)]
PLAY RECAP ***********************************************************************************************************************************************************************************************************************************************
k8smas01 : ok=17 changed=1 unreachable=0 failed=0 skipped=13 rescued=0 ignored=0
k8smas02 : ok=14 changed=1 unreachable=0 failed=0 skipped=12 rescued=0 ignored=0
k8smas03 : ok=57 changed=22 unreachable=0 failed=0 skipped=29 rescued=0 ignored=0
Tuesday 22 April 2025 16:56:53 +0200 (0:00:00.290) 0:04:18.615 *********
===============================================================================
reset : Reset | stop all cri pods --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 181.12s
reset : Reset | force remove all cri pods -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 20.37s
reset : Reset | delete some files and directories ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 15.39s
remove-node/pre-remove : Remove-node | Drain node except daemonsets resource ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- 7.40s
Confirm Execution --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 6.70s
Gather information about installed services ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.76s
reset : Reset | stop services --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2.57s
reset : Reset | remove containerd binary files ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.86s
reset : Reset | remove services ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.82s
reset : Gather active network services ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 1.29s
bootstrap-os : Ensure iproute2 is installed ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 1.23s
reset : Reset | systemctl daemon-reload ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.90s
reset : Flush iptables ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.89s
Gather necessary facts (hardware) ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.87s
Gather necessary facts (network) ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.76s
bootstrap-os : Gather facts ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.74s
reset : Reset | unmount kubelet dirs -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.72s
reset : Reset | stop all cri containers ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.68s
Gather minimal facts ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 0.48s
reset : Reset | remove dns settings from dhclient.conf -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 0.46sConfirm Nodes count in your cluster
List current list of all nodes in your Kubernetes cluster:
# kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8smas01 Ready control-plane 33m v1.31.4
k8smas02 Ready control-plane 32m v1.31.4We can confirm the node count has decreased by one, reflecting the removal of the specified node.
Update inventory / Reconfigure node(s)
After removing a node, you have the flexibility to reconfigure it – this could involve reinstalling the operating system, updating hardware specs, or any other necessary changes.
If you don’t plan to reconfigure the node, simply remove its entry from the inventory file to clean up your configuration.
all:
hosts:
k8smas01:
ansible_host: 192.168.20.18
k8smas02:
ansible_host: 192.168.20.19
children:
kube_control_plane:
hosts:
k8smas01:
k8smas02:
kube_node:
hosts:
k8smas01:
k8smas02:
etcd:
hosts:
k8smas01:
k8smas02:
k8s_cluster:
children:
kube_control_plane:
kube_node:
calico_rr:
hosts: {}If the node will be reused, once the OS is reset to your desired state – Update the node’s hostname and IP address in the inventory file. For example:
all:
hosts:
k8smas01:
ansible_host: 192.168.20.18
k8smas02:
ansible_host: 192.168.20.19
k8snode01:
ansible_host: 192.168.20.22
children:
kube_control_plane:
hosts:
k8smas01:
k8smas02:
k8snode01:
kube_node:
hosts:
k8smas01:
k8smas02:
k8snode01:
etcd:
hosts:
k8smas01:
k8smas02:
k8snode01:
k8s_cluster:
children:
kube_control_plane:
kube_node:
calico_rr:
hosts: {}Execute the cluster.yml playbook to set up and reconfigure your cluster nodes.
ansible-playbook -i inventory/cluster1/hosts.yaml --become --become-user=root cluster.ymlIf you’re only adding control plane and etcd nodes, you can limit the Ansible playbook run to just those nodes to avoid unnecessary reconfiguration of the entire cluster.
ansible-playbook -i inventory/cluster1/hosts.yaml cluster.yml \
--become \
--become-user=root \
--limit "kube_control_plane:etcd"You can also limit by specific hostnames if needed, for example when adding worked node.
--limit "k8snode01"After reconfiguring, check available nodes in the cluster:
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
k8smas01 Ready control-plane 52m v1.31.4
k8smas02 Ready control-plane 52m v1.31.4
k8snode01 Ready control-plane 118s v1.31.4This wraps up our guide on removing a node from a Kubernetes cluster. For more in-depth articles on Kubernetes, be sure to explore our blog page.
