There are cases where either an operations or security incident may occur and Nodes should be evacuated and placed in an unused state for further analysis. For example, a Node is found to be running a vulnerable version of a CRI engine or kernel and to minimize chances of a compromise may need to be decommissioned so another can be built. This policy shows how to use Kyverno to both cordon and drain a given Node and uses a hypothetical label being written to it called `testing=drain` to illustrate the point. For production use, the match block should be modified to trigger on the appropriate condition.
apiVersion: kyverno.io/v1kind: ClusterPolicymetadata:name: cordon-and-drain-nodeannotations:policies.kyverno.io/title: Cordon and Drain Nodepolicies.kyverno.io/category: Otherpolicies.kyverno.io/subject: Nodekyverno.io/kyverno-version: 1.10.1policies.kyverno.io/minversion: 1.10.0kyverno.io/kubernetes-version: "1.26"policies.kyverno.io/description: There are cases where either an operations or security incident may occur and Nodes should be evacuated and placed in an unused state for further analysis. For example, a Node is found to be running a vulnerable version of a CRI engine or kernel and to minimize chances of a compromise may need to be decommissioned so another can be built. This policy shows how to use Kyverno to both cordon and drain a given Node and uses a hypothetical label being written to it called `testing=drain` to illustrate the point. For production use, the match block should be modified to trigger on the appropriate condition.spec:rules:- name: mutate-nodematch:any:- resources:kinds:- Nodeoperations:- UPDATEselector:matchLabels:testing: drainmutate:targets:- apiVersion: v1kind: Nodename: "{{request.object.metadata.name}}"patchStrategicMerge:spec:unschedulable: truetaints:- effect: NoExecutekey: kyverno-evictedtimeAdded: "{{ time_now_utc() }}"
The Kubernetes cluster autoscaler does not evict pods that use hostPath or emptyDir volumes. To allow eviction of these pods, the annotation cluster-autoscaler.kubernetes.io/safe-to-evict=true must be added to the pods.
The Kubernetes cluster autoscaler does not evict pods that use hostPath or emptyDir volumes. To allow eviction of these pods, the annotation cluster-autoscaler.kubernetes.io/safe-to-evict=true must be added to the pods.
CAST AI will not downscale a node that includes a pod with the autoscaling.cast.ai/removal-disabled="true" label on it, this protects sensitive workloads from being evicted and can be attributed to any pod to protect against unwanted downscaling. This policy will mutate jobs and cronjobs to add the removal-disabled label to protect against eviction.