Cordon and Drain Node

There are cases where either an operations or security incident may occur and Nodes should be evacuated and placed in an unused state for further analysis. For example, a Node is found to be running a vulnerable version of a CRI engine or kernel and to minimize chances of a compromise may need to be decommissioned so another can be built. This policy shows how to use Kyverno to both cordon and drain a given Node and uses a hypothetical label being written to it called `testing=drain` to illustrate the point. For production use, the match block should be modified to trigger on the appropriate condition.

Policy Definition

/other/cordon-and-drain-node/cordon-and-drain-node.yaml

 1apiVersion: kyverno.io/v1
 2kind: ClusterPolicy
 3metadata:
 4  name: cordon-and-drain-node
 5  annotations:
 6    policies.kyverno.io/title: Cordon and Drain Node
 7    policies.kyverno.io/category: Other
 8    policies.kyverno.io/subject: Node
 9    kyverno.io/kyverno-version: 1.10.1
10    policies.kyverno.io/minversion: 1.10.0
11    kyverno.io/kubernetes-version: "1.26"
12    policies.kyverno.io/description: >-
13      There are cases where either an operations or security incident may occur and Nodes
14      should be evacuated and placed in an unused state for further analysis. For example,
15      a Node is found to be running a vulnerable version of a CRI engine or kernel and to
16      minimize chances of a compromise may need to be decommissioned so another can be built.
17      This policy shows how to use Kyverno to both cordon and drain a given Node and uses a
18      hypothetical label being written to it called `testing=drain` to illustrate the point.
19      For production use, the match block should be modified to trigger on the appropriate
20      condition.
21spec:
22  rules:
23    - name: mutate-node
24      match:
25        any:
26        - resources:
27            kinds:
28            - Node
29            operations:
30            - UPDATE
31            selector:
32              matchLabels:
33                testing: drain
34      mutate:
35        targets:
36        - apiVersion: v1
37          kind: Node
38          name: "{{request.object.metadata.name}}"
39        patchStrategicMerge:
40          spec:
41            unschedulable: true
42            taints:
43            - effect: NoExecute
44              key: kyverno-evicted
45              timeAdded: "{{ time_now_utc() }}"

Create policy issue