Add Karpenter nodeSelector

Selecting the correct Node(s) provisioned by Karpenter is a way to specify the appropriate resource landing zone for a workload. This policy injects a nodeSelector map into the Pod based on the Namespace type where it is deployed.

View on GitHub

Policy Definition

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-karpenter-nodeselector
  annotations:
    policies.kyverno.io/title: Add Karpenter nodeSelector
    policies.kyverno.io/category: Karpenter, EKS Best Practices
    policies.kyverno.io/severity: medium
    policies.kyverno.io/subject: Pod
    kyverno.io/kyverno-version: 1.7.1
    policies.kyverno.io/minversion: 1.6.0
    kyverno.io/kubernetes-version: "1.23"
    policies.kyverno.io/description: Selecting the correct Node(s) provisioned by Karpenter is a way to specify the appropriate resource landing zone for a workload. This policy injects a nodeSelector map into the Pod based on the Namespace type where it is deployed.
spec:
  rules:
    - name: add-medium
      match:
        any:
          - resources:
              kinds:
                - Pod
              namespaceSelector:
                matchLabels:
                  type: medium
      mutate:
        patchStrategicMerge:
          spec:
            nodeSelector:
              kubernetes.io/arch: amd64
              karpenter.sh/capacity-type: spot
    - name: add-large
      match:
        any:
          - resources:
              kinds:
                - Pod
              namespaceSelector:
                matchLabels:
                  type: large
      mutate:
        patchStrategicMerge:
          spec:
            nodeSelector:
              kubernetes.io/arch: amd64
              karpenter.sh/capacity-type: on-demand

Related Policies

ValidateMedium

Require Encryption with AWS LoadBalancers

Services of type LoadBalancer when deployed inside AWS have support for transport encryption if it is enabled via an annotation. This policy requires that Services of type LoadBalancer contain the annotation service.beta.kubernetes.io/aws-load-balancer-ssl-cert with some value.

Service

ValidateMedium

Require aws-node DaemonSet use IRSA

According to EKS best practices, the `aws-node` DaemonSet is configured to use a role assigned to the EC2 instances to assign IPs to Pods. This role includes several AWS managed policies that effectively allow all Pods running on a Node to attach/detach ENIs, assign/unassign IP addresses, or pull images from ECR. Since this presents a risk to your cluster, it is recommended that you update the `aws-node` DaemonSet to use IRSA. This policy ensures that the `aws-node` DaemonSet running in the `kube-system` Namespace is not still using the `aws-node` ServiceAccount.

DaemonSet

GenerateMedium

Add Network Policy

By default, Kubernetes allows communications across all Pods within a cluster. The NetworkPolicy resource and a CNI plug-in that supports NetworkPolicy must be used to restrict communications. A default NetworkPolicy should be configured for each Namespace to default deny all ingress and egress traffic to the Pods in the Namespace. Application teams can then configure additional NetworkPolicy resources to allow desired traffic to application Pods from select sources. This policy will create a new NetworkPolicy resource named `default-deny` which will deny all traffic anytime a new Namespace is created.

NetworkPolicy