Reporting

View and audit Kyverno policy results with reports.

Policy reports are Kubernetes Custom Resources, generated and managed automatically by Kyverno, which contain the results of applying matching Kubernetes resources to Kyverno ClusterPolicy or Policy resources. They are created for validate, mutate, generate and verifyImages rules when a resource is matched by one or more rules according to the policy definition. If resources violate multiple rules, there will be multiple entries. When resources are deleted, their entry will be removed from the report. Reports, therefore, always represent the current state of the cluster and do not record historical information.

For example, if a validate policy in Audit mode exists containing a single rule which requires that all resources set the label team and a user creates a Pod which does not set the team label, Kyverno will allow the Pod’s creation but record it as a fail result in a policy report due to the Pod being in violation of the policy and rule. Policies configured with spec.rules[*].validate[*].failureAction: Enforce immediately block violating resources and results will only be reported for pass evaluations. Policy reports are an ideal way to observe the impact a Kyverno policy may have in a cluster without causing disruption. The insights gained from these policy reports may be used to provide valuable feedback to both users/developers so they may take appropriate action to bring offending resources into alignment, and to policy authors or cluster operators to help them refine policies prior to changing them to Enforce mode. Because reports are decoupled from policies, standard Kubernetes RBAC can then be applied to separate those who can see and manipulate policies from those who can view reports.

Policy reports are created based on two different triggers: an admission event (a CREATE, UPDATE, or DELETE action performed against a resource) or the result of a background scan discovering existing resources. Policy reports, like Kyverno policies, have both Namespaced and cluster-scoped variants; a PolicyReport is a Namespaced resource while a ClusterPolicyReport is a cluster-scoped resource. Reports are stored in the cluster on a per resource basis. Every namespaced resource will (eventually) have an associated PolicyReport and every clustered resource will (eventually) have an associated ClusterPolicyReport.

Kyverno uses a standard and open format published by the Kubernetes Policy working group which proposes a common policy report format across Kubernetes tools. Below is an example of a PolicyReport generated for a Pod which shows passing and failed rules.

 1apiVersion: wgpolicyk8s.io/v1alpha2
 2kind: PolicyReport
 3metadata:
 4  creationTimestamp: "2023-12-06T13:19:03Z"
 5  generation: 2
 6  labels:
 7    app.kubernetes.io/managed-by: kyverno
 8  name: 487df031-11d8-4ab4-b089-dfc0db1e533e
 9  namespace: kube-system
10  ownerReferences:
11  - apiVersion: v1
12    kind: Pod
13    name: kube-apiserver-kind-control-plane
14    uid: 487df031-11d8-4ab4-b089-dfc0db1e533e
15  resourceVersion: "720507"
16  uid: 0ec04a57-4c3d-492d-9278-951cd1929fe3
17results:
18- category: Pod Security Standards (Baseline)
19  message: validation rule 'adding-capabilities' passed.
20  policy: disallow-capabilities
21  result: pass
22  rule: adding-capabilities
23  scored: true
24  severity: medium
25  source: kyverno
26  timestamp:
27    nanos: 0
28    seconds: 1701868762
29- category: Pod Security Standards (Baseline)
30  message: 'validation error: Sharing the host namespaces is disallowed. The fields
31    spec.hostNetwork, spec.hostIPC, and spec.hostPID must be unset or set to `false`.
32    rule host-namespaces failed at path /spec/hostNetwork/'
33  policy: disallow-host-namespaces
34  result: fail
35  rule: host-namespaces
36  scored: true
37  severity: medium
38  source: kyverno
39  timestamp:
40    nanos: 0
41    seconds: 1701868762
42# ...
43scope:
44  apiVersion: v1
45  kind: Pod
46  name: kube-apiserver-kind-control-plane
47  namespace: kube-system
48  uid: 487df031-11d8-4ab4-b089-dfc0db1e533e
49summary:
50  error: 0
51  fail: 2
52  pass: 10
53  skip: 0
54  warn: 0

The report’s contents can be found under the results[] object in which it displays a number of fields including the resource that was matched against the rule in the parent policy.

Policy reports have a few configuration options available. For details, see the container flags section.

Report result logic

Entries in a policy report contain a result field which can be either pass, skip, warn, error, or fail.

ResultDescription
passThe resource was applicable to a rule and the pattern passed evaluation.
skipPreconditions were not satisfied (if applicable) in a rule, or an applicable PolicyException exists and so further processing was not performed.
failThe resource failed the pattern evaluation.
warnThe annotation policies.kyverno.io/scored has been set to "false" in the policy converting otherwise fail results to warn.
errorVariable substitution failed outside of preconditions and elsewhere in the rule (ex., in the pattern).

Scenarios for skipped evaluations

A skip result signifies that Kyverno decided not to fully evaluate the resource against a specific rule. This is different from a pass where the resource was evaluated and deemed compliant. A skip means the rule was essentially bypassed.

Here’s a breakdown of common scenarios resulting in a skip:

  1. Preconditions Not Met:

This is the most frequent reason for a skip. If a rule has preconditions defined and any of the conditions within the any or all blocks evaluate to FALSE, the entire rule is skipped. Kyverno won’t even attempt to apply the pattern, effectively bypassing the rule.

  1. Policy Exceptions:

Kyverno allows you to define exceptions to policies using PolicyException resources. If an exception exists that matches a specific resource and rule, Kyverno will skip the rule for that resource.

  1. Conditional Anchors () with Unmet Conditions:

When using a conditional anchor, the corresponding section is skipped if the condition within the anchor evaluates to FALSE.

  1. Global Anchors <() with Unmet Conditions:

Similar to conditional anchors, if the condition inside a global anchor is FALSE, the entire rule is skipped. The difference is that global anchors apply to the whole rule, not just a specific section.

  1. Anchor Logic Resulting in Skip:

As explained in the validate documentation, a combination of anchors and their evaluation results can lead to a skip. Specifically, a conditional anchor might be skipped, but if it’s a sibling to another condition that results in a pass or fail, the overall result will reflect that of the sibling, potentially masking the skip.

Example: If we have the following policy:

1spec:
2  =(initContainers):
3    - (name): "!istio-init"
4      =(securityContext):
5        =(runAsUser): ">0"
6  =(containers):
7    - =(securityContext):
8        =(runAsUser): ">0"

The following resource would result in pass:

1spec:
2  initContainers:
3  - name: istio-init
4    securityContext:
5      runAsUser: 0
6  containers:
7  - name: nginx
8    image: nginx

That’s because for the initContainers block the condition isn’t met so it’s a skip. But the containers block is a pass. So the overall result is a pass.

Key Points to Remember:

  • A skip result is not a failure; it’s a deliberate bypass based on predefined conditions or exceptions.
  • Understanding the distinction between pass and skip is crucial for accurately interpreting policy report data.
  • When troubleshooting a skip, carefully examine preconditions, exceptions, and the logic within your anchors to pinpoint the reason for the bypass.

Viewing policy report summaries

You can view a summary of the Namespaced policy reports using the following command:

1kubectl get policyreport -A

For example, below are the policy reports found in the kube-system namespace of a small test cluster created with kind.

 1$ kubectl get polr -n kube-system -o wide
 2NAME                                   KIND         NAME                                         PASS   FAIL   WARN   ERROR   SKIP   AGE
 3049a4ec1-32a5-4417-9184-1a59cfaa1ca6   DaemonSet    kindnet                                      9      3      0      0       0      16m
 4049d2cca-c30f-4f26-a70a-dfcc2cc5f433   DaemonSet    kube-proxy                                   9      3      0      0       0      16m
 51d491ec4-ca84-4b3a-960a-a2aefa3219ba   Pod          kube-controller-manager-kind-control-plane   10     2      0      0       0      16m
 634fa05b8-40cc-4bd3-836e-077abf4c126e   Pod          kindnet-qtq54                                9      3      0      0       0      16m
 73997d5d0-363a-4820-8768-4be3788b3968   Pod          kube-proxy-tcgcz                             9      3      0      0       0      16m
 84434c0ac-e27f-41eb-b4c2-b1a7aca8056a   ReplicaSet   coredns-5dd5756b68                           12     0      0      0       0      16m
 9487df031-11d8-4ab4-b089-dfc0db1e533e   Pod          kube-apiserver-kind-control-plane            10     2      0      0       0      16m
10553c0601-b995-4ed8-a36b-11e7cb38893b   Pod          kube-proxy-jdsck                             9      3      0      0       0      16m
1189044d72-8a1e-4af0-877b-9be727dc3ec4   Pod          kindnet-7rrns                                9      3      0      0       0      16m
129eb8c5c0-fe5c-4c7d-96c3-3ff65c361f4f   Pod          etcd-kind-control-plane                      10     2      0      0       0      16m
13b7968d37-4337-4756-bfe8-3c111f7a7356   Pod          kube-proxy-ncvxk                             9      3      0      0       0      16m
14cc894ef1-6a45-44e0-99f6-3765a59088e7   Pod          kube-scheduler-kind-control-plane            10     2      0      0       0      16m
15cf538bcc-4752-45d4-9712-480c425dc8d3   Pod          kindnet-c8fv6                                9      3      0      0       0      16m
16d9ea5169-17a7-458d-a971-09028a73cddd   Pod          coredns-5dd5756b68-z5whj                     12     0      0      0       0      16m
17e23946aa-17c3-4b96-b72b-eb7fd72eba62   Deployment   coredns                                      12     0      0      0       0      16m
18e666a741-c9cf-499c-a9c7-b8e0c600239a   Pod          kindnet-2rkgr                                9      3      0      0       0      16m
19e6f5aa6a-74e0-4c30-bb2b-1a6ee046e5ad   Pod          coredns-5dd5756b68-tnv25                     12     0      0      0       0      16m
20fd2aa944-3fc7-42b0-a6c0-1304e0aa473f   Pod          kube-proxy-p4x82                             9      3      0      0       0      16m

Similarly, you can view the cluster-wide report using:

1kubectl get clusterpolicyreport

Viewing policy violations

Since the report provides information on all rule and resource execution, returning only select entries requires a filter expression.

Policy reports can be inspected using either kubectl describe or kubectl get. For example, here is a command, requiring yq, to view only failures for the (Namespaced) report 1d491ec4-ca84-4b3a-960a-a2aefa3219ba:

1kubectl get polr 1d491ec4-ca84-4b3a-960a-a2aefa3219ba -o jsonpath='{.results[?(@.result=="fail")]}' | yq -p json -

 1category: Pod Security Standards (Baseline)
 2message: 'validation error: Privileged mode is disallowed. The fields spec.containers[*].securityContext.privileged and spec.initContainers[*].securityContext.privileged must be unset or set to `false`.          . rule privileged-containers failed at path /spec/containers/0/securityContext/privileged/'
 3policy: disallow-privileged-containers
 4result: fail
 5rule: privileged-containers
 6scored: true
 7severity: medium
 8source: kyverno
 9timestamp:
10  nanos: 0
11  seconds: 1.666094801e+09
12---
13category: Pod Security Standards (Baseline)
14message: 'validation error: Privileged mode is disallowed. The fields spec.containers[*].securityContext.privileged and spec.initContainers[*].securityContext.privileged must be unset or set to `false`.          . rule privileged-containers failed at path /spec/containers/0/securityContext/privileged/'
15policy: disallow-privileged-containers
16result: fail
17rule: privileged-containers
18scored: true
19severity: medium
20source: kyverno
21timestamp:
22  nanos: 0
23  seconds: 1.666095335e+09

Report internals

The PolicyReport and ClusterPolicyReport are the final resources composed of matching resources as determined by Kyverno Policy and ClusterPolicy objects, however these reports are built of four intermediary resources. For matching resources which were caught during admission mode, AdmissionReport and ClusterAdmissionReport resources are created. For results of background processing, BackgroundScanReport and ClusterBackgroundScanReport resources are created. An example of a ClusterAdmissionReport is shown below.

 1apiVersion: kyverno.io/v1alpha2
 2kind: ClusterAdmissionReport
 3metadata:
 4  creationTimestamp: "2022-10-18T13:15:09Z"
 5  generation: 1
 6  labels:
 7    app.kubernetes.io/managed-by: kyverno
 8    audit.kyverno.io/resource.hash: a7ec5160f220c5b83c26b5c8f7dc35b6
 9    audit.kyverno.io/resource.uid: 61946422-14ba-4aa2-94b4-229d38446381
10    cpol.kyverno.io/require-ns-labels: "4773"
11  name: c0cc7337-9bcd-4d53-abb2-93f7f5555216
12  resourceVersion: "4986"
13  uid: 10babc6c-9e6e-4386-abed-c13f50091523
14spec:
15  owner:
16    apiVersion: v1
17    kind: Namespace
18    name: testing
19    uid: 61946422-14ba-4aa2-94b4-229d38446381
20  results:
21  - message: 'validation error: The label `thisshouldntexist` is required. rule check-for-labels-on-namespace
22      failed at path /metadata/labels/thisshouldntexist/'
23    policy: require-ns-labels
24    result: fail
25    rule: check-for-labels-on-namespace
26    scored: true
27    source: kyverno
28    timestamp:
29      nanos: 0
30      seconds: 1666098909
31  summary:
32    error: 0
33    fail: 1
34    pass: 0
35    skip: 0
36    warn: 0

These intermediary resources have the same basic contents as a policy report and are used internally by Kyverno to build the final policy report. Kyverno will merge these results automatically into the appropriate policy report and there is no manual interaction typically required.

For more details on the internal reporting processes, see the developer docs here.

Background Scans

Periodically reapply policies to existing resources for reporting.

ValidatingAdmissionPolicy Reports

Generate Policy Reports for ValidatingAdmissionPolicies and their bindings.

Example Scenarios

Follow along scenarios for creating and viewing your first policy reports.


Last modified October 29, 2024 at 4:35 PM PST: fix: add warning to set permission in reports controller (#1403) (125df17)