Generate Kyverno Admission Controller Policies with KubeArmor®

What are Kubernetes admission controllers?

Admission Controllers enforce semantic validation of objects during create, update, and delete operations. Kubernetes Admission controllers leverage admission webhooks which are HTTP callbacks. These receive admission requests before k8s resources are deployed such that the controllers can perform some actions such as security, compliance, and governance checks before admitting the resources. Admission controllers use validating and mutating webhooks (some extend to ValidatingAdmissionPolicy too) to control the operations performed through the kube-api server.

Why is Kubernetes admission controller required?

Kubernetes admission controller can enforce custom policies that are not possible with Kubernetes RBAC or Pod Security Policies. Almost all Kubernetes admission controllers are easy to integrate with an existing cluster and being highly configurable, they can severely enhance security posture. Some notable policy examples are:

1. Resource Limits
2. Trusted Container Images
3. Label Existence
4. Security Context
5. Ingress, Egress, and NetworkPolicy governance

How can KubeArmor runtime visibility enhance Kubernetes admission controller policies?

Admission controllers, as the name suggests, enforce policies at the time of admission. Consequently, administrators need to understand application behavior before enforcing policies to ensure that a mistake does not lead to downtime in application availability.

Runtime security platforms like AccuKnox and its OpenSource companion, KubeArmor can detect application behavior and generate admission controller policies based on the container or the application behavior. Administrators can deploy applications in the pre-production stage for AccuKnox/KubeArmor to collect runtime data and generate admission controller policies according to application behavior.

Zero Trust with service account token

For any pod deployed on the cluster, the service account token is automatically mounted onto that container unless explicitly setting automountServiceAccountToken: false . Unless explicitly specified, the default service account token of the particular namespace is mounted onto the pod. Though the permissions of the default token are highly restricted, these can be enhanced by binding it with some role or clusterrole.

What attacks are possible with service account tokens?

"Every possible attack that can be performed after accessing the kube-api server"
The service account token is used to access the kube-api server. The attack surface varies according to the role assigned to the service account token. After getting the service account token, the attacker can impersonate the application and access the API server to even perform crypto-jacking attacks.

Disallow mounting of unused tokens with AccuKnox®

While access to the service account token can be restricted at runtime using KubeArmor (https://docs.kubearmor.io/kubearmor/use-cases/use-cases#block-access-to-k8s-service-account-token), it is better to stop auto-mounting of service account tokens if the container does not access it.

How does it work?

Discovery Engine works as a plug-in with KubeArmor. Discovery Engine acquires system logs from KubeArmor Relay which are processed and analyzed to check whether there are any accesses to the service account token mounted into the container.

Discovered policy uses labels to match on the pod or the pod controller. Discovery Engine stores these policies internally inside a database which are updated and deleted as per application behavior. It is suggested to run applications in the pre-production stage, trying to hit maximum code points so that KubeArmor has maximum visibility into application behavior and highly accurate policies can be generated.

These discovered policies will then prevent bad configurations at the admission time only, enhancing the security posture and not leaving it for runtime pre or post attack mitigation.

Note: Only Kyverno policies are recommended at present. The framework can be extended to generate OPA (Open Policy Agent) Rego policies or KubeWarden policies.

Demo Time ⏳
I. Download and install karmor cli-tool
$ curl -sfL http://get.kubearmor.io/ | sudo sh -s -- -b /usr/local/bin
II. Install KubeArmor
$ karmor install
III. Install Discovery Engine
$ kubectl apply -f
https://raw.githubusercontent.com/kubearmor/discovery-engine/dev/deployments/k8s/deployment.yaml
IV. Create an nginx deployment
$ kubectl create deploy nginx --image=nginx --replicas=1
V. Let’s try to recommend policies using karmorCLI, nginx image does not access the service account token.

$ karmor recommend -p KyvernoPolicy
...
created policy out/default-nginx/nginx-restrict-automount-sa-token.yaml ...
...
  Deployment              | default/nginx      
  Container               | nginx              
  OS                      |                    
  Arch                    |                    
  Distro                  |                    
  Output Directory        | out/default-nginx  
  policy-template version | v0.1.9        

Actual Policy

apiVersion: kyverno.io/v1
kind: Policy
metadata:
  annotations:
    policies.kyverno.io/minversion: 1.6.0
    recommended-policies.kubearmor.io/description: Don't mount service account token
      when it is not needed
    recommended-policies.kubearmor.io/description-detailed: If the Service Account
      Token is not used by a pod, then it should not be automounted. Service account
      token provide access to the kubeapi-server which potentially increases the surface
      area of attack.
    recommended-policies.kubearmor.io/tags: AUTOMOUNT_SERVICE_ACCOUNT
    recommended-policies.kubearmor.io/title: Restrict Auto-Mount of Service Account
      Tokens
  name: nginx-restrict-automount-sa-token
  namespace: default
spec:
  background: true
  rules:
  - match:
      any:
      - resources:
          kinds:
          - Deployment
    name: validate-automountServiceAccountToken
    preconditions:
      all:
      - key: ''
        operator: NotEquals
        value: DELETE
        - key: ''
        operator: Equals
        value: nginx
    validate:
      message: Auto-mounting of Service Account tokens is not allowed.
      pattern:
        spec:
          template:
            spec:
              automountServiceAccountToken: "false"
  validationFailureAction: Audit

If in the future, the application behavior changes and it starts accessing the SA token, the above mentioned policy won’t be recommended.

To verify this:

Exec into the pod to cat the service account token

$ POD=$(kubectl get pod -l app=nginx -o jsonpath="{.items[0].metadata.name}")
$ kubectl exec $POD -it -- cat /var/run/secrets/kubernetes.io/serviceaccount/token

Recommend the policies again using karmor CLI

$ karmor recommend -p KyvernoPolicy
...
  Deployment              | default/nginx      
  Container               | nginx              
  OS                      |                    
  Arch                    |                    
  Distro                  |                    
  Output Directory        | out/default-nginx  
  policy-template version | v0.1.9

Policy for restricting service account token is not generated for deployment nginx.

In summary, dynamic Kyverno admission controller policy generation with KubeArmor can greatly enhance the security posture of a Kubernetes cluster. Leveraging KubeArmor’s ability to automatically generate fine-grained policies based on the observed behavior of workloads, organizations can improve the accuracy and efficiency of their run-time security policies and embark on a path of Zero Trust Security.