A Deep Dive into Pod and Container Security

To keep Kubernetes clusters safe, it’s crucial to prevent unwanted changes that could mess things up or jeopardize security. One way to control pod security was through something called Pod Security Policy (PSP). PSPs (complex and could be challenging to set up correctly.) were like rules that pods had to follow before being allowed into the cluster. But, starting from Kubernetes version 1.21, PSPs are no longer supported, and in version 1.25, they’ve been removed altogether and replaced with Pod Security Admission controller.

Pod Security Admission Controller: A Simplified API

The Pod Security Admission (PSA) controller introduces a simplified API for securing pods, aligning with the Pod Security Standards (PSS). This transition offers a more accessible and standardized approach to ensuring the security of containerized workloads.

Understanding Pod Security Standards (PSS) and PSA Modes

According to the Kubernetes PSS documentation, the PSS “define three different policies to broadly cover the security spectrum. These policies are cumulative and range from highly-permissive to highly-restrictive.”

Pod Security admission places requirements on a Pod’s Security Context and other related fields according to the three Pod Security Standards levels :

• Privileged: Unrestricted policy, providing the widest possible level of permissions. This policy allows for known privilege escalations.
• Baseline: Minimally restrictive policy which prevents known privilege escalations. Allows the default (minimally specified) pod configuration.
• Restricted: Heavily restricted policy, following current pod hardening best practices.

The PSA admission controller follows the rules from the PSS policies in three different ways, and here they are:

• Enforce: If a policy is violated, the pod gets rejected, meaning it won’t be allowed.
• Audit: If a policy is violated, it won’t stop the pod, but it leaves a note in the audit log for later review.
• Warn: If a policy is violated, a warning is issued to the user, but the pod is still allowed to proceed.

A Word of Caution: ‘WARNING’

Be careful when turning on Pod Security Admission control in existing clusters because it might block your work if you’re not ready. It’s a good idea to begin with the warn and audit modes to make sure your rules are working correctly.

Pod Security Admission labels for namespaces

From Kubernetes version 1.23 or newer version Pod security Admission is feature enabled, Once feature gate is enabled, you can set up namespaces to determine the admission control mode for pod security in each namespace. Kubernetes provides a set of labels that allow you to specify which predefined Pod Security Standard level you intend to use for a particular namespace.The chosen label dictates how the control plane reacts if it identifies a potential violation.

A namespace can configure any or all modes, or even set a different level for different modes.

For each mode, there are two labels that determine the policy used:

 1# The per-mode level label indicates which policy level to apply for the mode.
 2#
 3# MODE must be one of `enforce`, `audit`, or `warn`.
 4# LEVEL must be one of `privileged`, `baseline`, or `restricted`.
 5pod-security.kubernetes.io/<MODE>: <LEVEL>
 6
 7# Optional: per-mode version label that can be used to pin the policy to the
 8# version that shipped with a given Kubernetes minor version (for example v1.28).
 9#
10# MODE must be one of `enforce`, `audit`, or `warn`.
11# VERSION must be a valid Kubernetes minor version, or `latest`.
12pod-security.kubernetes.io/<MODE>-version: <VERSION>

Test Setup

1. Annotate the namespace with the required PSS and PSA for our example we are annotating with restricted pod security standards.

 1apiVersion: v1
 2kind: Namespace
 3metadata:
 4  creationTimestamp: "2023-11-18T08:36:20Z"
 5  labels:
 6    kubernetes.io/metadata.name: default
 7    pod-security.kubernetes.io/audit: restricted
 8    pod-security.kubernetes.io/enforce: restricted
 9    pod-security.kubernetes.io/enforce-version: latest
10    pod-security.kubernetes.io/warn: restricted
11  name: default
12  resourceVersion: "30414"
13  uid: 948e2263-e72f-4751-961e-ebbe4591651a
14spec:
15  finalizers:
16  - kubernetes
17status:
18  phase: Active

2. Pod yaml file with bad securitycontext defined at container level

 1apiVersion: v1
 2kind: Pod
 3metadata:
 4  name: rootuser
 5spec:
 6  containers:
 7  - name: my-container
 8    image: nginx
 9    securityContext:
10      runAsUser: 0

3. When you apply the above yaml file using kubectl create -f pod.yaml, you will get below error message as it violates the restricted pod security standards.

1
2root@master-k8s:/home/ubuntu# kubectl create -f pod.yaml 
3Error from server (Forbidden): error when creating "pod.yaml": pods "rootuser" is forbidden: violates PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "my-container" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "my-container" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "my-container" must set securityContext.runAsNonRoot=true), runAsUser=0 (container "my-container" must not set runAsUser=0), seccompProfile (pod or container "my-container" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
4root@master-k8s:/home/ubuntu# 

Now let’s add the required security context to the pod yaml file as required by the restricted pod security standards

 1
 2apiVersion: v1
 3kind: Pod
 4metadata:
 5  name: rootuser
 6spec:
 7  containers:
 8  - name: goodcontainer
 9    image: nginx
10    ports:
11      - containerPort: 8080
12    securityContext:
13      allowPrivilegeEscalation: false
14      runAsUser: 1000
15      readOnlyRootFilesystem: true
16      runAsNonRoot: true
17      capabilities:
18        drop:
19         - ALL
20      seccompProfile:
21        type: "RuntimeDefault"

Conclusion

By using these PSS and PSA, Kubernetes users can make sure their pods and containers are secure and ready to handle the complexities of modern container management. It’s like giving your Kubernetes clusters a strong shield to handle whatever comes their way