Kubernetes Security Best Practices for Production Workloads
Kubernetes has become the de facto standard for container orchestration, but its power comes with complexity -- and complexity is where security vulnerabilities hide. A default Kubernetes installation is not secure. It is designed for flexibility, which means every security control must be deliberately configured.
Our team at Zindagi Technologies has hardened Kubernetes clusters for defense organizations, financial services, and SaaS platforms. This post covers the security practices that matter most in production, organized from foundational controls to advanced runtime protection.
The Kubernetes Attack Surface
Before hardening, understand what you are defending against. The Kubernetes attack surface spans multiple layers:
- Control plane: The API server, etcd, scheduler, and controller manager are high-value targets. Compromise here means full cluster takeover.
- Node level: Kubelet, container runtime, and the host OS. A container escape gives an attacker node-level access.
- Network: Pod-to-pod communication is unencrypted and unrestricted by default. Any pod can talk to any other pod.
- Supply chain: Container images, Helm charts, and third-party operators can introduce vulnerabilities or malicious code.
- Workload configuration: Overprivileged pods, missing resource limits, and exposed secrets create exploitable conditions.
RBAC: The Foundation of Cluster Security
Role-Based Access Control is your first line of defense. Kubernetes RBAC controls who can do what within the cluster.
Principle of Least Privilege
Every service account, user, and CI/CD pipeline should have the minimum permissions required. Avoid cluster-admin bindings except for break-glass scenarios.
Audit Default Service Accounts
Every namespace gets a default service account that is automatically mounted into pods. In most cases, your application does not need Kubernetes API access. Disable auto-mounting:
apiVersion: v1
kind: ServiceAccount
metadata:
name: default
automountServiceAccountToken: false
Create Purpose-Specific Service Accounts
Instead of using the default service account, create dedicated service accounts with specific role bindings for each workload:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: production
name: configmap-reader
rules:
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["get", "list"]
Regularly Audit RBAC Bindings
Use tools like kubectl auth can-i --list --as=system:serviceaccount:default:default to audit permissions. Consider deploying rbac-lookup or KubiScan for comprehensive audits.
Network Policies: Micro-Segmentation for Pods
By default, every pod can communicate with every other pod in the cluster. Network Policies are Kubernetes-native micro-segmentation.
Default Deny All Traffic
Start with a default deny policy in every namespace, then explicitly allow only required traffic:
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-all
namespace: production
spec:
podSelector: \{\}
policyTypes:
- Ingress
- Egress
Allow Only Required Communication Paths
Define explicit ingress and egress rules for each workload. For example, allow your web frontend to communicate with the API service, and the API service to communicate with the database -- but not the frontend directly to the database.
Use a CNI That Supports Network Policies
Not all CNI plugins enforce Network Policies. Calico, Cilium, and Weave Net support them. Flannel does not. If you are using a managed Kubernetes service (EKS, AKS, GKE), verify that your CNI plugin enforces policies.
Consider Cilium for Advanced Use Cases
Cilium provides Layer 7 (HTTP/gRPC) aware network policies, DNS-aware egress filtering, and transparent encryption via WireGuard. For production workloads handling sensitive data, this level of control is worth the operational overhead.
Pod Security: Restricting Container Privileges
Containers share the host kernel. A misconfigured pod can escalate privileges, mount host filesystems, or disable security controls.
Pod Security Standards
Kubernetes defines three built-in security profiles: Privileged, Baseline, and Restricted. Enforce the Restricted profile for production workloads:
apiVersion: v1
kind: Namespace
metadata:
name: production
labels:
pod-security.kubernetes.io/enforce: restricted
pod-security.kubernetes.io/audit: restricted
pod-security.kubernetes.io/warn: restricted
Security Context Essentials
Every production pod should specify a security context:
runAsNonRoot: true-- Never run containers as rootreadOnlyRootFilesystem: true-- Prevent filesystem modificationsallowPrivilegeEscalation: false-- Block privilege escalationcapabilities.drop: ["ALL"]-- Drop all Linux capabilities, add back only what is neededseccompProfile.type: RuntimeDefault-- Apply the default seccomp profile
Resource Limits
Always set CPU and memory limits. Without them, a compromised or buggy container can starve other workloads (denial of service) or consume resources for cryptomining:
resources:
limits:
cpu: "500m"
memory: "256Mi"
requests:
cpu: "100m"
memory: "128Mi"
Image Security: Trust Your Supply Chain
The container image is your software supply chain. A compromised image defeats every runtime control.
Use Minimal Base Images
Distroless images, Alpine Linux, or scratch-based images reduce the attack surface. A full Ubuntu image includes hundreds of packages your application does not need -- each one is a potential vulnerability.
Scan Images in CI/CD
Integrate vulnerability scanning into your build pipeline. Trivy, Grype, and Snyk Container are popular choices. Block deployments that contain critical or high-severity CVEs. Configure your scanner to fail builds, not just report:
trivy image --exit-code 1 --severity HIGH,CRITICAL myapp:latest
Sign and Verify Images
Use Cosign (from the Sigstore project) to sign images in your CI/CD pipeline and verify signatures before admission. This prevents tampered or unauthorized images from running.
Use an Admission Controller
Deploy an admission controller (OPA Gatekeeper, Kyverno, or the built-in ValidatingAdmissionPolicy) to enforce policies at deploy time:
- Only allow images from trusted registries
- Require image digests instead of mutable tags
- Enforce security context requirements
- Block privileged containers
Secrets Management
Kubernetes Secrets are base64-encoded, not encrypted. Anyone with API access to the namespace can read them.
Encrypt etcd at Rest
Configure the API server to encrypt Secrets stored in etcd using AES-CBC or AES-GCM encryption. This is a one-time configuration change that protects against etcd data breaches.
Use External Secrets Management
For production, integrate with a dedicated secrets management solution -- HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault. The External Secrets Operator syncs secrets from external vaults into Kubernetes Secrets automatically.
Rotate Secrets Regularly
Automate secret rotation. If a secret is compromised, the blast radius is limited to the rotation interval. Vault's dynamic secrets are ideal -- they generate short-lived credentials on demand.
Runtime Security and Monitoring
Prevention is essential, but detection catches what prevention misses.
Deploy a Runtime Security Tool
Falco, Sysdig Secure, or Aqua Security monitor system calls and container behavior at runtime. They detect anomalies like:
- Shell spawned inside a container
- Unexpected network connections
- File modifications in a read-only filesystem
- Privilege escalation attempts
Centralize Audit Logs
Enable Kubernetes audit logging and ship logs to a centralized SIEM. Audit logs capture every API request, providing a complete trail of who did what in the cluster.
Monitor for Drift
Container images should be immutable. Any modification to a running container (new binaries, changed configurations) indicates compromise. Runtime security tools can alert on drift from the original image.
Cluster Hardening Checklist
For quick reference, here are the critical controls in priority order:
- Enable RBAC and enforce least privilege
- Deploy Network Policies with default deny
- Enforce Pod Security Standards (Restricted)
- Scan images in CI/CD and block vulnerable deployments
- Encrypt etcd at rest
- Disable auto-mounting of service account tokens
- Use an admission controller for policy enforcement
- Set resource limits on all workloads
- Deploy runtime security monitoring
- Enable and centralize audit logging
- Rotate secrets and use external secrets management
- Keep Kubernetes and node OS patched and updated
Final Thoughts
Kubernetes security is not a one-time configuration. It requires continuous attention -- updating policies as workloads change, patching components, monitoring for anomalies, and evolving your controls as the threat landscape shifts.
The practices outlined here represent what we consider the minimum standard for production workloads. Organizations handling highly sensitive data (defense, healthcare, financial services) should layer additional controls including hardware-backed attestation, service mesh with mTLS, and formal compliance frameworks like CIS Kubernetes Benchmark.
At Zindagi Technologies, we help organizations design, deploy, and operate secure Kubernetes infrastructure. Whether you are migrating your first workload or hardening an existing cluster, our DevSecOps team brings the expertise to do it right.