AWS VPC CNI 1.7: Pod Security Groups and Enhanced Security

Introduction

On November 15, 2020, AWS released VPC CNI Plugin version 1.7, introducing pod-level security groups—one of the most requested features for EKS. This release enables fine-grained network security at the pod level, allowing different pods to have different security group rules, a significant improvement over node-level security groups.

Security groups are fundamental to AWS networking security, but until 1.7, they could only be applied at the node level. This meant all pods on a node shared the same security group rules, limiting the ability to implement fine-grained network policies. VPC CNI 1.7 changed this by enabling security groups per pod.

Key Features

1. Pod Security Groups: Assign security groups directly to pods using annotations.
2. Automatic IP Range Detection: Automatically detects new IP address ranges added to existing VPCs.
3. Enhanced Security: Better integration with AWS security features.
4. Backward Compatibility: Works with existing node-level security groups.

Pod Security Groups

Assign security groups to pods using annotations:

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  annotations:
    vpc.amazonaws.com/security-groups: sg-12345678,sg-87654321
spec:
  containers:
  - name: app
    image: nginx

Benefits:

• Fine-Grained Control: Different pods can have different security group rules.
• Micro-Segmentation: Implement network segmentation at the pod level.
• Compliance: Meet compliance requirements with pod-level network policies.
• Flexibility: Mix node-level and pod-level security groups as needed.

Automatic IP Range Detection

VPC CNI 1.7 automatically detects when new IP address ranges are added to existing VPCs:

• No Restart Required: Detects new ranges without CNI daemon restart.
• Automatic Configuration: Automatically configures new ranges for pod IP allocation.
• Simplified Operations: Reduces manual configuration when expanding VPC subnets.

Getting Started

VPC CNI 1.7 became the default for new EKS clusters. To upgrade:

kubectl apply -f https://raw.githubusercontent.com/aws/amazon-vpc-cni-k8s/release-1.7/config/master/aws-k8s-cni.yaml

Enable pod security groups (requires IAM permissions):

apiVersion: v1
kind: ConfigMap
metadata:
  name: aws-vpc-cni
  namespace: kube-system
data:
  ENABLE_POD_ENI: "true"

Security Group Configuration

Using Pod Annotations

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  template:
    metadata:
      annotations:
        vpc.amazonaws.com/security-groups: sg-web
    spec:
      containers:
      - name: app
        image: nginx

Using Service Account Annotations

apiVersion: v1
kind: ServiceAccount
metadata:
  name: my-service-account
  annotations:
    vpc.amazonaws.com/security-groups: sg-app

Use Cases

• Micro-Segmentation: Implement network segmentation at the pod level for security.
• Multi-Tenant Clusters: Different tenants can have different security group rules.
• Compliance: Meet compliance requirements with pod-level network policies.
• Application Isolation: Isolate applications using different security groups.

Comparison: Node vs Pod Security Groups

Operational Considerations

• IAM Permissions: Pod security groups require additional IAM permissions for ENI management.
• ENI Limits: Each pod with a security group consumes an ENI; understand instance type limits.
• Performance: Pod security groups may have slight performance overhead compared to node-level.
• Cost: Additional ENIs may increase costs; monitor ENI usage.

Common Patterns

• Application Tiers: Use different security groups for frontend, backend, and database pods.
• Tenant Isolation: Assign different security groups to different tenants in multi-tenant clusters.
• Compliance Zones: Use security groups to implement compliance zones (PCI, HIPAA, etc.).
• Development/Production: Different security groups for dev and prod environments.

Limitations

• ENI Consumption: Pod security groups consume ENIs; limited by instance type ENI limits.
• Performance Overhead: Slight performance overhead compared to node-level security groups.
• IAM Requirements: Requires additional IAM permissions for ENI management.
• Complexity: More complex configuration and troubleshooting compared to node-level.

Looking Ahead

VPC CNI 1.7 set the foundation for:

• Automatic Subnet Discovery: Dynamic subnet discovery and IP allocation.
• Enhanced Security: Additional security features and integrations.
• Performance Improvements: Optimizations for pod security group operations.
• Advanced Features: More sophisticated security group management capabilities.

Summary

AWS VPC CNI 1.7 introduced pod-level security groups, enabling fine-grained network security that was previously only possible at the node level. This feature, combined with automatic IP range detection, made VPC CNI more suitable for security-sensitive and multi-tenant environments.