EKS Autoscaling

Autoscaling on EKS involves scaling at multiple levels: pods (HPA/VPA), nodes (Cluster Autoscaler/Karpenter), and cluster capacity. EKS provides several autoscaling solutions that work together to ensure your applications have the right resources at the right time while optimizing costs.

Autoscaling Overview

EKS autoscaling operates at different levels:

graph TB subgraph pod_level[Pod Level] HPA[Horizontal Pod Autoscaler] --> SCALE_PODS[Scale Pod Replicas] VPA[Vertical Pod Autoscaler] --> ADJUST_RESOURCES[Adjust Pod Resources] end subgraph node_level[Node Level] CA[Cluster Autoscaler] --> SCALE_NODES[Scale Node Groups] KARPENTER[Karpenter] --> PROVISION_NODES[Provision Nodes] end subgraph cluster_level[Cluster Level] SCALE_PODS --> TRIGGER_NODES[Trigger Node Scaling] ADJUST_RESOURCES --> OPTIMIZE[Optimize Resource Usage] SCALE_NODES --> ADD_NODES[Add/Remove Nodes] PROVISION_NODES --> ADD_NODES end style HPA fill:#e1f5ff style CA fill:#fff4e1 style KARPENTER fill:#e8f5e9

Cluster Autoscaler

Cluster Autoscaler automatically adjusts the size of node groups based on pod scheduling demands. When pods can’t be scheduled due to insufficient resources, it adds nodes. When nodes are underutilized, it removes them.

How Cluster Autoscaler Works

graph LR A[Pod Pending] --> B{Resources Available?} B -->|No| C[Cluster Autoscaler Detects] C --> D[Increase Node Group Desired Capacity] D --> E[New Nodes Created] E --> F[Pods Scheduled] G[Node Underutilized] --> H{Can Pods Move?} H -->|Yes| I[Cluster Autoscaler Detects] I --> J[Drain Node] J --> K[Decrease Desired Capacity] K --> L[Node Terminated] style A fill:#e1f5ff style E fill:#fff4e1 style L fill:#e8f5e9

Installation

Using Helm:

# Add Helm repository
helm repo add autoscaler https://kubernetes.github.io/autoscaler
helm repo update

# Install Cluster Autoscaler
helm install cluster-autoscaler autoscaler/cluster-autoscaler \
  --namespace kube-system \
  --set autoDiscovery.clusterName=my-cluster \
  --set aws.region=us-west-2 \
  --set rbac.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::123456789012:role/ClusterAutoscalerRole

Using YAML:

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cluster-autoscaler
  namespace: kube-system
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/ClusterAutoscalerRole
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cluster-autoscaler
  namespace: kube-system
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cluster-autoscaler
  template:
    metadata:
      labels:
        app: cluster-autoscaler
    spec:
      serviceAccountName: cluster-autoscaler
      containers:
      - image: k8s.gcr.io/autoscaling/cluster-autoscaler:v1.28.0
        name: cluster-autoscaler
        resources:
          limits:
            cpu: 100m
            memory: 300Mi
          requests:
            cpu: 100m
            memory: 300Mi
        command:
        - ./cluster-autoscaler
        - --v=4
        - --stderrthreshold=info
        - --cloud-provider=aws
        - --skip-nodes-with-local-storage=false
        - --expander=least-waste
        - --node-group-auto-discovery=asg:tag=k8s.io/cluster-autoscaler/enabled,k8s.io/cluster-autoscaler/my-cluster
        env:
        - name: AWS_REGION
          value: us-west-2

IAM Permissions

Cluster Autoscaler needs IAM permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:DescribeAutoScalingInstances",
        "autoscaling:DescribeLaunchConfigurations",
        "autoscaling:DescribeScalingActivities",
        "autoscaling:DescribeTags",
        "ec2:DescribeInstanceTypes",
        "ec2:DescribeLaunchTemplateVersions"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:SetDesiredCapacity",
        "autoscaling:TerminateInstanceInAutoScalingGroup",
        "ec2:DescribeImages",
        "ec2:GetInstanceTypesFromInstanceRequirements",
        "eks:DescribeNodegroup"
      ],
      "Resource": "*"
    }
  ]
}

Configuration

Node Group Tagging:

Tag node groups for autoscaling:

# Tag managed node group
aws eks update-nodegroup-config \
  --cluster-name my-cluster \
  --nodegroup-name general-workers \
  --labels addOrUpdateLabels={k8s.io/cluster-autoscaler/enabled=true,k8s.io/cluster-autoscaler/my-cluster=owned}

# Tag Auto Scaling Group (self-managed)
aws autoscaling create-or-update-tags \
  --tags \
    ResourceId=eks-nodes,ResourceType=auto-scaling-group,Key=k8s.io/cluster-autoscaler/enabled,Value=true,PropagateAtLaunch=false \
    ResourceId=eks-nodes,ResourceType=auto-scaling-group,Key=k8s.io/cluster-autoscaler/my-cluster,Value=owned,PropagateAtLaunch=false

Scaling Parameters:

command:
- ./cluster-autoscaler
- --nodes=1:10:eks-general-workers  # min:max:nodegroup
- --scale-down-delay-after-add=10m
- --scale-down-unneeded-time=10m
- --scale-down-utilization-threshold=0.5
- --skip-nodes-with-local-storage=false
- --skip-nodes-with-system-pods=false

Karpenter

Karpenter is a next-generation node autoscaler that provisions nodes faster and more efficiently than Cluster Autoscaler. It evaluates pod scheduling requirements and provisions the right nodes at the right time.

Karpenter vs Cluster Autoscaler

Feature	Cluster Autoscaler	Karpenter
Scaling Speed	Minutes	Seconds
Node Selection	From existing node groups	Optimal instance types
Bin Packing	Basic	Advanced
Spot Integration	Manual	Automatic
Cost Optimization	Limited	Advanced

Installation

Using Helm:

# Add Helm repository
helm repo add karpenter https://charts.karpenter.sh
helm repo update

# Install Karpenter
helm install karpenter karpenter/karpenter \
  --namespace karpenter \
  --create-namespace \
  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=arn:aws:iam::123456789012:role/KarpenterControllerRole \
  --set settings.clusterName=my-cluster \
  --set settings.defaultInstanceProfile=KarpenterNodeInstanceProfile \
  --set settings.interruptionQueue=my-cluster

Create Node IAM Role:

# Create node instance profile
eksctl create iamserviceaccount \
  --name karpenter \
  --namespace karpenter \
  --cluster my-cluster \
  --role-name KarpenterControllerRole \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly \
  --attach-policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore \
  --approve

Node Provisioner

Define how Karpenter provisions nodes:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: default
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot", "on-demand"]
    - key: kubernetes.io/arch
      operator: In
      values: ["amd64"]
    - key: kubernetes.io/os
      operator: In
      values: ["linux"]
    - key: karpenter.k8s.aws/instance-family
      operator: In
      values: ["t3", "t4g", "m5", "m6i", "c5", "c6i"]
    - key: karpenter.k8s.aws/instance-size
      operator: In
      values: ["small", "medium", "large", "xlarge", "2xlarge"]
  
  limits:
    resources:
      cpu: 1000
      memory: 1000Gi
  
  provider:
    subnetSelector:
      karpenter.sh/discovery: my-cluster
    securityGroupSelector:
      karpenter.sh/discovery: my-cluster
    instanceProfile: KarpenterNodeInstanceProfile
  
  taints:
    - key: example.com/special-taint
      value: "true"
      effect: NoSchedule
  
  labels:
    environment: production
  
  ttlSecondsAfterEmpty: 30
  ttlSecondsUntilExpired: 2592000

Key Features:

Requirements - Instance type and capacity type constraints
Limits - Maximum resources to provision
Provider - Subnet and security group selection
TTL - Time to live for empty/expired nodes

Spot Instance Integration

Karpenter automatically uses spot instances:

spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot", "on-demand"]  # Prefer spot, fallback to on-demand

Spot Interruption Handling:

Karpenter monitors spot interruption notices and gracefully drains nodes:

apiVersion: karpenter.sh/v1alpha5
kind: Provisioner
metadata:
  name: spot-provisioner
spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot"]
  taints:
    - key: spot
      value: "true"
      effect: NoSchedule

Horizontal Pod Autoscaler (HPA)

HPA automatically scales the number of pod replicas based on observed metrics like CPU, memory, or custom metrics.

Basic HPA

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 2
  maxReplicas: 10
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70
  - type: Resource
    resource:
      name: memory
      target:
        type: Utilization
        averageUtilization: 80
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 50
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 15
      - type: Pods
        value: 4
        periodSeconds: 15
      selectPolicy: Max

HPA Behavior:

Scale Down - Conservative scaling down to prevent thrashing
Scale Up - Aggressive scaling up to handle traffic spikes
Stabilization Window - Time to wait before scaling

Custom Metrics HPA

Scale based on custom application metrics:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-hpa-custom
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "100"

Requires:

Metrics Server (for resource metrics)
Prometheus Adapter (for custom metrics)
External Metrics API

Vertical Pod Autoscaler (VPA)

VPA automatically adjusts CPU and memory requests and limits for pods based on historical usage.

Installation

# Clone VPA repository
git clone https://github.com/kubernetes/autoscaler.git
cd autoscaler/vertical-pod-autoscaler/

# Install VPA
./hack/vpa-up.sh

VPA Configuration

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: web-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web
  updatePolicy:
    updateMode: "Auto"  # Auto, Off, Initial, Recreate
  resourcePolicy:
    containerPolicies:
    - containerName: app
      minAllowed:
        cpu: 100m
        memory: 128Mi
      maxAllowed:
        cpu: 2
        memory: 4Gi
      controlledResources: ["cpu", "memory"]

Update Modes:

Auto - Automatically update pod resources (requires pod restart)
Off - Only provide recommendations
Initial - Set resources on pod creation only
Recreate - Recreate pods with new resources

Note: VPA and HPA should not be used together for the same resource (CPU/memory). Use VPA for resource optimization, HPA for replica scaling.

Custom Metrics and Scaling Policies

Prometheus Adapter

Install Prometheus Adapter for custom metrics:

# Add Helm repository
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

# Install Prometheus Adapter
helm install prometheus-adapter prometheus-community/prometheus-adapter \
  --namespace monitoring \
  --set prometheus.url=http://prometheus-server.monitoring.svc \
  --set prometheus.port=80

External Metrics

Scale based on external metrics (e.g., SQS queue length):

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: queue-processor-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: queue-processor
  minReplicas: 1
  maxReplicas: 10
  metrics:
  - type: External
    external:
      metric:
        name: sqs_queue_length
        selector:
          matchLabels:
            queue: my-queue
      target:
        type: AverageValue
        averageValue: "10"

Spot Instance Integration

Cluster Autoscaler with Spot

Configure node groups for spot instances:

managedNodeGroups:
  - name: spot-workers
    instanceTypes:
      - t3.medium
      - t3.large
      - t3.xlarge
    capacityType: SPOT
    minSize: 0
    maxSize: 20
    labels:
      capacity-type: spot
    taints:
      - key: spot
        value: "true"
        effect: NoSchedule

Pod Tolerations:

apiVersion: v1
kind: Pod
metadata:
  name: spot-workload
spec:
  tolerations:
  - key: spot
    value: "true"
    effect: NoSchedule
  containers:
  - name: app
    image: my-app:latest

Karpenter Spot Integration

Karpenter automatically handles spot instances:

spec:
  requirements:
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot", "on-demand"]

Cost Optimization Strategies

Right-Sizing

Use VPA to right-size pod resources:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: optimize-resources
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Auto"

Spot Instances

Use spot instances for fault-tolerant workloads:

Up to 90% cost savings
Automatic interruption handling
Multiple instance types for availability
Fallback to on-demand

Scheduled Scaling

Scale down during off-hours:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: scale-down
spec:
  schedule: "0 20 * * *"  # 8 PM daily
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: kubectl
            image: bitnami/kubectl:latest
            command:
            - kubectl
            - scale
            - deployment
            - web
            - --replicas=1

Scaling Best Practices

Set Appropriate Limits - Configure min/max replicas and node limits
Use Multiple Metrics - Combine CPU, memory, and custom metrics
Configure Behavior - Tune scale-up and scale-down policies
Monitor Scaling - Watch HPA and autoscaler behavior
Test Scaling - Verify autoscaling works before production
Use Spot Instances - For cost optimization where appropriate
Right-Size Resources - Use VPA to optimize resource requests
Plan for Spikes - Configure aggressive scale-up for traffic spikes
Prevent Thrashing - Use stabilization windows and conservative scale-down
Combine Solutions - Use HPA for pods, Cluster Autoscaler/Karpenter for nodes

Common Issues

HPA Not Scaling

Problem: HPA not scaling pods

Solutions:

Verify Metrics Server is running
Check HPA target metrics
Verify resource requests are set
Check HPA status and events

Cluster Autoscaler Not Scaling Nodes

Problem: Nodes not being added

Solutions:

Verify node group tags
Check IAM permissions
Verify Auto Scaling Group limits
Check Cluster Autoscaler logs
Verify pods are unschedulable

Karpenter Not Provisioning

Problem: Karpenter not creating nodes

Solutions:

Check Provisioner configuration
Verify subnet and security group selectors
Check IAM permissions
Review Karpenter logs
Verify pod requirements

EKS Autoscaling

Autoscaling Overview

Cluster Autoscaler

How Cluster Autoscaler Works

Installation

IAM Permissions

Configuration

Karpenter

Karpenter vs Cluster Autoscaler

Installation

Node Provisioner

Spot Instance Integration

Horizontal Pod Autoscaler (HPA)

Basic HPA

Custom Metrics HPA

Vertical Pod Autoscaler (VPA)

Installation

VPA Configuration

Custom Metrics and Scaling Policies

Prometheus Adapter

External Metrics

Spot Instance Integration

Cluster Autoscaler with Spot

Karpenter Spot Integration

Cost Optimization Strategies

Right-Sizing

Spot Instances

Scheduled Scaling

Scaling Best Practices

Common Issues

HPA Not Scaling

Cluster Autoscaler Not Scaling Nodes

Karpenter Not Provisioning

See Also