Horizontal Pod Autoscaler setup

Horizontal scale means more replicas, not a bigger pod. An HPA watches a metric and adjusts the replica count to keep that metric near a target. For CPU-bound embedding, CPU utilization is the right signal.

k8s/hpa.yaml

yaml

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: enterprise-rag
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: enterprise-rag
  minReplicas: 2
  maxReplicas: 8
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300

CPU utilization is computed against the pod request, not the limit. With a 250m request and a 70 percent target, each pod tries to stay around 175m. Cross that and the HPA adds replicas up to maxReplicas.

terminal

bash

# The HPA needs the metrics server to read CPU utilization
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

# Apply the HPA
kubectl apply -f k8s/hpa.yaml

# Inspect
kubectl get hpa enterprise-rag

kind and minikube ship without the metrics server. Install it once and every HPA in the cluster can read CPU and memory metrics. Managed clusters usually have it preinstalled.

HPA scaling triggers

How the HPA loop reads metrics and decides whether to add or remove replicas.

Quiz: Quiz

Loading practice…