Scale Kubernetes GPU workloads from real hardware metrics. No DCGM. No PromQL. Optional Prometheus metrics built in.

A KEDA External Scaler that reads NVIDIA GPU metrics directly from NVML C-bindings and autoscales your vLLM, Triton, and custom inference deployments — including scale-to-zero.

Kubernetes HPA watches CPU and memory. It can't see GPU utilization. Your vLLM pod shows 8% CPU while the GPU is at 100%.

The usual fix is dcgm-exporter → Prometheus → KEDA, but that's 5 components and 15-30s of latency.

This project reads GPU metrics directly from NVML and serves them to KEDA over gRPC. 2 components, 2-4 second latency.

Putting GPU support inside KEDA core doesn't work:

1. CGO Constraint: NVIDIA's Go bindings (go-nvml) require CGO_ENABLED=1. KEDA builds with CGO_ENABLED=0.
2. Node-Level Hardware Access: The KEDA operator runs as a central pod. NVML requires local GPU device access via libnvidia-ml.so, which only a DaemonSet on GPU nodes can provide.
3. Independent Release Cycle: Ship GPU scaling improvements without waiting for KEDA release cycles.

This design is documented in KEDA issue #7538.

1. DaemonSet — Runs on nodes labeled with nvidia.com/gpu.present: "true".
2. NVML Bindings — Directly reads Streaming Multiprocessor (SM) utilization and Frame Buffer Memory via go-nvml C-bindings.
3. gRPC Interface — Implements externalscaler.ExternalScalerServer (IsActive, StreamIsActive, GetMetricSpec, GetMetrics) to natively integrate with the central KEDA operator.
4. ScaledObject Trigger — Kubernetes deployments scale up/down (including to zero) based on GPU thresholds defined in the ScaledObject.

Instead of configuring raw metric thresholds, use a profile optimized for your workload:

• A Kubernetes cluster (e.g., OKE, GKE, EKS, AKS) with NVIDIA GPU worker nodes
• KEDA v2.10+ installed in the cluster
• NVIDIA GPU drivers and Device Plugin installed

Deploy the DaemonSet and gRPC service into your cluster. (Ensure KEDA is already installed.)

kubectl apply -f deploy/manifests.yaml

This deploys a DaemonSet that runs on every GPU node in your cluster, plus a ClusterIP Service for KEDA to discover it.

Or use Helm:

helm install keda-gpu-scaler deploy/helm/keda-gpu-scaler \
  --namespace keda \
  --set nodeSelector."nvidia\.com/gpu\.present"=true

Create a ScaledObject pointing to the external scaler service:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: vllm-inference-scaler
  namespace: ai-workloads
spec:
  scaleTargetRef:
    name: vllm-deepseek-deployment
  minReplicaCount: 1
  maxReplicaCount: 50
  triggers:
    - type: external
      metadata:
        scalerAddress: "keda-gpu-scaler.keda.svc.cluster.local:6000"
        targetGpuUtilization: "80"

Or use a pre-built profile:

triggers:
  - type: external
    metadata:
      scalerAddress: "keda-gpu-scaler.keda.svc.cluster.local:6000"
      profile: "vllm-inference"

Override any profile default or use raw GPU metrics directly:

triggers:
  - type: external
    metadata:
      scalerAddress: "keda-gpu-scaler.keda.svc.cluster.local:6000"
      metricType: "gpu_utilization"
      targetValue: "85"
      activationThreshold: "10"
      gpuIndex: "0"              # specific GPU index, or omit for all
      aggregation: "max"         # max, min, avg, sum across GPUs

See deploy/examples/ for ready-to-use ScaledObject manifests.

The scaler exposes an optional Prometheus-compatible /metrics endpoint for monitoring the scaler itself and GPU fleet health. This is independent of the KEDA scaling path — scaling works identically with or without it.

# Enabled by default on port 9090
--metrics-port=9090

# Disable entirely (zero overhead)
--metrics-port=0

Helm:

metrics:
  enabled: true   # set to false to disable
  port: 9090

All per-GPU metrics are labeled with gpu_index, gpu_uuid, and gpu_name.

The scaler exposes liveness and readiness endpoints on a dedicated probe port:

• /healthz returns 200 while the process is alive.
• /readyz returns 200 after NVML initializes and the first metrics collection succeeds.

--probe-port=8081

Helm:

probes:
  enabled: true
  port: 8081

This project requires CGO_ENABLED=1 to compile the NVIDIA C-bindings.

# Build KEDA scaler binary (requires CGO for NVML)
make build

# Build standalone GPU metrics CLI (no KEDA/gRPC needed)
make build-metrics

# Build all binaries
make build-all

# Run unit tests
make test

# Run linter
make lint

# Generate protobuf Go code
make proto

# Build and push a release image
make docker-release VERSION=v0.1.0

# Deploy to cluster
make deploy

Collect GPU metrics without Kubernetes — works on bare metal, SLURM jobs, Flux jobs, Kubernetes pods, and Singularity containers. The same binary and the same JSON schema work everywhere.

gpu-metrics                       # one-shot table output (env auto-detected)
gpu-metrics --format json         # JSON for scripting
gpu-metrics --format csv          # CSV for analysis
gpu-metrics --interval 5s         # continuous collection
gpu-metrics --device 0 --quiet    # single GPU, no logs
gpu-metrics --env slurm           # force environment (auto|k8s|slurm|flux|standalone)

The --env flag auto-detects the orchestrator by default. Detection priority: SLURM → Flux → Kubernetes → standalone.

Every environment emits the same unified JSON schema with an environment block so you can compare GPU performance across on-prem and cloud with identical tooling:

{
  "environment": { "orchestrator": "slurm", "node": "compute-01", "job_id": "123", "task_rank": 0 },
  "collected_at": "2026-06-17T10:00:00Z",
  "devices": [...]
}

SLURM — auto-detected when SLURM_JOB_ID is set; collects only the GPUs assigned to your job step:

srun --gres=gpu:2 gpu-metrics --format json

Flux — auto-detected when FLUX_JOB_ID is set; collects only the GPUs in CUDA_VISIBLE_DEVICES:

flux run -N1 -g2 gpu-metrics --format json

See HPC & Cross-Environment Metrics for full usage, and Cross-Environment Comparison Guide for comparing on-prem vs cloud GPU runs.

Or build the Docker image directly:

docker build -t your-registry/keda-gpu-scaler:v0.1.0 .
docker push your-registry/keda-gpu-scaler:v0.1.0

• Design Document — Architecture decisions, gRPC interface, scaling profiles, testing strategy
• Migration Guide — Replace dcgm-exporter + Prometheus with keda-gpu-scaler
• HPC & Cross-Environment Metrics — SLURM, Flux, Kubernetes, and standalone GPU metrics
• Cross-Environment Comparison — Compare GPU performance across on-prem and cloud
• FAQ — Common questions about GPU scaling, MIG, multi-GPU, scale-to-zero
• Changelog — Release history

• CNCF Blog: GPU Autoscaling on Kubernetes with KEDA
• KEDA issue #7538 — original discussion
• CNCF TOC initiative #2188 — whitepaper proposal

Using keda-gpu-scaler? Add your organization to ADOPTERS.md.

• AMD ROCm support
• MIG per-instance metrics
• vLLM queue depth scaling

Contributions welcome — GPU autoscaling use cases, vendor support (AMD ROCm, Intel), or docs improvements. See CONTRIBUTING.md.

Apache License 2.0. See LICENSE for details.