Cloud Run External Metrics Autoscaling

The Cloud Run External Metrics Autoscaling (CREMA) project leverages KEDA to provide autoscaling for Cloud Run services and worker pools. This guide will walk you through the deployment of Cloud Run's pre-built CREMA container image.

Architecture

How it works

CREMA can be deployed to Cloud Run to scale Cloud Run workloads based on external metrics in generally two steps:

Metrics Polling: Polls external sources (e.g., Kafka lag, Pub/Sub queue depth) using KEDA scalers to retrieve current metric values.
Setting Instances: Calculates the recommended instance count and updates the target workload via the Cloud Run Admin API.

Compatibility

This project currently depends on KEDA v2.20. The included table lists various KEDA scalers and their compatibility for use with Cloud Run.

Scalers	Cloud Run Compatible	Notes
Apache Kafka	Verified
Cron	Verified
GCP Pub/Sub	Verified
GCP Stackdriver	Verified
Github Runner Scaler	Verified
Prometheus	Verified
RabbitMQ Queue	Verified
Redis Lists	Verified
Temporal	Verified
CPU	Incompatible	Scaler is Kubernetes-specific
Kubernetes Workload	Incompatible	Scaler is Kubernetes-specific
Memory	Incompatible	Scaler is Kubernetes-specific

See https://keda.sh/docs/2.20/scalers/ for the full list of KEDA's scalers. The compatibility for any KEDA scaler not listed above is currently unknown. Please file an issue if you believe a scaler does not work.

Setup

Follow the instructions below to configure and deploy CREMA as a Cloud Run service to scale your Cloud Run workloads on metrics external to Cloud Run.

Prerequisites

Google Cloud SDK: Ensure you have the Google Cloud SDK installed and configured.

Authentication: Authenticate with Google Cloud:

gcloud auth login
gcloud auth application-default login

Project Configuration: Set your default project:
```
gcloud config set project MY_PROJECT_ID
```
Replace MY_PROJECT_ID with your actual Google Cloud project ID.

Create a GCP Service Account

Create a GCP service account that will be used by the Cloud Run CREMA service. We'll grant this service account the necessary permissions throughout the setup. Those permissions will be:

Parameter Manager Parameter Viewer (roles/parametermanager.parameterViewer) to retrieve from Parameter Manager the CREMA configuration you'll be creating.
Cloud Run Developer (roles/run.developer) and Service Account User (roles/iam.serviceAccountUser) to set the number of instances in your scaled workloads.

PROJECT_ID=my-project
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account

gcloud iam service-accounts create $CREMA_SERVICE_ACCOUNT_NAME \
  --display-name="CREMA Service Account"

Configure

Follow the steps below to create a yaml configuration file for CREMA in Parameter Manager.

Create a Parameter in Parameter Manager to store the configuration used by CREMA:

PARAMETER_ID=crema-config
PARAMETER_REGION=global
gcloud parametermanager parameters create $PARAMETER_ID --location=$PARAMETER_REGION --parameter-format=YAML

Locally, create a YAML file for your CREMA configuration. See the Configuration README for reference.

Upload your local YAML file to Parameter Manager as a new parameter version:

LOCAL_YAML_CONFIG_FILE=./my-crema-config.yaml
PARAMETER_ID=crema-config
PARAMETER_REGION=global
PARAMETER_VERSION=1

gcloud parametermanager parameters versions create $PARAMETER_VERSION \
  --location=$PARAMETER_REGION \
  --parameter=$PARAMETER_ID \
  --payload-data-from-file=$LOCAL_YAML_CONFIG_FILE

Grant your CREMA service account permission to read the parameter version from Parameter Manager:

PROJECT_ID=my-project
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:$CREMA_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/parametermanager.parameterViewer"

Grant your CREMA service account permission to scale the workloads that you've specified in your CREMA configuration. This can be done by granting roles/run.developer at the project level or for each individual service or worker pool to be scaled.

Granting the required permissions at the project level will enable CREMA to scale any workloads that you specify in the configuration--you'll be able to add more workloads in the future without having to further modify permissions. To grant these permissions at the project level:

PROJECT_ID=my-project
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:$CREMA_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/run.developer"

Alternatively, you can grant the required permissions for each individual service or worker pool. This minimizes the permissions to strictly what's necessary and is considered a security best practice. To grant these permissions for each individual service or worker pool:

# For a service
PROJECT_ID=my-project
SERVICE_NAME=my-service-to-be-scaled
SERVICE_REGION=us-central1
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account


gcloud run services add-iam-policy-binding $SERVICE_NAME \
  --region=$SERVICE_REGION \
  --member="serviceAccount:$CREMA_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/run.developer"

# For a worker pool
PROJECT_ID=my-project
WORKER_POOL_NAME=my-worker-pool-to-be-scaled
WORKER_POOL_REGION=us-central1
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account

gcloud beta run worker-pools add-iam-policy-binding $WORKER_POOL_NAME \
  --region=$WORKER_POOL_REGION \
  --member="serviceAccount:$CREMA_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/run.developer"

Grant your CREMA service account roles/iam.serviceAccountUser on the service accounts which run the Cloud Run workloads to be scaled:

PROJECT_ID=my-project
CONSUMER_SERVICE_ACCOUNT_NAME=my-worker-pool-sa
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account

gcloud iam service-accounts add-iam-policy-binding \
    $CONSUMER_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com \
    --member="serviceAccount:$CREMA_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
    --role="roles/iam.serviceAccountUser"

Deploy

We recommend deploying CREMA using Cloud Run's pre-built container image in us-central1-docker.pkg.dev/cloud-run-oss-images/crema-v1/autoscaler. Review details in the Image Versioning reference section below.

However, you can optionally build the container image yourself from source code using Cloud Build (see instructions below), subject to a 30+ minute build process.

The command here deploys CREMA as a Cloud Run service using the pre-built container image; if you want to deploy your own built container image, update the IMAGE variable to specify it.

Configure the variables and the command deploy command:

SERVICE_NAME: The name for your CREMA service
SERVICE_REGION: The region to run your CREMA service in.
CREMA_SERVICE_ACCOUNT_NAME: The name of the service account which will run CREMA
PARAMETER_VERSION: The parameter version you created

SERVICE_NAME=my-crema-service
SERVICE_REGION=us-central1
BASE_IMAGE_REGION=us-central1
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account
PARAMETER_VERSION=1

CREMA_CONFIG_PARAM_VERSION=projects/$PROJECT_ID/locations/$PARAMETER_REGION/parameters/$PARAMETER_ID/versions/$PARAMETER_VERSION
IMAGE=us-central1-docker.pkg.dev/cloud-run-oss-images/crema-v1/autoscaler:1.2

gcloud run deploy $SERVICE_NAME \
  --image=${IMAGE} \
  --region=${SERVICE_REGION} \
  --base-image=${BASE_IMAGE_REGION}-docker.pkg.dev/serverless-runtimes/google-24/runtimes/java25 \
  --service-account="${CREMA_SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \
  --no-allow-unauthenticated \
  --no-cpu-throttling \
  --scaling=1 \
  --labels=created-by=crema \
  --set-env-vars="CREMA_CONFIG=${CREMA_CONFIG_PARAM_VERSION},OUTPUT_SCALER_METRICS=False,ENABLE_CLOUD_LOGGING=False"

Note: The container is built using FROM scratch to support automatic base image updates. Although matching the base image region to the service region is ideal, it's not strictly required as Cloud Run does not provide base images in all Cloud Run regions.

The following environment variables are checked by the container:

CREMA_CONFIG: Required. The fully qualified name (FQN) of the parameter version which contains your CREMA config.
OUTPUT_SCALER_METRICS: Optional. If true, CREMA will emit metrics to Cloud Monitoring.
ENABLE_CLOUD_LOGGING: Optional. If true, CREMA will log errors to Cloud Logging for improved log searchability.
LOG_FORMAT: Optional. If set to json, CREMA will output JSON-structured payloads natively instead of legacy plain-text logs.

Note: The OUTPUT_SCALER_METRICS and ENABLE_CLOUD_LOGGING flags are disabled by default as these may incur additional costs. See Cloud Observability Pricing for details.

If you set the OUTPUT_SCALER_METRICS=True environment variable, you'll also have to grant your CREMA service account permission to write metrics:

PROJECT_ID=my-project
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:$CREMA_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/monitoring.metricWriter"

If you set the ENABLE_CLOUD_LOGGING=True environment variable, you'll also have to grant your CREMA service account permission to write log entries:

PROJECT_ID=my-project
CREMA_SERVICE_ACCOUNT_NAME=crema-service-account

gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:$CREMA_SERVICE_ACCOUNT_NAME@$PROJECT_ID.iam.gserviceaccount.com" \
  --role="roles/logging.logWriter"

Verify

Use the resource below to verify that your CREMA service is running correctly.

Cloud Logging Logs

CREMA writes logs to Cloud Logging during each scaling cycle. By default, these are emitted as legacy plain-text logs. If you deployed CREMA with the LOG_FORMAT=json environment variable, CREMA will emit fully structured JSON logs with a jsonPayload.

Plain-text mode: Each log message is labeled with the component that emitted it.

[INFO] [METRIC-PROVIDER] Starting metric collection cycle
[INFO] [METRIC-PROVIDER] Successfully fetched scaled object metrics ...
[INFO] [METRIC-PROVIDER] Sending scale request ...
[INFO] [SCALER] Received ScaleRequest ...
[INFO] [SCALER] Current instances ...
[INFO] [SCALER] Recommended instances ...

TIP: Use the following Cloud Logging query for filtering plain-text logs: "[SCALER]" OR "[METRIC-PROVIDER]"

JSON Structured mode (LOG_FORMAT=json): Logs are natively parsed by Google Cloud Logging, placing custom fields into jsonPayload. This allows you to easily filter and alert on specific metadata:

{
  "jsonPayload": {
    "message": "Recommendation was clamped to range",
    "resource": "projects/my-project/locations/us-central1/workerpools/my-pool",
    "minReplicaCount": 1,
    "maxReplicaCount": 100,
    "component": "scaler"
  }
}

TIP: Use the following Cloud Logging queries for filtering structured logs:

View all CREMA logs: jsonPayload.component="scaler" OR jsonPayload.message=~"\[METRIC-PROVIDER\]"
View scaling events for a specific resource: jsonPayload.resource="projects/my-project/locations/us-central1/workerpools/my-pool"

Optional: Build the container image from source

Follow the steps below to build the CREMA container image yourself. The resulting container image will be pushed to Artifact Registry. Note that this build typically takes 30+ minutes.

Create an Artifact Registry repository to store the CREMA container image if you don't already have one:

PROJECT_ID=my-project
CREMA_REPO_NAME=crema
AR_REGION=us-central1

gcloud artifacts repositories create "${CREMA_REPO_NAME}" --repository-format=docker --location=$AR_REGION --description="Docker repository for CREMA images"

Use Google Cloud Build and the included Dockerfile to build the container image and push it to Artifact Registry. Run the following command from the root of this project:

PROJECT_ID=my-project
CREMA_REPO_NAME=crema
AR_REGION=us-central1

gcloud builds submit --tag $AR_REGION-docker.pkg.dev/$PROJECT_ID/$CREMA_REPO_NAME/crema:latest .

Metrics

If configured, CREMA will emit the following metrics:

custom.googleapis.com/$TRIGGER_TYPE/metric_value: The metric value it received, per trigger type
custom.googleapis.com/recommended_instance_count: The number of instances recommended, per Cloud Run scaled object
custom.googleapis.com/requested_instance_count: The number of instances requested, per Cloud Run scaled object

Image Versioning

Release Image Tag	Supported KEDA Version	Release Status / Details
`1.2`	`v2.20`	Upgraded version supporting latest KEDA frameworks. Preferred release tag.
`1.1`	`v2.19`	Legacy release. This version will no longer receive future updates or bug fixes.
`1.0`	`v2.17`	Legacy release. This version will no longer receive future updates or bug fixes.

Note: Referencing designated, explicit release revisions (e.g., the latest point release tag) is the preferred application pattern. Avoid using early revisions.

Note: Images published before April 29, 2026 contain known memory leaks; see Known Issues for more details.

Known Issues

Go memory leak: Builds before April 29, 2026 left scaler connections open under repeated scaling iteration cycles, leading to gradual memory leaks. This issue is fixed in GitHub and across public release images tagged on or after April 29, 2026.
Slow metrics in Cloud Monitoring: Many Google Cloud Monitoring metrics have 2+ minute ingestion delay which may affect scaling responsiveness for Google Cloud Platform scalers. See the Google Cloud metrics list for the underlying metrics used by the scaler for latency details.
A given Cloud Run service or worker pool should only be scaled by a single CREMA deployment: Scaling the same service or worker pool from multiple CREMA deployments can lead to race conditions and unexpected scaling behavior.

Support

For any technical issues and help with troubleshooting please file a support case to receive a response according to our initial response times. Please follow step-by-step instructions to create a new support case.

Name		Name	Last commit message	Last commit date
.github/ISSUE_TEMPLATE		.github/ISSUE_TEMPLATE
images		images
metric-provider		metric-provider
proto		proto
scaler		scaler
terraform		terraform
CONTRIBUTING		CONTRIBUTING
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
entrypoint.sh		entrypoint.sh

Navigation Menu

Uh oh!

Repository files navigation

Cloud Run External Metrics Autoscaling

Architecture

How it works

Compatibility

Setup

Prerequisites

Create a GCP Service Account

Configure

Deploy

Verify

Cloud Logging Logs

Optional: Build the container image from source

Metrics

Image Versioning

Known Issues

Support

About

Releases

Packages

Used by

Contributors

Languages

Uh oh!

Latest commit

History

Folders and files

Repository files navigation

Cloud Run External Metrics Autoscaling

Architecture

How it works

Compatibility

Setup

Prerequisites

Create a GCP Service Account

Configure

Deploy

Verify

Cloud Logging Logs

Optional: Build the container image from source

Metrics

Image Versioning

Known Issues

Support

About

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages