Make Terraform provider lock-file hashing concurrency configurable (OOMs on large providers)

[christianc-bdev]

Problem

When Renovate regenerates .terraform.lock.hcl for a large multi-platform provider (e.g. hashicorp/aws), TerraformProviderHash.createHashes() downloads and extracts provider builds 4 at a time with a hardcoded concurrency. For hashicorp/aws v6 (provider binary ~600 MB extracted per platform), four concurrent extractions peak at ~4.6 GiB of anonymous (unreclaimable) memory, OOM-killing (exit 137) any container/job capped below that.

There is currently no way to lower this concurrency, so memory-constrained self-hosted runners (e.g. Kubernetes CI shards with a per-job memory limit) have no mitigation other than raising the limit fleet-wide.

Request

Expose the hashing concurrency as a configurable value (e.g. an experimental env var such as RENOVATE_X_TERRAFORM_PROVIDER_HASH_CONCURRENCY), defaulting to the current value of 4.

Setting it to 1 lets constrained runners trade speed for memory.

Root Cause

// lib/modules/manager/terraform/lockfile/hash.ts

return p.map(
  builds,
  (build) => this.calculateSingleHash(build, cacheDir),
  { concurrency: 4 } // hardcoded
);

Each calculateSingleHash() streams a provider ZIP to disk, extracts it, and hashes the extracted contents.

The memory footprint is dominated by the extraction workload of the active workers, so peak memory scales with concurrency width rather than total platform count.

Evidence That Peak Memory Is Concurrency-Bound (Not Platform-Count-Bound)

Measured on the official Renovate image against a real repository during a single hashicorp/aws ~> 5.0 → ~> 6.0 update:

• 4 GiB memory limit → OOMKilled during the first batch of 4 builds (Calculating hashes for 15 builds, 0 hashes completed).
• 8 GiB memory limit → succeeds, all 15 builds hashed.
• Peak anonymous memory (cgroup v2 memory.stat anon): ~4.6 GiB.

Finished batches free memory; only 4 builds are ever in flight simultaneously.

This indicates the peak tracks the fixed 4-wide concurrency window and does not grow with the total number of platforms.

Reducing concurrency from 4 to 1 is therefore expected to reduce peak memory approximately 4× (~1.2 GiB), allowing the same workload to fit comfortably within a 4 GiB job.

The output remains byte-identical:

• Same hashes
• Same platforms
• Same lock file contents

Only execution time changes.

Why Other Configuration Levers Don't Help

hostRules.concurrentRequestLimit

No effect.

Provider ZIP downloads use http.stream(), which bypasses the per-host request queue/throttling mechanism.

This was verified during testing:

• Host queue logged concurrency=2
• Provider downloads still ran 4-wide
• Memory usage and OOM behavior were unchanged

prConcurrentLimit / branchConcurrentLimit

No effect.

These limits operate at the PR/branch level.

The OOM occurs within a single branch while hashing a single Terraform provider.

Reducing Locked Platforms

Limited impact.

Even if the lock file contains fewer platforms, the current implementation re-hashes all registry platforms rather than preserving the existing platform subset.

Additionally, memory pressure is driven primarily by the active concurrency window.

Raising Job Memory Limits

Works, but is a costly fleet-wide workaround for an infrequent operation.

Reproduction

1. Repository contains a .terraform.lock.hcl pinning hashicorp/aws ~> 5.0.

2. Let Renovate open the ~> 6.0 major update. During updateArtifacts, logs show:

Creating hashes for hashicorp/aws@6.50.0
Got 16 zip hashes for hashicorp/aws@6.50.0
Calculating hashes for 15 builds

Hash generation for ...terraform-provider-aws_6.50.0_<platform>.zip took ~26000–41000 ms

3. Run Renovate with a 4 GiB cgroup memory limit.

Result:

OOMKilled (137)

during Calculating hashes for 15 builds.

Environment

• Renovate 43.220.0
• Official renovate/renovate image
• Self-hosted on GitLab CI (Kubernetes executor)
• Per-job memory limit: 4 GiB