Skip to content

feat: Add init-concurrency limiter primitive and configuration - #780

Merged
kinyoklion merged 1 commit into
feat/concurrency-init-limitsfrom
rlamb/relay-init-concurrency-primitives
Jul 29, 2026
Merged

feat: Add init-concurrency limiter primitive and configuration#780
kinyoklion merged 1 commit into
feat/concurrency-init-limitsfrom
rlamb/relay-init-concurrency-primitives

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Jul 28, 2026

Copy link
Copy Markdown
Member

Adds the foundation for bounding initialization-delivery concurrency in the Relay Proxy: an admission-control limiter and a [Concurrency] configuration section. The limiter caps how many SDK initialization deliveries — the full-dataset payloads Relay materializes and sends when an SDK first receives its data — may be in flight at once, across both polling and streaming, so a reconnect storm can't drive unbounded resident-payload memory or egress.

This change adds only the limiter primitive and its configuration surface. Wiring it into the initialization paths is a follow-up; nothing consumes the limiter yet, so behavior is unchanged and the feature stays off unless configured.

Limiter (internal/concurrency)

A two-limit keyed semaphore:

  • MaxConcurrent — units of work held at once (in-flight deliveries).
  • MaxQueued — bounded FIFO of waiters once concurrency is saturated; 0 sheds immediately.
  • optional per-environment try-only gate, so one environment can't consume the whole budget and starve the others.
  • MaxConcurrent <= 0 ⇒ disabled (zero-overhead pass-through); nil-safe; Acquire(ctx, envKey) (release, ok); Close() unblocks all waiters. Unit-tested under -race.

Configuration ([Concurrency])

File key Env var Meaning
maxConcurrent INIT_MAX_CONCURRENT max in-flight deliveries; unset/≤0 disables
maxQueued INIT_MAX_QUEUED bounded FIFO backlog; 0 = shed when full
perEnvMaxPercent INIT_PER_ENV_MAX_PERCENT one env's max share of the budget; 0 = global-only
sendTimeout INIT_SEND_TIMEOUT backstop to reclaim a slot from a stalled delivery (default 30s)

maxConcurrent is an OptInt (not OptIntGreaterThanZero) so an explicit 0 disables gracefully instead of failing config validation.

Testing

  • Limiter unit tests under -race: admission, bounded-queue shed, disabled pass-through, exactly-once release, per-env gate.
  • Config parse test for the INIT_* variables, including a regression that INIT_MAX_CONCURRENT=0 disables rather than crashlooping.
  • Existing config suite passes with the new section.

Note

Low Risk
New code is isolated; no production paths call the limiter yet, so default behavior is unchanged.

Overview
Introduces a foundation for capping SDK initialization-delivery concurrency (polling and streaming). Runtime behavior is unchanged until a follow-up wires the limiter into init paths; with no config, the limiter stays disabled.

Adds internal/concurrency with an admission Limiter: global in-flight slots (MaxConcurrent), optional bounded wait queue (MaxQueued, 0 = reject when saturated), and an optional per-environment try-only gate so one env cannot take the whole budget. Acquire(ctx, envKey) returns a one-shot release; disabled/nil limiters admit with no overhead.

Adds [Concurrency] / INIT_* settings on Config (INIT_MAX_CONCURRENT, INIT_MAX_QUEUED, INIT_PER_ENV_MAX_PERCENT, INIT_SEND_TIMEOUT) and loads them from the environment. MaxConcurrent uses OptInt so INIT_MAX_CONCURRENT=0 disables the feature without failing validation.

Unit tests cover limiter semantics (queueing, rejection, per-env isolation, context cancel, idempotent release) and config parsing including the zero-disable regression.

Reviewed by Cursor Bugbot for commit e3c6628. Bugbot is set up for automated code reviews on this repo. Configure here.

Comment thread config/config.go Outdated
Comment thread config/config.go Outdated
Comment thread config/config.go Outdated
Comment thread config/config.go Outdated
Comment thread internal/concurrency/limiter.go Outdated
Adds the admission-control limiter (internal/concurrency) -- a two-limit keyed
semaphore (max in-flight + bounded FIFO queue, optional per-environment gate) --
and a [Concurrency] configuration section with INIT_* environment variables that
will bound how many concurrent SDK-initialization deliveries the Relay Proxy
performs across polling and streaming. This adds only the primitive and its
configuration surface; wiring it into the initialization paths follows separately.
The limiter is disabled unless configured, so behavior is unchanged by default.
@kinyoklion
kinyoklion force-pushed the rlamb/relay-init-concurrency-primitives branch from 3a417aa to e3c6628 Compare July 28, 2026 23:24
Comment thread internal/concurrency/limiter.go
@kinyoklion
kinyoklion changed the base branch from v9 to feat/concurrency-init-limits July 29, 2026 15:48
@kinyoklion
kinyoklion marked this pull request as ready for review July 29, 2026 15:50
@kinyoklion
kinyoklion requested a review from a team as a code owner July 29, 2026 15:50
@kinyoklion
kinyoklion merged commit 3a8de8f into feat/concurrency-init-limits Jul 29, 2026
17 of 18 checks passed
@kinyoklion
kinyoklion deleted the rlamb/relay-init-concurrency-primitives branch July 29, 2026 17:27
Comment thread config/config.go
// make progress to its client within this duration. It defaults to 30s.
SendTimeout ct.OptDuration `conf:"INIT_SEND_TIMEOUT"`
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: there's a pattern in the code for validating the Config object's elements: [link]. Would it be beneficial to perform that validation on these new parameters?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants