feat: Add init-concurrency limiter primitive and configuration - #780
Merged
kinyoklion merged 1 commit intoJul 29, 2026
Merged
Conversation
kinyoklion
commented
Jul 28, 2026
kinyoklion
commented
Jul 28, 2026
kinyoklion
commented
Jul 28, 2026
kinyoklion
commented
Jul 28, 2026
kinyoklion
commented
Jul 28, 2026
Adds the admission-control limiter (internal/concurrency) -- a two-limit keyed semaphore (max in-flight + bounded FIFO queue, optional per-environment gate) -- and a [Concurrency] configuration section with INIT_* environment variables that will bound how many concurrent SDK-initialization deliveries the Relay Proxy performs across polling and streaming. This adds only the primitive and its configuration surface; wiring it into the initialization paths follows separately. The limiter is disabled unless configured, so behavior is unchanged by default.
kinyoklion
force-pushed
the
rlamb/relay-init-concurrency-primitives
branch
from
July 28, 2026 23:24
3a417aa to
e3c6628
Compare
kinyoklion
commented
Jul 28, 2026
keelerm84
approved these changes
Jul 29, 2026
kinyoklion
merged commit Jul 29, 2026
3a8de8f
into
feat/concurrency-init-limits
17 of 18 checks passed
| // make progress to its client within this duration. It defaults to 30s. | ||
| SendTimeout ct.OptDuration `conf:"INIT_SEND_TIMEOUT"` | ||
| } | ||
|
|
Contributor
There was a problem hiding this comment.
Question: there's a pattern in the code for validating the Config object's elements: [link]. Would it be beneficial to perform that validation on these new parameters?
This was referenced Jul 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the foundation for bounding initialization-delivery concurrency in the Relay Proxy: an admission-control limiter and a
[Concurrency]configuration section. The limiter caps how many SDK initialization deliveries — the full-dataset payloads Relay materializes and sends when an SDK first receives its data — may be in flight at once, across both polling and streaming, so a reconnect storm can't drive unbounded resident-payload memory or egress.This change adds only the limiter primitive and its configuration surface. Wiring it into the initialization paths is a follow-up; nothing consumes the limiter yet, so behavior is unchanged and the feature stays off unless configured.
Limiter (
internal/concurrency)A two-limit keyed semaphore:
MaxConcurrent— units of work held at once (in-flight deliveries).MaxQueued— bounded FIFO of waiters once concurrency is saturated;0sheds immediately.MaxConcurrent <= 0⇒ disabled (zero-overhead pass-through); nil-safe;Acquire(ctx, envKey) (release, ok);Close()unblocks all waiters. Unit-tested under-race.Configuration (
[Concurrency])maxConcurrentINIT_MAX_CONCURRENT≤0disablesmaxQueuedINIT_MAX_QUEUED0= shed when fullperEnvMaxPercentINIT_PER_ENV_MAX_PERCENT0= global-onlysendTimeoutINIT_SEND_TIMEOUTmaxConcurrentis anOptInt(notOptIntGreaterThanZero) so an explicit0disables gracefully instead of failing config validation.Testing
-race: admission, bounded-queue shed, disabled pass-through, exactly-once release, per-env gate.INIT_*variables, including a regression thatINIT_MAX_CONCURRENT=0disables rather than crashlooping.configsuite passes with the new section.Note
Low Risk
New code is isolated; no production paths call the limiter yet, so default behavior is unchanged.
Overview
Introduces a foundation for capping SDK initialization-delivery concurrency (polling and streaming). Runtime behavior is unchanged until a follow-up wires the limiter into init paths; with no config, the limiter stays disabled.
Adds
internal/concurrencywith an admissionLimiter: global in-flight slots (MaxConcurrent), optional bounded wait queue (MaxQueued,0= reject when saturated), and an optional per-environment try-only gate so one env cannot take the whole budget.Acquire(ctx, envKey)returns a one-shot release; disabled/nil limiters admit with no overhead.Adds
[Concurrency]/INIT_*settings onConfig(INIT_MAX_CONCURRENT,INIT_MAX_QUEUED,INIT_PER_ENV_MAX_PERCENT,INIT_SEND_TIMEOUT) and loads them from the environment.MaxConcurrentusesOptIntsoINIT_MAX_CONCURRENT=0disables the feature without failing validation.Unit tests cover limiter semantics (queueing, rejection, per-env isolation, context cancel, idempotent release) and config parsing including the zero-disable regression.
Reviewed by Cursor Bugbot for commit e3c6628. Bugbot is set up for automated code reviews on this repo. Configure here.