Skip to content

Comments

feat(service): Add concurrency limit to StorageService#324

Merged
jan-auer merged 7 commits intomainfrom
feat/service-concurrency-limit
Feb 21, 2026
Merged

feat(service): Add concurrency limit to StorageService#324
jan-auer merged 7 commits intomainfrom
feat/service-concurrency-limit

Conversation

@jan-auer
Copy link
Member

@jan-auer jan-auer commented Feb 20, 2026

Builds on #322 (task spawning with panic isolation).

Adds a semaphore-based concurrency limit to StorageService that caps the total number of in-flight backend operations. When the limit is reached, new operations are rejected immediately with HTTP 429 rather than queueing, preventing backend overload during traffic bursts.

The limit is acquired before spawning each operation task and the permit is held until the task completes (including after panics). Configured via service.max_concurrency (default: 500, effectively unlimited without configuration).

Includes tests for rejection at capacity and permit release after panics.

Next steps

  • Web request concurrency limit: Add a high web-tier limit (middleware, 503 on exhaustion) that protects against connection floods, composing with the existing InFlightRequestsLayer. Excludes health/ready endpoints.
  • Bounded task queue: Replace direct tokio::spawn with a bounded channel dispatching to a fixed worker pool, enabling backpressure, fire-and-forget operations, and priority scheduling.
  • Cancellation: Use CancellationToken to actively cancel backend operations on client disconnect, with safety invariants protecting tombstone writes from cancellation.

Ref FS-171

@jan-auer jan-auer marked this pull request as ready for review February 20, 2026 20:26
@jan-auer jan-auer requested a review from a team as a code owner February 20, 2026 20:26
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

///
/// This value is used when no explicit limit is set via
/// [`StorageService::with_concurrency_limit`].
pub const DEFAULT_CONCURRENCY_LIMIT: usize = 500;
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This default is chosen arbitrarily so that it is not unreasonably high, but still high enough to not limit in practice. We will tweak this.

@jan-auer jan-auer merged commit 033cf40 into main Feb 21, 2026
20 checks passed
@jan-auer jan-auer deleted the feat/service-concurrency-limit branch February 21, 2026 12:23
@linear
Copy link

linear bot commented Feb 23, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants