Feature Request: Add upload backpressure and queue position to the recordings upload API

[devkit415]

Description

The browser upload path enforces a concurrency cap through MAX_CONCURRENT_UPLOADS, but the API upload path accepts an unlimited number of uploads and enqueues them all.

Under concurrent upload load, 8 files were uploaded in 0.7 seconds with no feedback to the caller, and the queue took over 10 minutes to drain.

Callers have no way to learn their queue position or estimated wait at upload time, even though the queue already computes it.

This makes it easy to flood the queue without any signal, and it leaves users guessing whether to wait or come back later.

Affected Version

v0.10.18-alpha

Requested Feature

Apply a per-user or per-instance in-flight upload cap to POST /api/v1/recordings/upload, returning HTTP 429 with a Retry-After header when exceeded, and include the resulting queue position and an estimated wait in both the upload response and the recording status payload.

Proposed Implementation

• Reuse the existing MAX_CONCURRENT_UPLOADS setting (src/api/system.py:40) or add an API-specific equivalent, and enforce it in upload_recording (src/api/api_v1.py:3825) by counting the caller's recordings in PENDING or QUEUED state before accepting a new upload.
• When the cap is exceeded, return HTTP 429 with a Retry-After value and a body that states the current in-flight count and the cap, instead of accepting and silently enqueuing.
• Expose get_position_in_queue (src/services/job_queue.py:827) in the upload response and in the recording detail payload, so the caller learns its position immediately after a successful upload.
• Add an estimated wait derived from the position and the observed per-job duration, or at least document the formula, so clients can show a meaningful ETA.
• Keep the cap configurable through environment variables, consistent with MAX_CONCURRENT_UPLOADS, and document it in the runner and deployment notes.
• Preserve the existing behavior for the browser /upload path, which already enforces the cap.

Expected Behavior

• An upload that would exceed the cap returns HTTP 429 with Retry-After instead of being accepted.
• A successful upload response includes the queue position for that recording.
• The recording detail endpoint exposes the same position and an estimated wait while the job is queued.
• The cap is configurable and defaults to a sensible value consistent with the existing UI cap.
• No change to the JSON shape of successful uploads beyond the added fields, and no change to authorization decisions.

Root Cause Analysis

MAX_CONCURRENT_UPLOADS (src/api/system.py:40) only governs the browser upload path. upload_recording in src/api/api_v1.py:3825 accepts every request and calls ingest_uploaded_recording (src/api/recordings.py:2323), which enqueues a transcription job without checking how many jobs the user already has in flight.

The queue already computes a position through get_position_in_queue (src/services/job_queue.py:827), but that value is not surfaced at upload time.

The current behavior deviates from the desired one because the API has no backpressure and no feedback channel, so a caller can enqueue an unbounded number of jobs and learn nothing about when they will run.

[murtaza-nasir]

Thanks for the proposal. With MAX_CONCURRENT_UPLOADS, only the number of files the browser sends at once is limited, and waiting jobs were already taken from users in turn. While testing the turns, I found a case where a user lost a turn when their next file arrived after other users' files. In the next release, every user with a waiting job gets one job started before anyone gets a second. queue_position in GET /api/v1/recordings/{id}/status was null for most queued recordings; it now holds the recording's place in its queue, following the turns. A new stage field is set to conversion, transcription or summary, according to what the recording is waiting for. I am not adding a per-user upload limit for now. This will be included in the next release.