feat(chunk-upload): Add zstd compression support for sourcemap uploads

[BYK]

Summary

Extends the chunk-upload endpoint used by sourcemap / artifact-bundle / DIF / preprod uploads to accept zstd-compressed chunks in addition to the existing gzip path, advertised to clients via the compression capability list in the GET response.

Protocol changes

• New Content-Encoding detection path. When Content-Encoding: gzip or Content-Encoding: zstd is set on a chunk upload POST, every chunk delivered under the standard file multipart field is decoded using that codec.
• Legacy file_gzip field still works for backwards compatibility with older sentry-cli / @sentry/cli versions that only know how to send chunks under that field name.
• Ambiguous requests are rejected with HTTP 400 (Cannot combine Content-Encoding with file_gzip field). A single request cannot use both mechanisms.
• Unsupported encodings return HTTP 400 (Unsupported Content-Encoding).
• compression list extended to ["gzip", "zstd"]. No CHUNK_UPLOAD_ACCEPT entry is added — clients have always feature-detected compression via this list. The existing chunk-upload.no-compression org allowlist kill-switch continues to disable all compression codecs.

Security / safety

Decompression is streamed through zstandard.ZstdDecompressor().stream_reader(file) (and GzipFile for gzip) via a bounded-read helper that loops until EOF or a hard cap of settings.SENTRY_CHUNK_UPLOAD_BLOB_SIZE (8 MiB) on decoded output per chunk. The loop handles short reads from streaming decoders (important for ZstdDecompressor.stream_reader + file-backed TemporaryUploadedFile). Excess bytes raise ChunkTooLarge → 400. Peak memory stays bounded at limit + 1 bytes regardless of how well-compressed the input is, protecting against decompression-bomb inputs where a tiny compressed payload would expand to hundreds of megabytes.

Invalid / malformed payloads are all converted to 400 (never 500):

• zstandard.ZstdError → Invalid zstd payload
• gzip.BadGzipFile (bad magic / header) → Invalid gzip payload
• EOFError (truncated stream) → Invalid gzip payload

End-to-end verified

Ran the full request pipeline (middleware + auth + DRF + endpoint) against the live getsentry devservices stack on a fresh Coder VM and confirmed:

• GET options endpoint advertises "compression": ["gzip", "zstd"].
• POST with Content-Encoding: zstd + a zstd-compressed chunk → 200 + FileBlob persisted.
• POST with Content-Encoding: gzip + a gzipped chunk → 200 (new gzip-via-header path).
• POST with legacy file_gzip field (no header) → 200 (backwards compat).
• POST combining both forms → 400 {"error": "Cannot combine Content-Encoding with file_gzip field"}.
• POST with Content-Encoding: br → 400 {"error": "Unsupported Content-Encoding"}.
• POST with a zstd payload that would decompress to 8 MiB+1 → 400 {"error": "Chunk size too large"} without OOMing.

Tests

• Updated test_chunk_parameters to assert compression == ["gzip", "zstd"].
• Added 12 new tests in tests/sentry/api/endpoints/test_chunk_upload.py:
  • test_chunk_parameters_no_compression_option — kill switch zeroes the list.
  • test_upload_content_encoding_zstd — happy path, 2 chunks.
  • test_upload_content_encoding_zstd_multi_frame — concatenated zstd frames decode fully (read_across_frames=True).
  • test_upload_content_encoding_zstd_mixed_case — Content-Encoding: ZSTD accepted (RFC 7231 case-insensitive).
  • test_upload_content_encoding_gzip — gzip-via-header path.
  • test_upload_legacy_file_gzip_still_works — backwards compat.
  • test_upload_rejects_content_encoding_with_file_gzip — 400 on ambiguous request.
  • test_upload_rejects_unsupported_content_encoding — 400 on br.
  • test_upload_zstd_decompression_bomb — rejects 8 MiB+1 expansion.
  • test_upload_invalid_zstd_payload — malformed zstd → 400, not 500.
  • test_upload_invalid_gzip_payload_content_encoding — malformed gzip via header → 400.
  • test_upload_invalid_gzip_payload_legacy_field — malformed gzip via file_gzip → 400.
  • test_upload_truncated_gzip_payload — valid header, truncated stream → 400.

Negative tests also pin the exact error-body shape so a typo in the error string would be caught.

pytest --reuse-db tests/sentry/api/endpoints/test_chunk_upload.py → 30 passed. mypy + pre-commit clean.

Scope boundaries

• No changes to src/sentry/tasks/assemble.py, src/sentry/debug_files/upload.py, or the assemble endpoints — they operate on already-decompressed FileBlobs keyed by sha1.
• zstandard>=0.18.0 is already a direct dep in pyproject.toml and used extensively elsewhere (src/sentry/utils/codecs.py, src/sentry/attachments/base.py, src/sentry/spans/buffer.py, etc.). No new dependencies.
• No protocol version bump needed — clients feature-detect via the compression list, matching the existing pattern.
• Error responses use {"error": ...} to stay consistent with the rest of this file (4 pre-existing "error" sites at lines 264/276/283/304) and with the neighbouring chunk-upload surface (organization_artifactbundle_assemble.py). src/AGENTS.md now recommends {"detail": ...} for new endpoints; migrating the whole chunk-upload subsystem is out of scope for this PR.

Companion PR

CLI side: getsentry/cli#823 (merged) — teaches the new TS/Bun @loreai/cli to send zstd-compressed chunks via the Content-Encoding: zstd header when the server advertises it.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 646ddf2. Configure here.}

[armenzg]

The changes look good.
Feel free to ignore the logging suggestions. It's a pet-peeve of mine.
My only question for you is the option gate in the post method.