api: uniform 422 for a too-deeply-nested JSON request body - #64
Merged
Conversation
The structured deep-nesting 500 was already contained but responses were inconsistent by depth: <=200 normal 422, ~500-900 422 too_deeply_nested, >=1000 a 400 'error parsing the body' (Starlette catching the parse-level RecursionError). Add a small ASGI middleware that rejects a JSON body nested past MAX_JSON_NESTING_DEPTH (200) with a uniform 422 BEFORE the parser recurses. Scans only the first 64KB of the raw body (early-exiting json_nesting_exceeds): the attack payload is a few KB, a legit large body is shallow, so the guard costs ~0.8ms; scanning a full 6.9MB body would be ~280ms. Registered inside CORS so the 422 carries CORS headers. A body padding shallow content past 64KB before nesting deep still fails safely downstream (clean 4xx, never 500). ADR 0015. Verified: uniform 422 across depths 200..20000, normal requests still 200, guard ~0.8ms, full suite 19->16 fails (the 3 deeply_nested tests flip to pass, zero new failures). Backlog flagged: large-shallow-body DoS (~0.7ms/point -> a 100k-point request ties a worker ~73s) is separate and unaddressed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uniform 422 for a too-deeply-nested JSON request body
The 500 crash was already gone — but the responses were inconsistent by depth: ≤200 → normal 422, ~500–900 → 422
too_deeply_nested, ≥1000 → 400 "error parsing the body" (Starlette catching the parse-levelRecursionError). This adds a small ASGI middleware that returns a uniform 422too_deeply_nestedfor any body nested pastMAX_JSON_NESTING_DEPTH(200), before the parser recurses.Verified
deeply_nestedtests flip to pass, zero new failures.The analytics pipeline runs ~0.7 ms/point, so a single unauthenticated 100 000-point request (the
MAX_PUBLIC_POINTScap) ties up a worker for ~73 s. That's a real availability issue but orthogonal to deep-nesting — it needs its own cycle. Noted in ADR 0015 and memory.