fix(pingora): read Contents-API-oversized files through the Git Blobs API - #1946
fix(pingora): read Contents-API-oversized files through the Git Blobs API#1946seonghobae wants to merge 2 commits into
Conversation
… API The Contents API stops inlining content at 1 MiB and answers `encoding: "none"`; `_load_raw_file_bytes` turned that into ContentSizeExceededError and, for any file that is not a documentation suffix or a verified binary document, the whole check failed closed. #1678's regenerated docs/sbom/inventory.json (1,148,611 bytes, no diff patch, zero Nginx runtime forms when scanned offline with this module) therefore failed required-workflow-bootstrap in 5 s on every push. Follow the Contents response's blob `sha` to the Git Blobs API (bounded by a new MAX_BLOB_BYTES of 11 MiB so the 60-column-wrapped base64 response fits the 16 MiB reader), bind every blob field back to the Contents metadata it was reached from, and scan the bytes like any inline file. ContentSizeExceededError keeps its narrow meaning for a file over the blob ceiling or a response with no well-formed blob sha; the oversized-documentation-PDF suffix convention now applies only there, so a .pdf within the ceiling is verified by its magic bytes. Gate: 2921 passed / 1 skipped with GITHUB_ACTIONS=true, coverage 100%, interrogate 100%; tests/test_pingora_edge_policy.py 85 passed (18 new). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
|
Warning Review limit reachedNext included review available in 10 minutes. View limit detailsLimit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Team Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
CI status note on head
Everything else on this head is green or queued: Generated by Claude Code |
|
CI note on head This run is the first of mine created after #1944, so it shipped
So the ceiling on this class is not the pool alone: the breaker's 30 s reset is shorter than a single 90 s attempt, so a stalled route is re-admitted every cycle, and the candidate order keeps putting it first. That is contextual-orchestrator#1045's "request-scoped candidate exclusion" RED, now with the evidence it asked for (posted there). Standing down on this check; no re-run while the served set contains a route that stalls on every contact. The next base-merge push carries the current sidecar pin and re-runs it. Generated by Claude Code |
|
CI note on head Nothing in this diff is involved, and no push or re-run from this side changes the outcome — a re-run re-dispatches into the same rejected gate, and the dispatch workflow re-runs this exact job by itself once a verdict is published. Standing down on this check. Current head state: Generated by Claude Code |
… size contract) Signature 3: the first post-#1939 noema-review runs split 1/1; the failing run's policy report shows the diversified pool #1939 promised and still ends in a 502 after a ~548 s-per-route walk (host 1's arithmetic from source), so a base-merge push recovers pre-#1939 heads but does not shorten the post-#1939 walk; #1943/#1944/#1945 make the per-route timeline readable from the noema-sidecar-evidence artifact; the remaining lever is inside contextual-orchestrator. Signature 12: required-workflow-bootstrap exit 2 in ~5 s on "exceeds the size contract" -- the Contents API's 1 MiB inline ceiling on a patchless text file, fixed by #1946's Git Blobs API route. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
![opencode-agent[bot]](https://avatars.githubusercontent.com/in/1549082?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
OpenCode reviewed the current-head mergeability evidence and changed-file flow before approval, then found merge conflicts on the affected path.
Findings
1. HIGH Merge Conflict Guidance - Resolve the PR branch against the latest base branch
- Problem: GitHub reports mergeStateStatus
DIRTYfor this pull request. - Root cause: Branch
lane-jan/pingora-oversized-blob-evidencecannot be merged cleanly intomain; the changed-file flow below shows which review/runtime path is blocked by the conflict. - Fix: Merge or rebase the latest
mainintolane-jan/pingora-oversized-blob-evidence, resolve conflict markers in the PR branch, rerun the focused checks, and push the same branch. - Repair commands:
gh pr checkout 1946 --repo ContextualWisdomLab/.github
git fetch origin main
git merge --no-ff origin/main # or: git rebase origin/main
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:lane-jan/pingora-oversized-blob-evidence
# rebase path only: git push --force-with-lease origin HEAD:lane-jan/pingora-oversized-blob-evidence- Regression test: Keep OpenCode approval gated on mergeability so model-output failures cannot approve a conflicted PR.
Merge Conflict Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Repository file: CHANGELOG.md"]
S1 --> I1["repository behavior"]
I1 --> Conflict["Merge conflict blocks this path"]
Conflict --> V1["required checks"]
Evidence --> S2["Docs: PINGORA_EDGE_POLICY.md"]
S2 --> I2["operator or user guidance"]
I2 --> Conflict["Merge conflict blocks this path"]
Conflict --> V2["docs review"]
Evidence --> S3["CI script: pingora_edge_policy.py"]
S3 --> I3["review and security gate shell path"]
I3 --> Conflict["Merge conflict blocks this path"]
Conflict --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test: test_pingora_edge_policy.py"]
S4 --> I4["regression suite"]
I4 --> Conflict["Merge conflict blocks this path"]
Conflict --> V4["targeted test run"]
- Result: REQUEST_CHANGES
- Reason: mergeStateStatus is
DIRTY; mergeable isCONFLICTING. - Head SHA:
db361daa819ef31cf99e5cc5431405218abe6f2a - Workflow run: 33998627665
- Workflow attempt: 1
Changed-File Evidence Map
flowchart LR
PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
Evidence --> S1["Repository file: CHANGELOG.md"]
S1 --> I1["repository behavior"]
I1 --> Conflict["Merge conflict blocks this path"]
Conflict --> V1["required checks"]
Evidence --> S2["Docs: PINGORA_EDGE_POLICY.md"]
S2 --> I2["operator or user guidance"]
I2 --> Conflict["Merge conflict blocks this path"]
Conflict --> V2["docs review"]
Evidence --> S3["CI script: pingora_edge_policy.py"]
S3 --> I3["review and security gate shell path"]
I3 --> Conflict["Merge conflict blocks this path"]
Conflict --> V3["bash -n plus Strix self-test"]
Evidence --> S4["Test: test_pingora_edge_policy.py"]
S4 --> I4["regression suite"]
I4 --> Conflict["Merge conflict blocks this path"]
Conflict --> V4["targeted test run"]
OpenCode Review Overview
|
…ence CHANGELOG.md: both new top entries kept, main's pin-bump entry first. Binds the sidecar pin bump to contextual-orchestrator@414f2297 (contextual-orchestrator#1081's retry-stacking fix, .github efb8926) and the #1943/#1944/#1945 workflows to this head's required runs; the previous head's Noema/Strix runs failed on the stalled-route loop that fix removes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
|
CI note and push: head Merges Local gate on Generated by Claude Code |
2 participants
Summary
required-workflow-bootstrapfails closed in about 5 s withPingora edge policy could not establish complete evidence: GitHub content evidence for docs/sbom/inventory.json exceeds the size contracton #1678 (run33989047645, and every push of that automation branch). The mechanism is deterministic and has nothing to do with Nginx:docs/sbom/inventory.jsonat 1,148,611 bytes (236 bytes onmain).patchfor a file that large, so_needs_content_scanreturnsTrueon patch absence (.jsonis neither a documentation suffix nor a binary document format).encoding: "none"with an accuratesizefor anything over its 1 MiB inline ceiling, and_load_raw_file_bytesturned that intoContentSizeExceededError, which only the documentation-PDF path knows how to absorb. Everything else fails the whole check closed.main's ownscan_content, the file carries zero Nginx runtime forms (it does not even contain the stringnginx).So the SBOM inventory could never pass the required policy, and the same would hold for any PR that adds a text file over 1 MiB.
Change
scripts/ci/pingora_edge_policy.py_resolve_oversized_contentfollows a Contents response that could not inline the file to its blobshaand fetches the same blob through the Git Blobs API (/repos/{repo}/git/blobs/{sha}, already inside_validate_github_api_url's origin and/repos/path rule, samecontents: readpermission the Contents call needs)._load_oversized_blob_bytesbinds every blob field back to the Contents metadata it was reached from: the responseshamust be the one requested, itssizemust equal the declared size, and the decoded bytes must have exactly that length; any other shape is a malformed-evidencePolicyErrorthat fails closed, exactly as for an inline response.MAX_BLOB_BYTES = 11 MiB, chosen so a 60-column-wrapped base64 blob response plus its JSON envelope fits_github_open_json's existing 16 MiBMAX_RESPONSE_BYTESbound with margin;test_blob_ceiling_fits_the_bounded_responsepins the arithmetic.ContentSizeExceededErrorkeeps its narrow meaning for the remainder: a declared size over the blob ceiling, or a response with no well-formed blob sha to follow. The oversized-documentation-PDF suffix convention therefore now applies only there, and a.pdfwithin the blob ceiling is verified by its%PDF-magic bytes instead of trusted on its suffix (strictly stronger than before).docs/policies/PINGORA_EDGE_POLICY.mdandCHANGELOG.mddescribe the new bound.Verification
tests/test_pingora_edge_policy.py: 18 new tests — the chore: refresh org SBOM inventory #1678 shape end to end (oversized patchlessdocs/sbom/inventory.json, clean →(), withFROM nginxappended →nginx_container_imageat line 2, proving it is scanned rather than exempted); an oversized real PDF passes on magic bytes and a textual file named.pdfis rejected; a PDF overMAX_BLOB_BYTESstill uses the suffix convention with no blob request; the olderbase64-with-empty-content oversized shape takes the same route; five malformed/missing-sha shapes keep the size-contract signal without any blob request; seven malformed blob responses fail closed as plainPolicyError, never asContentSizeExceededError; the ceiling arithmetic contract. Every pre-existing test is unchanged and passes.db361daa8withGITHUB_ACTIONS=true: 2921 passed, 1 skipped,coverage report --fail-under=100→ 100%,interrogate→ 100%.scan_content("docs/sbom/inventory.json", <#1678 head content>)→ 0 violations, 1,148,611 bytes.Chicken-and-egg note:
required-workflow-bootstrapruns the base branch'spingora_edge_policy.pyat the immutable required-workflow SHA, so this PR's own contexts cannot exercise the change; #1678 (or any oversized-text PR) will, once this is onmain.Developer experience
Two small helpers with docstrings that state the binding rules and why the ceiling is 11 MiB; no new dependency, no change to the opener contract, no change to how any inline file is handled.
User experience
None visible to PR authors of ordinary files. Authors of a PR that adds or regenerates a text file over 1 MiB get a real content verdict instead of an unconditional "exceeds the size contract" failure; the SBOM inventory automation becomes mergeable on its merits.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4
Generated by Claude Code