Skip to content

Context expansion: fall back to raw.githubusercontent.com for tokenless public file fetches #11

Description

@hsyntax

Context

Hunk context expansion (GitHub-style "expand lines above/below" arrows) will fetch full file contents through a loadDiffFiles loader passed to @pierre/diffs. The straightforward implementation fetches every file through the contents API:

GET https://api.github.com/repos/{owner}/{repo}/contents/{path}?ref={sha}
Accept: application/vnd.github.raw

That endpoint counts against the unauthenticated api.github.com rate limit of 60 requests/hour/IP. A tokenless user reviewing a public PR spends 2 requests per expanded file (old + new side) plus the ref-resolution requests, so a moderately sized review can exhaust the budget quickly — and the same budget is shared with diff loading itself.

Proposal

When no token is saved and the repository is public, fetch file contents from raw.githubusercontent.com instead:

GET https://raw.githubusercontent.com/{owner}/{repo}/{ref}/{path}
  • Not subject to the 60/hr core API limit and serves CORS (Access-Control-Allow-Origin: *), so it works from the browser.
  • Keep the contents-API path (with Authorization header) whenever a token is saved, since raw.githubusercontent.com does not serve private repositories that way and the token flow already works.

Pierre's reference implementation does exactly this split — see repos/pierre/apps/diffshub/lib/githubDiffFileServer.ts (fetchGitHubFileContents): contents API when a request token is present, raw.githubusercontent.com otherwise.

Notes

  • The README currently promises the GitHub token is "sent only to api.github.com". The fallback only fires on the tokenless path so the promise technically holds, but the wording should be revisited to mention raw.githubusercontent.com as a GitHub-operated host diffdump talks to.
  • Ref-resolution requests (pulls / compare / commits endpoints) still go through api.github.com; only the file-content fetches can move.
  • raw.githubusercontent.com resolves by ref, so the loader must pass the exact commit SHA (merge base / head) it resolved, same as the contents-API path.

Depends on

Context expansion (loadDiffFiles) landing first — this is a follow-up to that work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions