fix(warehouse): stop GitHub sync gracefully at deep-pagination cap

[posthog]

Problem

The GitHub warehouse source paginates stargazers oldest-first (sort=created&direction=asc). On a big repo, once the sync pages past GitHub's deep-pagination cap, GitHub answers with a 403 Forbidden whose body is a pagination-limit message ("pagination is limited for this resource") and carries no rate-limit markers.

raise_if_github_rate_limited only treats a 403 as retryable when it sees a retry-after header, x-ratelimit-remaining: 0, or a rate-limit / abuse-detection body. This 403 matches none, so it fell through to response.raise_for_status(), and the resulting HTTPError became a fatal NonRetryableException that killed the whole stargazers sync. A benign end-of-traversable-data response was being treated as a hard error.

This is newly shipped code, so it will recur for any repo whose stargazer count crosses the pagination cap, silently failing that table's import.

Changes

Recognize the pagination-cap 403 the same way the code already handles the empty-repository 409:

• _is_pagination_limit_response detects the 403 pagination-limit body.
• _fetch_page raises a GithubPaginationLimitError control-flow signal when it sees one (after the rate-limit check, before raise_for_status).
• Both the direct page loop (get_rows) and the shared fan-out paginator (_iter_pages) catch it and stop paginating, keeping the rows gathered so far.

A genuine permission 403 still carries no pagination body, so it stays fatal.

How did you test this code?

I (Claude) added and ran automated unit tests. The whole test_github_source.py file passes (138 tests).

• TestIsPaginationLimitResponse — parameterized detector cases: catches a regression where the pagination-cap 403 stops being recognized (and a permission or rate-limit 403 gets wrongly swallowed as a boundary). No existing test exercised this detector.
• test_pagination_limit_403_syncs_rows_gathered_so_far — the reported bug end to end: page 1 rows are kept and the cap on page 2 ends the loop without raising, where previously it raised the fatal HTTPError.
• test_pagination_limit_403_ends_iteration_without_raising — the same boundary through the shared _iter_pages fan-out path (workflow runs/jobs), which also calls _fetch_page.

I did not run the full sync against a live GitHub repo; verification is via the unit tests above.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

🤖 Agent context

Autonomy: Fully autonomous

Authored by Claude Code (Opus 4.8) from an inbox report about the stargazers sync failure. I confirmed the root cause against the error tracking issue: the captured exception is 403 Client Error: Forbidden for url: https://api.github.com/repos/posthog/posthog/stargazers?per_page=100&state=all&sort=created&direction=asc, with the stack running _fetch_page → get_rows → raise_for_status → NonRetryableException — exactly the path fixed here.

I mirrored the existing empty-repository 409 pattern (detector + control-flow exception + graceful loop break) rather than inventing a new mechanism, so the two benign-boundary cases read the same way. I invoked the /writing-tests skill before adding tests to keep them aimed at real regressions.

---

Created with PostHog Code from an inbox report.

[scheduled-actions-posthog]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, please remove the stale label – otherwise this will be closed in another week. If you want to permanently keep it open, use the waiting label.

[scheduled-actions-posthog]

This PR was closed due to lack of activity. Feel free to reopen if it's still relevant.