fix(data-imports): shrink Anthropic usage report page size to avoid 400

[Gilbert09]

Problem

The Anthropic data warehouse source is failing on the messages usage report with a 400 Client Error: Bad Request from /v1/organizations/usage_report/messages. It surfaced in error tracking firing across several teams' syncs at once (issue 019f61a0-5d7f-7091-889a-0304928cbd3e), raised from _fetch_page in the Anthropic source.

The request itself is well-formed: every group_by[] value we send is documented as valid, bucket_width=1d is valid, and starting_at is a valid RFC 3339 timestamp. So this is not a credential problem (those come back as 401/403 and are already treated as non-retryable) and not a single bad param.

The distinguishing factor is response size. usage_report groups by all eight dimensions and asks for limit=31, the maximum number of daily buckets. Grouping by every dimension multiplies the result rows per bucket, and 31 dense historical buckets per page pushes the response past the report's per-response result cap, which the API rejects with a 400. This bites on the initial full-refresh backfill (starting_at = the Anthropic launch date), where every page is packed with historical buckets. Incremental runs return only a handful of buckets regardless of limit, so they stay under the cap. cost_report, which groups by two low-cardinality dimensions and uses the API default page size, is unaffected.

Anthropic's own guidance for this 400 is to reduce the number of group_by dimensions or narrow the time range / bucket count. Pagination on this endpoint is per bucket via a next_page cursor, so the cap is per response.

Changes

Drop the usage_report page size from the 31-bucket max to the API default of 7. Each response now carries fewer buckets and stays under the result cap, while pagination via next_page still walks all history and every group_by dimension is preserved. This mirrors what cost_report already does.

-        limit=31,  # API max for the 1d bucket width
+        # Grouping by every dimension multiplies the result rows per bucket, so requesting the
+        # 31-bucket max overflows the report's per-response result cap and the API 400s. The API
+        # default keeps each page small; pagination still walks all history via `next_page`.
+        limit=7,

This is the least-lossy fix available: it changes only pagination granularity, never the data returned. Very large organizations could in theory still exceed the cap even at 7 buckets, in which case a follow-up would trim the highest-cardinality group_by dimensions, but that permanently drops breakdown fidelity so it is not worth doing preemptively.

How did you test this code?

Automated tests only (I am Claude, an agent; I did not run a live backfill against the Anthropic API).

• Updated TestReportParams.test_incremental_uses_watermark_as_starting_at to expect the new page size.
• Added TestReportParams.test_usage_report_page_size_stays_below_bucket_max, which asserts the endpoint still groups by the full eight-dimension set while capping the page size. This catches the specific regression of someone bumping limit back to the 31-bucket max under the full group_by, which is what triggers the 400.
• Ran the full Anthropic source suite: pytest .../sources/anthropic/tests/test_anthropic.py — 28 passed. Ran ruff check and ruff format --check on both changed files — clean.

🤖 Agent context

Autonomy: Fully autonomous

Triaged from the error-tracking issue above. I (Claude) pulled the stack trace and representative events via the PostHog error-tracking tools, read the Anthropic source under products/warehouse_sources/backend/temporal/data_imports/sources/anthropic/, and cross-checked the request against Anthropic's current Usage & Cost Admin API docs to confirm every param we send is individually valid.

Decision: classified this as a fixable request-construction bug rather than a user/upstream error. A 400 on a request we fully control, firing for every affected org at once, is our bug, not a per-customer credential or data issue, so extending NonRetryableErrors would have masked it and permanently disabled the syncs. I considered trimming the group_by set instead, but reducing the page size is strictly less lossy (no dimension is dropped) and is one of Anthropic's recommended remedies for this error. Verified no open PR already covers this: #70811 touches the same source but handles 429 Retry-After in anthropic.py, a different root cause and different file.

---

Created with PostHog Code

[github-actions]

Hey @Gilbert09! 👋

It looks like your git author email on this PR isn't your @posthog.com address (owerstom@gmail.com). Since you're on the PostHog team, it's worth pointing your local git author email at your @posthog.com address. Why it matters:

• Consistent work identity in git history — internal tooling that attributes commits to team members keys off your @posthog.com address.
• Keeps team contributions easy to tell apart from external community ones when scanning history.

You can fix it for this repo with:

git config user.email "you@posthog.com"

Or set it globally with git config --global user.email "you@posthog.com". No need to redo this PR — just a nudge for next time. 🙂

[stamphog]

Trivial, well-tested pagination limit fix by an owning-team author with strong familiarity; no risky territory touched.

• Author wrote 100% of the modified lines and has 3 merged PRs in these paths (familiarity STRONG).
• 👍 on the PR from hex-security-app[bot].

Gate mechanics and policy version

Gate		Result
prerequisites	✓	all clear
deny-list	✓	no deny categories matched
size	✓	5L, 1F substantive, 16L/2F incl. docs/generated/snapshots — within ceiling
tier	✓	T1-agent / T1a-trivial (16L, 2F, single-area, fix)
stamphog 2.0.0b3		.stamphog/policy.yml @ 4a9b203 · reviewed head 56cf70e

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-14 18:35 UTC	Run
prod-us	✅ Deployed	2026-07-14 18:52 UTC	Run
prod-eu	✅ Deployed	2026-07-14 18:50 UTC	Run