fix(integrations): classify hubspot BAD_HUB and 429 oauth refresh failures - #72609
Merged
Conversation
… failures
HubSpot reports a grant whose portal was deleted or disconnected as
{"status": "BAD_HUB", "error": "access_denied"} - no invalid_grant code, so
these rows bucket as 'other', never hit the invalid_grant terminal streak,
and are retried at the backoff cap forever. The cap synchronises that
zombie cohort into hourly retry herds that trip HubSpot's per-second rate
limit, which also fails healthy refreshes landing in the same second.
Map the BAD_HUB shape to invalid_grant for hubspot (mirroring the existing
reddit-ads special case) so dead hubs go terminal and the herd drains, and
bucket HTTP 429 as a new 'rate_limited' reason for all kinds so throttling
reads as transient provider pressure rather than an unclassified failure.
The cdp alerting split (PostHog/charts#13344) treats rate_limited as
provider-transient alongside http_5xx/network.
mayteio
marked this pull request as ready for review
July 21, 2026 14:55
Contributor
|
Reviews (1): Last reviewed commit: "fix(integrations): classify hubspot BAD_..." | Re-trigger Greptile |
![stamphog[bot]](https://avatars.githubusercontent.com/in/3000332?s=60&v=4){kind=small}
There was a problem hiding this comment.
The "auth" title flag is incidental — this only reclassifies OAuth refresh failure reasons for metrics/retry-termination logic, not authentication or credential handling itself; the diff is small, well-tested with parameterized cases, and the author has STRONG familiarity with this exact code path.
- Author wrote 100% of the modified lines and has 7 merged PRs in these paths (familiarity STRONG).
- 👍 on the PR from greptile-apps[bot], hex-security-app[bot].
Gate mechanics and policy version
| Gate | Result | |
|---|---|---|
| prerequisites | ✓ | all clear |
| deny-list | ✓ | no deny categories matched |
| size | ✓ | 16L, 1F substantive, 40L/2F incl. docs/generated/snapshots — within ceiling |
| tier | ✓ | T1-agent / T1b-small (40L, 2F, single-area, fix) |
| stamphog 2.0.0b3 | .stamphog/policy.yml @ cb180e3 · reviewed head 277802a |
mayteio
enabled auto-merge (squash)
July 21, 2026 15:01
Contributor
🤖 CI report |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
integration_oauth_refreshmetric buckets failures byreasonso alerting can tell credential breakage from provider trouble, and so dead grants can go terminal (an unbrokeninvalid_grantstreak) instead of being retried forever.HubSpot breaks both mechanisms for one failure shape. A grant whose portal (hub) was deleted or disconnected comes back as
{"status": "BAD_HUB", "error": "access_denied", ...}with noinvalid_grantcode, so it buckets asotherand never terminates. Those rows retry at the backoff cap forever, and because the cap is a fixed interval they synchronize into periodic retry herds. The herds are big enough to trip HubSpot's per-second rate limit, which then also fails healthy refreshes that land in the same second, and those 429s bucket asothertoo.Changes
BAD_HUBshape (4xx +"status": "BAD_HUB") toinvalid_grant, mirroring the existing reddit-ads special case for a dead grant reported without an OAuth error code. Dead hubs now go terminal after the usual streak and stop retrying. HubSpot'sBAD_REFRESH_TOKENresponses already carry"error": "invalid_grant"and need no special case.rate_limitedreason for all kinds, checked after the credential shapes and before the 5xx bucket, so throttling reads as transient provider pressure rather than an unclassified failure. Downstream alerting (PostHog/charts#13344) treatsrate_limitedas provider-transient alongsidehttp_5xx/network.Note
Existing
BAD_HUBrows start terminating only as they fail again after this deploys; the backlog drains over the following hours as each row accumulates its streak.How did you test this code?
test_oauth_refresh_failure_reasonwith six cases; regressions each group catches:hubspot_dead_hub_shape- the core mapping; fails without the new branch.hubspot_shape_on_other_kind- the mapping must stay scoped to hubspot, like the reddit case.hubspot_bad_hub_5xx_is_outage- a 5xx with a weird body is still an outage, not a dead grant.hubspot_bad_refresh_token_still_oauth_code- the standardinvalid_grantpath keeps precedence.rate_limited/rate_limited_any_kind- 429 buckets independently of body shape and kind.hogli test posthog/models/test/test_integration_model.py -- -k refresh- 68 passed (covers backoff, terminal-state, and streak-reset behavior around the changed function).Automatic notifications
Docs update
None needed - internal metric classification only.
🤖 Agent context
Autonomy: Human-driven (agent-assisted)
Authored with Claude Code. The failure shapes were taken verbatim from production refresh-failure logs (Loki) while investigating why hubspot's
otherfailure bursts recur on a fixed period; the burst timing matched the backoff cap and the in-burst failures were RATE_LIMIT bodies, which is what motivated the separaterate_limitedbucket rather than folding 429s intootherorhttp_5xx. Considered gatingBAD_HUBon exact status 400 like the reddit case, but chose< 500since the logged shape is unambiguous and a provider-side 5xx that happens to echo a body should still bucket as an outage (covered by a test). Tests follow the file's existing parameterized pattern.