fix(wordpress): treat REST API 404 as non-retryable - #68121
Merged
Conversation
A 404 from a wp/v2 collection means that endpoint is disabled or blocked on the customer's site (security plugins commonly block /users to prevent user enumeration). The base URL is already validated against /posts at setup, so a per-endpoint 404 at sync time is upstream config, not a transport blip. Retrying is pointless, so surface an actionable message and stop the schema from retrying. Generated-By: PostHog Code Task-Id: 8f191cf8-87a9-47dc-9ea8-248890968430
Contributor
|
Hey @Gilbert09! 👋 It looks like your git author email on this PR isn't your
You can fix it for this repo with: git config user.email "you@posthog.com"Or set it globally with |
Contributor
|
Reviews (1): Last reviewed commit: "fix(wordpress): treat REST API 404 as no..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The WordPress data warehouse source raises an uncaught
HTTPErrorwhen awp/v2collection returns 404. Seen in error tracking on the/usersendpoint:It originates in
wordpress.pyfetch_pageatresponse.raise_for_status(), so the sync fails and Temporal retries it repeatedly — burning retries and re-surfacing as error-tracking noise — for a condition that will never resolve on its own.A per-endpoint 404 here is a customer-side config issue, not our bug. The base site URL is already validated against
/postsat source setup, so by sync time the REST root is known-good. A 404 on a specific collection means that collection is disabled or blocked on the site. This is very common for/wp-json/wp/v2/users, which security plugins and managed hosts routinely block to prevent user enumeration.Error tracking issue: https://us.posthog.com/project/2/error_tracking/019f26df-2192-7bf0-8c1f-7323cef731f5
Changes
Add
404 Client ErrortoWordpressSource.get_non_retryable_errors, mirroring the existing 401/403 entries. The sync stops retrying, disables the affected schema, and surfaces an actionable message pointing at the blocked endpoint (and how to fix it or drop the table from the sync) instead of a raw stack trace.Scoped strictly to this: the substring
404 Client Erroris stable and carries no volatile URL, id, or timestamp. It disables only the failing schema, not the whole source, so other collections keep syncing.How did you test this code?
Automated tests only (agent — no manual sync performed):
TestWordpressSource::test_non_retryable_errorswith a404 Client Errorcase. It guards the regression where someone drops the 404 mapping and WordPress imports against a site that blocks a collection retry indefinitely. Extended rather than added a new function since the assertion is identical to the 401/403 cases.ruff check/ruff formaton the two changed files.Automatic notifications
Docs update
No docs change needed. The in-product source caption is unchanged and still accurate; this only affects the runtime error message on a failed sync.
🤖 Agent context
Autonomy: Fully autonomous
Authored by Claude Code (Opus 4.8) from an error-tracking webhook. Invoked the
/writing-testsskill before touching the test file to apply the value gate — the change is a single parametrized case, not a new test.Decision: I weighed a graceful per-endpoint skip vs. marking 404 non-retryable, and chose non-retryable. A 404 is deterministic (retrying can't help) and, since the REST root is already validated at setup, a sync-time 404 is a blocked/disabled collection — an upstream config issue best surfaced to the user with the schema disabled, exactly like the existing 401/403 handling. Checked against open PR #68046, which only reworks the 403 messaging and does not touch 404, so this is not a duplicate.