Skip to content

Fixed IndexNow pinging /404/ for posts with no web URL#28311

Merged
ErisDS merged 1 commit into
mainfrom
indexnow-skip-unresolved-url
Jun 2, 2026
Merged

Fixed IndexNow pinging /404/ for posts with no web URL#28311
ErisDS merged 1 commit into
mainfrom
indexnow-skip-unresolved-url

Conversation

@ErisDS
Copy link
Copy Markdown
Member

@ErisDS ErisDS commented Jun 2, 2026

What

IndexNow (behind the indexnow labs flag) pings api.indexnow.org when a post is published or edited, resolving the post URL via urlService.facade.getUrlForResource(...).

That returns a /404/ URL when a published post is not owned by any route — e.g. an imported or members post that falls outside the site's collections, or a routing config with no catch-all. In that case the ping submitted https://<site>/404/ instead of a post URL. This was seen in production on a site whose unrouted posts produced most of its IndexNow pings.

Change

After resolving the URL, if it's empty or ends in /404/, skip the ping and log an indexnow.unresolved_url event instead. Other behaviour is unchanged.

Notes

  • A published post with no web URL is usually a site routing/content issue; the indexnow.unresolved_url event makes it observable.
  • getUrlForResource returns /404/ as its not-found sentinel (see url-service.js), so matching it identifies an unrouted resource.

Testing

Added a unit test asserting no ping and an indexnow.unresolved_url event (with post_id/post_slug/url) when the URL resolves to /404/; existing ping tests stub a resolved URL. 22 unit tests pass; lint clean.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Jun 2, 2026

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 86260567-68c8-4937-9992-7a91794221fc

📥 Commits

Reviewing files that changed from the base of the PR and between 44652f9 and e3368ae.

📒 Files selected for processing (2)
  • ghost/core/core/server/services/indexnow.js
  • ghost/core/test/unit/server/services/indexnow.test.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • ghost/core/test/unit/server/services/indexnow.test.js

Walkthrough

This PR adds an early validation guard to the ping() function in the IndexNow service that prevents sending pings to search engines when a post URL cannot be properly resolved. When a resolved URL is missing or ends with /404/, the function now logs a structured warning event (indexnow.unresolved_url) containing the post ID/slug and the invalid URL, then exits before attempting the IndexNow API request. Test setup was updated to stub the URL resolver to return a valid URL by default, and a new test case verifies the early exit behavior for the /404/ scenario.

Possibly related PRs

  • TryGhost/Ghost#28295: Introduces structured system.event logging scheme used by this PR's indexnow.unresolved_url warning events.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately describes the main change: preventing IndexNow from pinging /404/ URLs for posts with no routable web URL.
Description check ✅ Passed The description clearly explains the problem, the change made, and testing approach, all directly related to the changeset.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch indexnow-skip-unresolved-url

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@ghost/core/test/unit/server/services/indexnow.test.js`:
- Around line 289-290: The test currently only asserts loggingStub was called
once and that loggingStub.args[0][0].system.event === 'indexnow.unresolved_url';
expand the assertion to validate the full structured warning payload by
asserting the presence and expected values/types of
loggingStub.args[0][0].system.event, loggingStub.args[0][0].context.post_id,
loggingStub.args[0][0].context.post_slug, and loggingStub.args[0][0].context.url
(use the same expected post id/slug/url values used to trigger the warning in
the test); update the assertions around loggingStub in indexnow.test.js to check
these fields in addition to the event to prevent regressions in the log
contract.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c6eb6f68-bd6f-4573-888f-96671336eb15

📥 Commits

Reviewing files that changed from the base of the PR and between 79a3b46 and 44652f9.

📒 Files selected for processing (2)
  • ghost/core/core/server/services/indexnow.js
  • ghost/core/test/unit/server/services/indexnow.test.js

Comment thread ghost/core/test/unit/server/services/indexnow.test.js Outdated
ref #28295
On publish, IndexNow resolves the post URL via getUrlForResource, which
returns a /404/ URL when a published post is not owned by any route
(e.g. an imported or members post outside the site's collections). The
ping then submitted the /404/ URL instead of a post URL.

Guard the ping: when the resolved URL is empty or a /404/ URL, skip it
and log an indexnow.unresolved_url event. Behaviour is otherwise
unchanged.
@ErisDS ErisDS force-pushed the indexnow-skip-unresolved-url branch from 44652f9 to e3368ae Compare June 2, 2026 19:07
@ErisDS
Copy link
Copy Markdown
Member Author

ErisDS commented Jun 2, 2026

Addressed CodeRabbit's suggestion: the /404/ skip test now asserts the full structured payload (system.event, system.post_id, system.post_slug, system.url), not just the event name.

@ErisDS ErisDS merged commit 96118fb into main Jun 2, 2026
50 checks passed
@ErisDS ErisDS deleted the indexnow-skip-unresolved-url branch June 2, 2026 19:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant