Skip to content

Fixed lazy URL shadow-comparison errors from thin resources - #29126

Merged
allouis merged 3 commits into
mainfrom
hkg-1822-lazy-url-thin-resources
Jul 8, 2026
Merged

Fixed lazy URL shadow-comparison errors from thin resources#29126
allouis merged 3 commits into
mainfrom
hkg-1822-lazy-url-thin-resources

Conversation

@allouis

@allouis allouis commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

ref https://linear.app/ghost/issue/HKG-1822/monitor-shadow-comparison-and-address-mismatches

Problem

Production is logging ~25k LAZY_URL_COMPARE_ERROR ("Lazy URL service threw during comparison") events across 169 sites. The error details all have the shape:

{"resourceType":"posts","resourceId":"...","baseFilter":"status:published+type:post","missing":["status"]}

i.e. the lazy backend's _assertBaseFieldsPresent rejecting a resource that reaches getUrlForResource without the status column, wrapped by the compare-mode facade. Eager stays authoritative so there's no user impact, but the noise drowns out real parity signals.

Two call sites were handing the URL service thin resources:

  1. Output serializer forPost — computed attrs.url for every post/page API response and only deleted it afterwards when ?fields=/columns excluded url. With narrowed fields the attrs carry no status (or tags/authors), so the lazy shadow threw on every such request. forceUrlColumnsWhenLazy only covers the case where url is requested.
  2. Feedback redirect endpoint (/members/feedback/:postId/:score) — built its redirect from a bare {id: postId} resource, so every newsletter feedback-button click threw in the lazy shadow. This explains the recurring per-post counts in the logs.

A sweep of the remaining getUrlForResource/ownsResource/resolveUrl callers also turned up a staff-users gap: ?fields=url on users browse/read strips slug, so lazy generates /author/undefined/ — a LAZY_URL_PARITY_MISMATCH rather than a throw.

Solution

  • forPost now skips URL generation entirely when url was not requested, matching the guards forUser/forTag already have (also saves wasted work under eager).
  • The feedback redirect controller loads the full post (withRelated: tags, authors) before building the link, and uses a new buildFallbackLink (home page + feedback fragment, no URL service involved) when the post no longer exists — same destination buildLink picks when the URL service returns /404/.
  • The users input serializer forces the lazy-required author columns into the fetch when url is requested, mirroring the authors serializers.

All other getUrlForResource callers (slack, indexnow, comments emails, member attribution, activity feed, RSS, previews, email-post, meta/url, theme helpers) already pass fully-loaded models.

Testing

  • Red/green TDD: each fix has unit tests that failed against the old behaviour.
  • New e2e coverage for the feedback redirect endpoint (302 → post URL, 302 → home for a deleted post) — it previously had none.
  • Full ghost/core unit suite (7180 tests) and the members feedback e2e suite pass.

🤖 Generated with Claude Code

@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Walkthrough

This PR adds fallback redirect handling for audience feedback when a target post no longer exists: getPostById accepts options, AudienceFeedbackService gains buildFallbackLink and a refactored hash builder, and redirectToPost becomes async, fetching the full post with related tags/authors and choosing between buildLink and buildFallbackLink. Separately, the users input serializer gains a browse handler and updates read to force lazy URL columns for authors, while the output URL utility short-circuits URL computation when columns excludes url. Corresponding unit and e2e tests are added/updated.

Possibly related PRs

  • TryGhost/Ghost#28890: Introduces the forceUrlColumnsWhenLazy/getRequiredFields mechanism that the users input serializer changes here directly build upon.

Suggested reviewers: vershwal, kevinansfield

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly matches the main change: fixing lazy URL comparison errors caused by thin resources.
Description check ✅ Passed The description is directly related to the PR and accurately summarizes the lazy URL fixes and tests.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch hkg-1822-lazy-url-thin-resources

Comment @coderabbitai help to get the list of available commands.

@nx-cloud

nx-cloud Bot commented Jul 6, 2026

Copy link
Copy Markdown

🤖 Nx Cloud AI Fix

Ensure the fix-ci command is configured to always run in your CI pipeline to get automatic fixes in future runs. For more information, please see https://nx.dev/ci/features/self-healing-ci


View your CI Pipeline Execution ↗ for commit ecb4ceb

Command Status Duration Result
nx run ghost:test:ci:integration ✅ Succeeded 4s View ↗
nx run ghost:test:integration ✅ Succeeded 2m 48s View ↗
nx run ghost:test:legacy ✅ Succeeded 2m 10s View ↗
nx run ghost:test:e2e ✅ Succeeded 2m 25s View ↗
nx run-many --target=build --projects=tag:publi... ✅ Succeeded 1s View ↗
nx run @tryghost/admin:build ✅ Succeeded 6s View ↗
nx run ghost:build:assets ✅ Succeeded 2s View ↗
nx run-many -t test:unit -p ghost ✅ Succeeded 2s View ↗
Additional runs (2) ✅ Succeeded ... View ↗

💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗


☁️ Nx Cloud last updated this comment at 2026-07-06 07:38:41 UTC

allouis added 3 commits July 6, 2026 07:30
… field

ref https://linear.app/ghost/issue/HKG-1822

A request like ?fields=id,title strips post attrs down to the requested
columns, but forPost still computed the URL and only deleted it again at the
end of the function. Under lazyRouting shadow comparison the stripped
resource carries no status (or tags/authors), so the lazy backend rejected it
as thin and every such request logged a LAZY_URL_COMPARE_ERROR. Skipping URL
generation when url was not requested matches the guards forUser and forTag
already have, and saves wasted work under the eager service too.
ref https://linear.app/ghost/issue/HKG-1822

The newsletter feedback button endpoint (/members/feedback/:postId/:score)
built its redirect from a bare {id} resource. The eager service looks URLs up
by id so this worked, but the lazy backend needs status (and tags/authors for
filtered routes) and rejects the resource as thin, so under shadow comparison
every feedback click logged a LAZY_URL_COMPARE_ERROR. The controller now
loads the full post with its tags and authors before building the link, and
falls back to the home-page link without touching the URL service when the
post no longer exists.
ref https://linear.app/ghost/issue/HKG-1822

A users browse or read with ?fields=url strips the permalink columns (slug),
so the lazy URL service generated /author/undefined/ while eager returned the
real URL, showing up as a parity mismatch under shadow comparison. Staff
users route through the authors router types, so this mirrors the
forceUrlColumnsWhenLazy wiring the authors serializers already have.
@allouis
allouis force-pushed the hkg-1822-lazy-url-thin-resources branch from f960747 to ecb4ceb Compare July 6, 2026 07:30
@allouis
allouis marked this pull request as ready for review July 6, 2026 07:31
@cursor

cursor Bot commented Jul 6, 2026

Copy link
Copy Markdown

Bugbot is not enabled for this team, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
ghost/core/core/server/api/endpoints/utils/serializers/output/utils/url.js (1)

15-23: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Consider extracting the shared "skip when url not requested" guard.

Per the comment, this exact guard (frame.options.columns && !frame.options.columns.includes('url')) is duplicated in forUser/forTag. Extracting a small shared helper (e.g. shouldComputeUrl(frame)) would reduce triplication and centralize the LAZY_URL_THIN_RESOURCE avoidance logic for future maintenance.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@ghost/core/core/server/api/endpoints/utils/serializers/output/utils/url.js`
around lines 15 - 23, Extract the duplicated “skip when url not requested” check
from the URL serializer utilities into a shared helper such as
shouldComputeUrl(frame) and use it in output/utils/url.js as well as the forUser
and forTag paths. Keep the existing behavior that returns attrs early when
frame.options.columns does not include url, but centralize the guard so the
LAZY_URL_THIN_RESOURCE avoidance logic is maintained in one place and all URL
computation entry points stay consistent.
ghost/core/core/server/services/audience-feedback/audience-feedback-service.js (1)

57-60: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

postId isn't URI-encoded in the hash, unlike uuid/key.

uuid and key are wrapped in encodeURIComponent, but postId is interpolated raw. postId can originate from req.params.postId (buildFallbackLink) or postData.id, so an unexpected value (e.g. containing /, ?, &) would alter the fragment structure the Portal SPA parses. This is pre-existing behavior extended by this refactor, not user-facing/exploitable server-side (fragments never leave the browser), but encoding it would remove the inconsistency and any parsing surprises.

🔒 Proposed fix
     `#withFeedbackHash`(url, postId, score, uuid, key) {
-        url.hash = `#/feedback/${postId}/${score}/?uuid=${encodeURIComponent(uuid)}&key=${encodeURIComponent(key)}`;
+        url.hash = `#/feedback/${encodeURIComponent(postId)}/${score}/?uuid=${encodeURIComponent(uuid)}&key=${encodeURIComponent(key)}`;
         return url;
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@ghost/core/core/server/services/audience-feedback/audience-feedback-service.js`
around lines 57 - 60, The `AudienceFeedbackService.#withFeedbackHash` method
interpolates `postId` into the fragment without encoding it, unlike `uuid` and
`key`. Update the hash construction in `#withFeedbackHash` so `postId` is passed
through `encodeURIComponent` before being inserted, keeping the fragment
structure consistent for values coming from `buildFallbackLink` or `postData.id`
and avoiding parsing surprises in the Portal SPA.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@ghost/core/core/server/api/endpoints/utils/serializers/output/utils/url.js`:
- Around line 15-23: Extract the duplicated “skip when url not requested” check
from the URL serializer utilities into a shared helper such as
shouldComputeUrl(frame) and use it in output/utils/url.js as well as the forUser
and forTag paths. Keep the existing behavior that returns attrs early when
frame.options.columns does not include url, but centralize the guard so the
LAZY_URL_THIN_RESOURCE avoidance logic is maintained in one place and all URL
computation entry points stay consistent.

In
`@ghost/core/core/server/services/audience-feedback/audience-feedback-service.js`:
- Around line 57-60: The `AudienceFeedbackService.#withFeedbackHash` method
interpolates `postId` into the fragment without encoding it, unlike `uuid` and
`key`. Update the hash construction in `#withFeedbackHash` so `postId` is passed
through `encodeURIComponent` before being inserted, keeping the fragment
structure consistent for values coming from `buildFallbackLink` or `postData.id`
and avoiding parsing surprises in the Portal SPA.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0235ebf1-e8bf-43b9-8317-e240e3da86d1

📥 Commits

Reviewing files that changed from the base of the PR and between 7d47672 and ecb4ceb.

📒 Files selected for processing (10)
  • ghost/core/core/server/api/endpoints/utils/serializers/input/users.js
  • ghost/core/core/server/api/endpoints/utils/serializers/output/utils/url.js
  • ghost/core/core/server/services/audience-feedback/audience-feedback-controller.js
  • ghost/core/core/server/services/audience-feedback/audience-feedback-service.js
  • ghost/core/core/server/services/audience-feedback/feedback-repository.js
  • ghost/core/test/e2e-api/members/feedback.test.js
  • ghost/core/test/unit/api/canary/utils/serializers/input/users.test.js
  • ghost/core/test/unit/api/canary/utils/serializers/output/utils/url.test.js
  • ghost/core/test/unit/server/services/audience-feedback/audience-feedback-controller.test.js
  • ghost/core/test/unit/server/services/audience-feedback/audience-feedback-service.test.js

@allouis
allouis requested a review from vershwal July 8, 2026 04:29
@allouis
allouis merged commit 158fd93 into main Jul 8, 2026
43 checks passed
@allouis
allouis deleted the hkg-1822-lazy-url-thin-resources branch July 8, 2026 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants