feat(plugin): status-aware result handling + redirect strikes — 404/410 die sooner, 401/403 and transient errors never suppress, perpetual redirects retire; v0.21.0

[harper-joseph]

What

An http-error verdict is not one thing, and neither is a redirect. This grades result handling by HTTP status and by persistence (plugin-only — the browser already posts statusCode, so no browser release or fleet roll is needed):

Result	Before	Now
404/410	suppress, recheck every 7d, delete after 4 strikes (~28 days of pointless re-renders)	suppress as http-gone, recheck at 14d, delete after 2 strikes (render.suppression.gone)
401/403	suppress + strike → mass-deletes healthy targets if the renderer credential or an origin access rule ever breaks	never suppress: keep target + cached page, reschedule at normal cadence, log at error
408/429/5xx	suppress for 7d, delete the last good cached page, strike toward deletion — over what may be one bad minute at the origin	never suppress: keep target + cached page, retry at normal cadence, no strike
temp redirect (302/303/307, client-side 200), or redirect onto an unserved route class	keep the source and retry forever — a permanent redirect wearing a temporary status costs a navigation every interval until the end of time	keep + retry as before, but each result counts a strike; render.redirects.maxStrikes consecutive ones retire the source (recordRedirectStrike). Any successful render clears strikes.
noindex / canonical-mismatch / redirect-loop (content verdicts)	default knobs	unchanged (classification keys on reason === 'http-error')

The same 401/403 rule applies to redirect destinations (processRedirectResult): an auth-shaped landing page no longer seeds a suppressed destination row.

Why retiring is safe

Deletion is self-healing by design: bot traffic for a retired URL proxies to the origin — which serves its own 404/redirect/page — and on-demand discovery re-creates whatever the origin actually serves. Retire-and-rediscover beats clever retention everywhere the origin has made a durable statement.

Prod motivation: ~600 dead product URLs (verified 404 with the render token, 200 control) are being suppressed via the generic path — 4 rechecks over 4 weeks each is pure waste. The 401/403 path was a loaded footgun: a broken bypass token 403s everything, and 4 strikes later the registry would be empty.

Notes for review

• rescheduleRedirectSource → rescheduleAtTargetCadence: it now serves several callers all wanting "keep it in rotation at its own cadence".
• Strikes remain ONE shared counter per target: suppression verdicts and redirect results both feed it; each verdict picks which maxStrikes applies; any successful render resets it to 0 (guarded by strikes > 0, so the hot path pays no extra write).
• suppressedReason gains a value: http-gone (404/410). Existing http-error rows re-classify on their next recheck.
• Tests: test/suppressionStatus.test.js (14 new) + 4 redirect-strike tests in test/renderQueueRedirect.test.js; full suite 334/334, lint + format clean.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces status-aware result handling for prerendered pages, optimizing suppression behavior based on HTTP status codes. Specifically, 404/410 errors are classified as "gone" (using a longer recheck interval and fewer strikes before deletion), while auth-shaped (401/403) and transient (408/429/5xx) errors are rescheduled at their normal cadence to prevent premature suppression. However, a critical issue was identified in the redirect handling logic within RenderQueue.js, where the source target is unconditionally deleted before checking if the destination returned an auth-shaped or transient error, potentially causing unintended deletions during outages.

[gemini-code-assist]

While preventing the creation of a suppressed destination row for 401/403 status codes is a great improvement, there is a critical issue here: the source target is still unconditionally deleted on line 384 (await Target.delete(CacheKey.extractUrl(result.id))) before this check is reached.

If a temporary credential outage or origin access rule change occurs, any page with a client-side redirect (e.g., redirecting to a login page) will land on a 401/403 page. This will trigger inspectedNonIndexable, resulting in the immediate and permanent deletion of the source target from the registry on the very first strike. This completely bypasses the protection against mass-deletions during outages.

Additionally, transient errors (408/429/5xx) on redirect destinations are not handled here and will also result in the source target being deleted and the destination suppressed.

Recommendation:
Check for auth-shaped and transient-shaped status codes before deleting the source target, and reschedule the source target at its normal cadence instead of retiring it. For example:

const authShaped = result.statusCode === 401 || result.statusCode === 403;
const transientShaped = result.statusCode === 408 || result.statusCode === 429 || result.statusCode >= 500;

if (authShaped || transientShaped) {
    logger.warn(
        `Prerendered url ${result.id} redirected to ${result.redirectedTo} which returned ${result.statusCode} — ` +
            `keeping the target and retrying at its normal cadence`
    );
    await this.rescheduleAtTargetCadence(result.id);
    return;
}

if (inspectedNonIndexable) {
    // ... proceed with deleting source and suppressing destination ...
}

[harper-joseph]

Confirmed and fixed in 2a9ba74 — this was a real hole. The auth/transient status check now runs at the top of processRedirectResult, before any branch that retires or strikes the source, exactly as you suggested: a rendered-through client-side redirect landing on 401/403 (auth-shaped, logged at error) or 408/429/5xx (transient, warn) keeps the source target and its cached pages and just reschedules at normal cadence. Since a bail-at-nav result posts the first hop's 3xx, only rendered-through results can carry these statuses, so the guard placement covers every path — the now-dead authShaped check inside the inspectedNonIndexable branch was removed.

Tests added: 403 and 503 landings keep the source and seed nothing; a 404 landing (genuine gone verdict) still retires the source and suppresses the destination as http-gone. 336/336 passing.

(agent reply — Claude)

[harper-joseph]

Pushed 2a9ba74 addressing the review:

• Gemini's critical finding (confirmed real): auth/transient-shaped statuses are now guarded at the TOP of processRedirectResult, before the inspectedNonIndexable branch can delete the source — a client-side redirect landing on 401/403 or 408/429/5xx keeps the source target + cached pages and reschedules at normal cadence. New tests pin the 403, 503, and (genuine-verdict) 404 landings.
• Self-review fix: recordRedirectStrike no longer causes a second Target point read — the strike read is threaded into rescheduleAtTargetCadence via an optional preloaded row.

336/336 tests, lint + format clean. Ready for re-review.

(agent — Claude)