seo: fix Moz and GSC flags via server-side redirects and noindex headers#874
seo: fix Moz and GSC flags via server-side redirects and noindex headers#874Jordy-Baby wants to merge 1 commit into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
📝 WalkthroughWalkthroughRemoved client-side redirect configuration from the Docusaurus config file and added server-side redirects and SEO headers to Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
vercel.json (1)
51-121: Consider a smoke test in CI to catch redirect regressions.69 permanent redirects is a lot of surface area to maintain by hand, and
permanent: truemeans mistakes get cached hard by browsers and search engines. A small post-deploy check (or even a preview-deploy check) that curls eachsourceand asserts a 301/308 to the expecteddestinationwould catch typos/conflicts before they hit prod. This can be a simple Node/bash script that readsvercel.jsonand hits the Vercel preview URL.Not blocking — just a cheap safety net given the scope here.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@vercel.json` around lines 51 - 121, The redirects list in vercel.json (the "redirects" array with many entries using "permanent": true) is high-risk; add a CI smoke-test that reads vercel.json, iterates the "redirects" entries, and for each {source,destination} verifies the deployed preview/prod URL for source returns a 301/308 Location header matching destination. Implement as a small script (e.g. verify-redirects.js or scripts/check-redirects.sh) that accepts a base URL (preview via VERCEL_URL or CI-provided URL), performs HEAD/GET requests, asserts the Location header equals the expected destination, and exit non-zero on any mismatch; wire this script into CI (preview and/or post-deploy job) to fail the pipeline on redirect regressions.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@vercel.json`:
- Around line 34-37: The route entry "/blog/page/:path*" in vercel.json is inert
because Docusaurus is configured with postsPerPage: "ALL" in
docusaurus.config.js, so no paginated blog pages are emitted; either remove the
"/blog/page/:path*" rule to clean up the config or keep it intentionally as a
guard and add a concise comment explaining it is inactive while postsPerPage is
"ALL" so future maintainers understand why it exists.
- Around line 14-21: The vercel.json header rules use the wrong path
("/blog/tags" and "/blog/tags/:path*") so they don't match the actual Docusaurus
output where tagsBasePath is set to 'tag'; update the rules to target
"/blog/tag/:path*" (or add "/blog/tags/:path*" as an additional rule if you want
both) and remove the redundant explicit "/blog/tag" entry since ":path*" already
covers the index.
---
Nitpick comments:
In `@vercel.json`:
- Around line 51-121: The redirects list in vercel.json (the "redirects" array
with many entries using "permanent": true) is high-risk; add a CI smoke-test
that reads vercel.json, iterates the "redirects" entries, and for each
{source,destination} verifies the deployed preview/prod URL for source returns a
301/308 Location header matching destination. Implement as a small script (e.g.
verify-redirects.js or scripts/check-redirects.sh) that accepts a base URL
(preview via VERCEL_URL or CI-provided URL), performs HEAD/GET requests, asserts
the Location header equals the expected destination, and exit non-zero on any
mismatch; wire this script into CI (preview and/or post-deploy job) to fail the
pipeline on redirect regressions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: cff29dd9-0176-4349-a1fa-206261bf1118
📒 Files selected for processing (3)
docusaurus.config.jsdocusaurus.config.llm.jsvercel.json
💤 Files with no reviewable changes (1)
- docusaurus.config.js
| { | ||
| "source": "/blog/tags", | ||
| "headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }] | ||
| }, | ||
| { | ||
| "source": "/blog/tags/:path*", | ||
| "headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }] | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
rg -nP --type=js -C2 '\btagsBasePath\b'
# Also list what the build actually emits for tag pages:
rg -nP --type=js -C2 '\barchiveBasePath\b|\bpageBasePath\b'Repository: enviodev/docs
Length of output: 281
Blog tag noindex rules target the wrong path — they will not match actual tag URLs.
In docusaurus.config.js the blog plugin is configured with tagsBasePath: 'tag' (singular). That means Docusaurus emits blog tag pages at:
/blog/tag(index)/blog/tag/<tagname>
…not /blog/tags/.... As written, these two header rules match nothing in the produced site, so the tag pages (one of the specifically called-out low-value/duplicate targets in the PR description) will continue to be indexable. This is the SEO regression the PR is trying to prevent.
Also note: in Vercel's path-to-regexp, :path* already matches zero or more segments, so once the path is corrected the explicit /blog/tag rule becomes redundant with /blog/tag/:path*.
🛠️ Proposed fix
{
- "source": "/blog/tags",
- "headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }]
- },
- {
- "source": "/blog/tags/:path*",
+ "source": "/blog/tag/:path*",
"headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }]
},If you'd rather keep both (in case tagsBasePath is changed later), add /blog/tags/:path* as a second rule instead of replacing — but the singular form is the one that actually matches today's routes.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| { | |
| "source": "/blog/tags", | |
| "headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }] | |
| }, | |
| { | |
| "source": "/blog/tags/:path*", | |
| "headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }] | |
| }, | |
| { | |
| "source": "/blog/tag/:path*", | |
| "headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }] | |
| }, |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@vercel.json` around lines 14 - 21, The vercel.json header rules use the wrong
path ("/blog/tags" and "/blog/tags/:path*") so they don't match the actual
Docusaurus output where tagsBasePath is set to 'tag'; update the rules to target
"/blog/tag/:path*" (or add "/blog/tags/:path*" as an additional rule if you want
both) and remove the redundant explicit "/blog/tag" entry since ":path*" already
covers the index.
| { | ||
| "source": "/blog/page/:path*", | ||
| "headers": [{ "key": "X-Robots-Tag", "value": "noindex, follow" }] | ||
| }, |
There was a problem hiding this comment.
/blog/page/:path* is effectively dead given postsPerPage: "ALL".
The blog is configured with postsPerPage: "ALL" in docusaurus.config.js, so Docusaurus does not emit any /blog/page/N paginated list pages. This rule is harmless but inert. Safe to leave as a guard in case pagination is re-enabled later; just flagging so you know it's not currently doing anything.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@vercel.json` around lines 34 - 37, The route entry "/blog/page/:path*" in
vercel.json is inert because Docusaurus is configured with postsPerPage: "ALL"
in docusaurus.config.js, so no paginated blog pages are emitted; either remove
the "/blog/page/:path*" rule to clean up the config or keep it intentionally as
a guard and add a concise comment explaining it is inactive while postsPerPage
is "ALL" so future maintainers understand why it exists.
Replaces @docusaurus/plugin-client-redirects (which emitted thin HTML stubs with meta refresh) with server-side 301s in vercel.json. This resolves Moz flags for missing H1, meta refresh, thin content, and title issues on 12+ redirect URLs. Adds 14 new redirect rules covering GSC 404s: - /blog/blog/:path* wildcard for double-prefix external link errors - Old /docs/HyperIndex/v2/* paths to current equivalents - Wrong product combos (HyperSync/HyperIndex <-> HyperRPC overviews) - Legacy docs paths (/docs/migration-guide, /docs/api, etc.) Adds X-Robots-Tag noindex headers for: - Blog archive/RSS/pagination/tags (low-value SEO, eat crawl budget) - /docs/HyperIndex-LLM/*, /docs/HyperSync-LLM/*, /docs/HyperRPC-LLM/* (consolidated LLM-consumption pages that duplicate main docs content, resolving GSC "Duplicate, Google chose different canonical" flag) Adds global noindex metadata to docusaurus.config.llm.js so the separate llm-docs.envio.dev build emits noindex meta on every page. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Jordy-Baby
force-pushed
the
claude/nifty-diffie
branch
from
8f4c033 to
971c6fd
Compare
|
Closing — the underlying meta-refresh stubs are a Moz audit finding, not a search-ranking problem. Google handles meta-refresh as equivalent to 301 and indexes the canonical pages correctly. Reopen if Moz dashboard cleanliness becomes a priority. |
1 participant
Summary
Fixes 40+ SEO issues flagged by Moz and Google Search Console through
vercel.jsonconfig changes. No impact on production content or functionality.Three changes to three files:
vercel.json(biggest change)@docusaurus/plugin-client-redirectswith 55 server-side 301 redirects. The plugin was generating thin HTML stubs with<meta http-equiv="refresh">which Moz flagged for missing H1, thin content, meta refresh, and title issues. Server-side 301s are cleaner for SEO and users./blog/blog/*external-link typos, old/docs/HyperIndex/v2/*paths, wrong product combos, and legacy docs URLs.X-Robots-Tagheader rules to noindex low-value pages: blog tag pages, archive, RSS, pagination, and theHyperIndex-LLM/HyperSync-LLM/HyperRPC-LLMconsolidated docs (which duplicate main docs content and were causing GSC "duplicate canonical" flags).docusaurus.config.jsredirectsListarray and@docusaurus/plugin-client-redirectsplugin usage. File shrinks from 623 to 393 lines.docusaurus.config.llm.js{ name: "robots", content: "noindex, nofollow" }tothemeConfig.metadataso the separatellm-docs.envio.devbuild emits noindex on every page.What this fixes
Moz (~30 issues cleared):
Google Search Console (~35 issues cleared):
What this does NOT change
@docusaurus/plugin-client-redirectspackage is still inpackage.jsonbut no longer imported — can be removed in a follow-up PRTest plan
Curl verification (optional, for devs)
Replace
<preview>with the Vercel preview URL posted in PR comments:🤖 Generated with Claude Code
Summary by CodeRabbit