Skip to content

feat(docs): SEO — sitemap, robots.txt, per-page og: tags, canonical URLs#266

Merged
ronaldtse merged 1 commit into
v5from
fix/docs-seo-sitemap-og-canonical
Jun 18, 2026
Merged

feat(docs): SEO — sitemap, robots.txt, per-page og: tags, canonical URLs#266
ronaldtse merged 1 commit into
v5from
fix/docs-seo-sitemap-og-canonical

Conversation

@ronaldtse

Copy link
Copy Markdown
Contributor

Audit findings — what was broken

Item Before After
robots.txt ❌ 404 ✅ Allow all + sitemap reference
sitemap.xml ❌ 404 ✅ Auto-generated with all 4,283 formula URLs
/browse/ static HTML ❌ 0 formula links ⚠️ Still client-rendered (see "Out of scope" below)
Per-page og:title ❌ Generic "Fontist Formulas" everywhere ✅ Per-page from frontmatter (abeezee - Fontist Formula)
Per-page og:description ❌ Generic ✅ Per-page (install command, family/style counts)
Per-page og:url ❌ Missing ✅ Per-page absolute URL
Canonical URLs ❌ Missing ✅ Per-page <link rel="canonical">
og:image ⚠️ Relative /logo-full.svg ✅ Absolute URL
Internal docs in sitemap TODO.style, V5_MIGRATION_PLAN ✅ Excluded via srcExclude

Critical problem solved: without a sitemap, Google could not discover the 4,283 formula pages. /browse/ is client-rendered (its static HTML has 0 formula links), so crawlers had no path to individual formulas.

Changes

docs/.vitepress/config.ts

  • sitemap config (VitePress built-in): emits /sitemap.xml at build time. transformItems prepends SITE_PATH (/formulas/) because VitePress sitemap doesn't auto-include the base path.
  • transformHead hook: emits per-page og:title, og:description, og:url, and <link rel="canonical"> from page frontmatter. Removed conflicting global og:title / og:description to avoid duplicates (verified: exactly 1 og:title per page).
  • Absolute og:image: was /logo-full.svg, now ${SITE_URL}/formulas/logo-full.svg.
  • Added TODO.style.md and V5_MIGRATION_PLAN.md to srcExclude — they were being built as public pages and appearing in the sitemap.
  • SITE_URL env var with localhost default — local dev (vitepress dev or npm run build) produces URLs that actually resolve during testing. CI sets SITE_URL=https://www.fontist.org.

docs/public/robots.txt (new)

User-agent: *
Allow: /

Sitemap: https://www.fontist.org/formulas/sitemap.xml

.github/workflows/docs.yml + .github/workflows/links.yml

Added workflow-level env: SITE_URL: https://www.fontist.org so production builds use the right origin.

Verification

Built locally in two modes:

Default (no env — local dev):

  • sitemap: http://localhost:5173/formulas/browse/google/abeezee
  • og:image: http://localhost:5173/formulas/logo-full.svg

Override (SITE_URL=https://www.fontist.org):

  • sitemap: https://www.fontist.org/formulas/browse/google/abeezee
  • Formula page (abeezee):
    og:title        = "abeezee - Fontist Formula"
    og:description  = "abeezee font package for Fontist. 1 font families, 2 styles. Install: ..."
    og:url          = "https://www.fontist.org/formulas/browse/google/abeezee"
    canonical       = "https://www.fontist.org/formulas/browse/google/abeezee"
    
  • No duplicate og:title (exactly 1 per page)
  • TODO.style.md / V5_MIGRATION_PLAN.md excluded from sitemap (count = 0)

Out of scope (still a gap)

/browse/ itself is still client-rendered. Its static HTML has the page shell but no formula links. This means:

  • Crawlers can't follow links FROM /browse/ to formulas (sitemap solves discovery)
  • /browse/ itself looks "empty" to a crawler (Google indexes it as a search page, not a content page)

Fixing this properly would require either:

  • Server-side rendering of the formula list (~5MB+ static HTML per page)
  • Pagination (browse/page/1, browse/page/2, ...)
  • Per-letter static index pages (browse/a/, browse/b/, ...)

These are architectural changes worth a separate effort. For pure discovery (the critical SEO need), the sitemap fully covers it.

Audit found that individual formula pages were SEO-perfect (title, description, content all in SSR HTML) but Google literally could not discover them: /browse/ renders its list client-side via Vue, so its static HTML has zero formula links, and there was no sitemap.xml or robots.txt.

Fixes:

1. Enable VitePress built-in sitemap config — generates /sitemap.xml listing all 4,283 formula pages. VitePress sitemap doesn't auto-include the base path, so transformItems prepends SITE_PATH.

2. Add docs/public/robots.txt — allows all crawlers, references sitemap.

3. Add transformHead hook — emits per-page og:title, og:description, og:url, and <link rel='canonical'> from page frontmatter. Removed conflicting og:title/og:description from global head to avoid duplicates (verified: exactly 1 og:title per page).

4. Make og:image absolute (was '/logo-full.svg', now full URL).

5. Add TODO.style.md and V5_MIGRATION_PLAN.md to srcExclude — they were being built as public pages and appearing in the sitemap.

Local-dev friendliness: SITE_URL comes from process.env.SITE_URL with 'http://localhost:5173' default. CI sets SITE_URL=https://www.fontist.org in docs.yml and links.yml env.

Verified locally with default (localhost URLs) and override (SITE_URL=https://example.org → example.org URLs).
@ronaldtse ronaldtse merged commit fc470f9 into v5 Jun 18, 2026
9 of 10 checks passed
@ronaldtse ronaldtse deleted the fix/docs-seo-sitemap-og-canonical branch June 18, 2026 02:27
ronaldtse added a commit that referenced this pull request Jun 18, 2026
…a pages

PR #266 added VitePress's built-in sitemap config, but on the live site the deployed sitemap only listed ~10 URLs (browse index, guide pages, licenses). The 4,283 formula pages were missing.

Root cause: VitePress generates its sitemap per-batch during vitepress build. The combine job in docs.yml deploys dist-main's sitemap, which only lists the main site. Browse pages merged in from build-batch matrix jobs don't update the sitemap.

Fix: docs/generate-sitemap.js — a post-build script that walks the final dist/ structure (after the multi-batch merge AND the clean-URLs post-process), finds every index.html, and emits a complete sitemap.xml. Excludes 404. Stable-sorted for reproducible builds.

Wired in two places: (1) docs/build.js — runs after rewriteCleanUrls, so local builds and link_checker get the complete sitemap; (2) docs.yml combine job — runs after Merge batch browse pages and assets AND after Rewrite clean URLs, just before upload-pages-artifact.

Verified locally: with 3 formula pages in dist, generated sitemap lists 42 URLs (was 10 before). In production with all 4,283 formulas, it will list ~4,300 URLs.
@github-actions

Copy link
Copy Markdown
Contributor

🔗 Link Check Failed

Full report: workflow run → download the link-check-results artifact.

Results (truncated — see artifact for full report)

Summary

Status Count
🔍 Total 189699
✅ Successful 52556
⏳ Timeouts 12
🔀 Redirected 4485
👻 Excluded 132491
❓ Unknown 0
🚫 Errors 155
⛔ Unsupported 0

Errors per input

Errors in .vitepress/dist/browse/adobe_reader_19/index.html

Errors in .vitepress/dist/browse/adobe_reader_20/index.html

Errors in .vitepress/dist/browse/andale/index.html

Errors in .vitepress/dist/browse/arial_black/index.html

Errors in .vitepress/dist/browse/cleartype/index.html

Errors in .vitepress/dist/browse/comic/index.html

Errors in .vitepress/dist/browse/courier/index.html

Errors in .vitepress/dist/browse/dejavu/index.html

Errors in .vitepress/dist/browse/euphemia/index.html

Errors in .vitepress/dist/browse/eurofix/index.html

Errors in .vitepress/dist/browse/fontoposubway/index.html

Errors in .vitepress/dist/browse/genkaimincho/index.html

Errors in .vitepress/dist/browse/georgia/index.html

Errors in .vitepress/dist/browse/gnu_freefont/index.html

Errors in .vitepress/dist/browse/google/finlandica/index.html

Errors in .vitepress/dist/browse/google/rokkitt/index.html

Errors in .vitepress/dist/browse/google/saira_stencil_one/index.html

Errors in .vitepress/dist/browse/google/ubuntu/index.html

Errors in .vitepress/dist/browse/google/ubuntu_condensed/index.html

Errors in .vitepress/dist/browse/google/ubuntu_mono/index.html

Errors in .vitepress/dist/browse/google/ubuntu_sans/index.html

Errors in .vitepress/dist/browse/google/ubuntu_sans_mono/index.html

Errors in .vitepress/dist/browse/i.ming/index.html

Errors in .vitepress/dist/browse/i.mingcp/index.html

Errors in .vitepress/dist/browse/i.mingvar/index.html

Errors in .vitepress/dist/browse/i.mingvarcp/index.html

Errors in .vitepress/dist/browse/impact/index.html

Errors in .vitepress/dist/browse/ipa/index.html

Errors in .vitepress/dist/browse/joongnajoche/index.html

  • [ERROR] http://joonggonara.co/ | Network error: Connection failed. Check network connectivity and firewall settings (error sending request for url (http://joonggonara.co/)): Connection failed. Check network connectivity and firewall settings

Errors in .vitepress/dist/browse/lucida_grande/index.html

Errors in .vitepress/dist/browse/macos/andale_mono/index.html

Errors in .vitepress/dist/browse/macos/andale_mono_10m11177/index.html

Errors in .vitepress/dist/browse/macos/arial_black/index.html

Errors in .vitepress/dist/browse/macos/arial_black_10m11177/index.html

Errors in .vitepress/dist/browse/macos/arial_unicode_ms/index.html

Errors in .vitepress/dist/browse/macos/arial_unicode_ms_10m11177/index.html

Errors in .vitepress/dist/browse/macos/bm_yeonsung_otf/index.html

Errors in .vitepress/dist/browse/macos/bm_yeonsung_otf_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font3/andale_mono/index.html

Errors in .vitepress/dist/browse/macos/font3/arial_black/index.html

Errors in .vitepress/dist/browse/macos/font3/arial_unicode_ms/index.html

Errors in .vitepress/dist/browse/macos/font3/gill_sans/index.html

Errors in .vitepress/dist/browse/macos/font3/lucida_grande/index.html

Errors in .vitepress/dist/browse/macos/font3/myriad_arabic/index.html

Errors in .vitepress/dist/browse/macos/font3/nanum_brush_script/index.html

  • [ERROR] http://www.nhncorp.com/ | Network error: Connection failed. Check network connectivity and firewall settings (error sending request for url (http://www.nhncorp.com/)): Connection failed. Check network connectivity and firewall settings

Errors in .vitepress/dist/browse/macos/font3/nanumgothic/index.html

  • [ERROR] http://www.nhncorp.com/ | Network error: Connection failed. Check network connectivity and firewall settings (error sending request for url (http://www.nhncorp.com/)): Connection failed. Check network connectivity and firewall settings

Errors in .vitepress/dist/browse/macos/font3/nanummyeongjo/index.html

Errors in .vitepress/dist/browse/macos/font4/andale_mono/index.html

Errors in .vitepress/dist/browse/macos/font4/arial_black/index.html

Errors in .vitepress/dist/browse/macos/font4/arial_unicode_ms/index.html

Errors in .vitepress/dist/browse/macos/font4/lucida_grande/index.html

Errors in .vitepress/dist/browse/macos/font4/myriad_arabic/index.html

Errors in .vitepress/dist/browse/macos/font4/nanum_brush_script/index.html

Errors in .vitepress/dist/browse/macos/font4/nanumgothic/index.html

Errors in .vitepress/dist/browse/macos/font4/nanummyeongjo/index.html

Errors in .vitepress/dist/browse/macos/font6/andale_mono/index.html

Errors in .vitepress/dist/browse/macos/font6/arial_black/index.html

Errors in .vitepress/dist/browse/macos/font6/arial_unicode_ms/index.html

Errors in .vitepress/dist/browse/macos/font6/bm_yeonsung_otf/index.html

Errors in .vitepress/dist/browse/macos/font6/lucida_grande/index.html

Errors in .vitepress/dist/browse/macos/font6/myriad_arabic/index.html

Errors in .vitepress/dist/browse/macos/font6/nanum_brush_script/index.html

Errors in .vitepress/dist/browse/macos/font6/nanumgothic/index.html

Errors in .vitepress/dist/browse/macos/font6/nanummyeongjo/index.html

Errors in .vitepress/dist/browse/macos/font6/sama_devanagari/index.html

Errors in .vitepress/dist/browse/macos/font6/sama_gujarati/index.html

Errors in .vitepress/dist/browse/macos/font6/sama_gurmukhi/index.html

Errors in .vitepress/dist/browse/macos/font6/sama_kannada/index.html

Errors in .vitepress/dist/browse/macos/font6/sama_malayalam/index.html

Errors in .vitepress/dist/browse/macos/font6/sama_tamil/index.html

Errors in .vitepress/dist/browse/macos/font7/andale_mono_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/arial_black_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/arial_unicode_ms_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/bm_yeonsung_otf_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/bm_yeonsung_otf_10m4185/index.html

Errors in .vitepress/dist/browse/macos/font7/lucida_grande_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/lucida_grande_10m1360/index.html

Errors in .vitepress/dist/browse/macos/font7/myriad_arabic_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/myriad_arabic_10m7626/index.html

Errors in .vitepress/dist/browse/macos/font7/nanum_brush_script_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/nanumgothic_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/nanummyeongjo_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_devanagari_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_devanagari_10m7626/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_gujarati_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_gujarati_10m7626/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_gurmukhi_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_gurmukhi_10m7626/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_kannada_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_kannada_10m7626/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_malayalam_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_malayalam_10m7626/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_tamil_10m1044/index.html

Errors in .vitepress/dist/browse/macos/font7/sama_tamil_10m7626/index.html

Errors in .vitepress/dist/browse/macos/font8/andale_mono_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/arial_black_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/arial_unicode_ms_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/bm_yeonsung_otf_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/lucida_grande_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/myriad_arabic_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/nanum_brush_script_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/nanumgothic_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/nanummyeongjo_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/sama_devanagari_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/sama_gujarati_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/sama_gurmukhi_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/sama_kannada_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/sama_malayalam_10m11177/index.html

Errors in .vitepress/dist/browse/macos/font8/sama_tamil_10m11177/index.html

Errors in .vitepress/dist/browse/macos/lucida_grande/index.html

Errors in .vitepress/dist/browse/macos/lucida_grande_10m11177/index.html

Errors in .vitepress/dist/browse/macos/myriad_arabic/index.html

Errors in .vitepress/dist/browse/macos/myriad_arabic_10m11177/index.html

Errors in .vitepress/dist/browse/macos/nanum/index.html

Errors in .vitepress/dist/browse/macos/nanum_brush_script_10m11177/index.html

Errors in .vitepress/dist/browse/macos/nanumgothic/index.html

Errors in .vitepress/dist/browse/macos/nanumgothic_10m11177/index.html

Errors in .vitepress/dist/browse/macos/nanummyeongjo/index.html

Errors in .vitepress/dist/browse/macos/nanummyeongjo_10m11177/index.html

Errors in .vitepress/dist/browse/macos/sama_devanagari/index.html

Errors in .vitepress/dist/browse/macos/sama_devanagari_10m11177/index.html

Errors in .vitepress/dist/browse/macos/sama_gujarati/index.html

Errors in .vitepress/dist/browse/macos/sama_gujarati_10m11177/index.html

Errors in .vitepress/dist/browse/macos/sama_gurmukhi/index.html

Errors in .vitepress/dist/browse/macos/sama_gurmukhi_10m11177/index.html

Errors in .vitepress/dist/browse/macos/sama_kannada/index.html

Errors in .vitepress/dist/browse/macos/sama_kannada_10m11177/index.html

Errors in .vitepress/dist/browse/macos/sama_malayalam/index.html

Errors in .vitepress/dist/browse/macos/sama_malayalam_10m11177/index.html

Errors in .vitepress/dist/browse/macos/sama_tamil/index.html

Errors in .vitepress/dist/browse/macos/sama_tamil_10m11177/index.html

Errors in .vitepress/dist/browse/noth/index.html

Errors in .vitepress/dist/browse/office_preview/index.html

Errors in .vitepress/dist/browse/opensuse_webcore_fonts/index.html

Errors in .vitepress/dist/browse/overpass/index.html

Errors in .vitepress/dist/browse/pclinuxos_webcore_fonts/index.html

Errors in .vitepress/dist/browse/pmingi.u/index.html

Errors in .vitepress/dist/browse/pmingi.uvar/index.html

Errors in .vitepress/dist/browse/sil/apparatus_sil/index.html

Errors in .vitepress/dist/browse/sil/khmer_7.100/index.html

Errors in .vitepress/dist/browse/sil/unicode_bmp_fallback_sil/index.html

Errors in .vitepress/dist/browse/simsun_18030/index.html

Errors in .vitepress/dist/browse/source/index.html

  • [ERROR] https://www.adobe.com/ | Network error: HTTP/2 protocol error. Server may not support HTTP/2 properly (error sending request for url (https://www.adobe.com/)): HTTP/2 protocol error. Server may not support HTTP/2 properly

Errors in .vitepress/dist/browse/tex/index.html

Errors in .vitepress/dist/browse/tex_gyre_math/index.html

Errors in .vitepress/dist/browse/timemachine_wa/index.html

Errors in .vitepress/dist/browse/urw-base35-fonts/index.html

Errors in .vitepress/dist/browse/webding/index.html

Errors in .vitepress/dist/browse/wine/index.html

Errors in .vitepress/dist/guide/choosing-formula/index.html

Errors in .vitepress/dist/licenses/adobe/index.html

Errors in .vitepress/dist/licenses/bitstream/index.html

Errors in .vitepress/dist/licenses/gpl/index.html

Errors in .vitepress/dist/licenses/lgpl/index.html

Errors in .vitepress/dist/licenses/ofl/index.html

Redirects per input

Redirects in .vitepress/dist/404/index.html

Redirects in .vitepress/dist/browse/adobe_reader_19/index.html

Redirects in .vitepress/dist/browse/adobe_reader_20/index.html

Redirects in .vitepress/dist/browse/akabara-cinderella/index.html

Redirects in .vitepress/dist/browse/andale/index.html

Redirects in .vitepress/dist/browse/aptos/index.html

Redirects in .vitepress/dist/browse/arabic/index.html

Redirects in .vitepress/dist/browse/arial_black/index.html

Redirects in .vitepress/dist/browse/asatte/index.html

Redirects in .vitepress/dist/browse/au/index.html

Redirects in .vitepress/dist/browse/au_passata_light_bold/index.html

Redirects in .vitepress/dist/browse/au_passata_oblique/index.html

Redirects in .vitepress/dist/browse/bokutachinogothic2bold/index.html

Redirects in .vitepress/dist/browse/bokutachinogothic2regular/index.html

Redirects in .vitepress/dist/browse/bokutachinogothic/index.html

Redirects in .vitepress/dist/browse/buddhism/index.html

Redirects in .vitepress/dist/browse/carlito/index.html

Redirects in .vitepress/dist/browse/chinese_simplified/index.html

Redirects in .vitepress/dist/browse/chinese_traditional/index.html

Redirects in .vitepress/dist/browse/cinderella/index.html

Redirects in .vitepress/dist/browse/cleartype/index.html

Redirects in .vitepress/dist/browse/comic/index.html

Redirects in .vitepress/dist/browse/courier/index.html

Redirects in .vitepress/dist/browse/courier_prime/index.html

Redirects in .vitepress/dist/browse/courier_prime_code/index.html

Redirects in .vitepress/dist/browse/courier_prime_cyrillic/index.html

Redirects in .vitepress/dist/browse/courier_prime_medium_and_semibold/index.html

Redirects in .vitepress/dist/browse/courier_prime_sans/index.html

Redirects in .vitepress/dist/browse/dejavu/index.html

Redirects in .vitepress/dist/browse/dengxian/index.html

Redirects in .vitepress/dist/browse/dmca_sans_serif/index.html

Redirects in .vitepress/dist/browse/dmca_sans_serif_10.0_dev1/index.html

Redirects in .vitepress/dist/browse/eb_garamond/index.html

Redirects in .vitepress/dist/browse/emoir-kaku/index.html

Redirects in .vitepress/dist/browse/eunomia/index.html

Redirects in .vitepress/dist/browse/euphemia/index.html

Redirects in .vitepress/dist/browse/eurofix/index.html

Redirects in .vitepress/dist/browse/f1.8/index.html

Redirects in .vitepress/dist/browse/fa-1/index.html

Redirects in .vitepress/dist/browse/ferrum/index.html

Redirects in .vitepress/dist/browse/fira_code/index.html

Redirects in .vitepress/dist/browse/fixedsys_excelsior/index.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant