feat(public): .md mirror for every route + /llms-full.txt aggregate#17
Merged
Conversation
LLMs and crawlers no longer have to parse HTML to consume the
marketing site. Every public HTML route now has a parallel .md
mirror at the same path with a .md suffix:
/ → /index.md
/pricing → /pricing.md
/for-agents → /for-agents.md
/status → /status.md
/docs → /docs.md (all 8 sections concatenated)
/blog → /blog.md (post index)
/blog/<slug> → /blog/<slug>.md
/use-cases → /use-cases.md (catalogue grouped by category)
/use-cases/<slug> → /use-cases/<slug>.md
A single aggregate /llms-full.txt (~361 KB) concatenates every
markdown page with URL-keyed section separators, so an LLM that
wants the entire site's text can fetch one file instead of 115.
Sources:
- Blog posts: copy .content/blog/<slug>.md verbatim
- Use cases: copy .content/use-cases/<slug>.md verbatim
- Docs: concatenate .content/docs/*.md (ordered by frontmatter
'order') into a single /docs.md
- React-only marketing pages (home, pricing, for-agents,
status): copy authored .content/pages/<name>.md
- Index pages (/blog, /use-cases): generated at build time from
the corresponding directory listings + frontmatter
Markdown link convention follows GFM — any URL in the rendered
text is in [label](url) form so an LLM can follow them. Internal
links use /path.md (the mirror), not /path (the HTML route), so
an LLM following links stays in markdown.
Total prerender output is now:
- 115 HTML files (per-route SPA-pre-render)
- 115 .md files (mirror routes)
- 1 llms.txt (manifest pointing at the .md routes)
- 1 llms-full.txt (361 KB, 115 sections, full text dump)
GitHub repo descriptions for InstaNode-dev/content and
InstaNode-dev/instanode-web were updated in the same change to
advertise the LLM-friendly URLs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
LLMs and crawlers no longer have to parse HTML to consume the marketing site. Every public HTML route now has a parallel
.mdmirror at the same path with a.mdsuffix.//index.md/pricing/pricing.md/for-agents/for-agents.md/status/status.md/docs/docs.md(all 8 sections in one file)/blog/blog.md/blog/<slug>/blog/<slug>.md/use-cases/use-cases.md/use-cases/<slug>/use-cases/<slug>.mdA single aggregate
/llms-full.txt(~361 KB) concatenates every markdown page with URL-keyed section separators, so an LLM that wants the entire site's text can fetch one file instead of 115.Where the content comes from
.content/blog/<slug>.mdverbatim.content/use-cases/<slug>.mdverbatim.content/docs/*.md(ordered by frontmatterorder) into a single/docs.md.content/pages/<name>.md(4 new files added toInstaNode-dev/contentin commit5fea532)/blog,/use-cases): generated at build time from the directory listings + frontmatterLink convention
All internal links in the markdown use the
.mdmirror (/use-cases/foo.md), not the HTML route (/use-cases/foo), so an LLM following links stays in markdown.GitHub descriptions updated
Both repos now advertise the LLM-friendly URLs in their description:
InstaNode-dev/content: "...LLM-friendly: https://instanode.dev/llms.txt · every page has a /.md mirror · full text at /llms-full.txt"InstaNode-dev/instanode-web: similarVerification
npm run build✓ — 115 HTML + 115 .md +llms.txt+llms-full.txt(~361 KB, 115 sections)dist/index.mdis the 2.5 KB homepage in markdowndist/docs.mdis the 5.5 KB single-page docs (8 sections concatenated)dist/blog.md,dist/use-cases.mdare generated indexes with markdown linksdist/use-cases/ephemeral-test-database-for-a-risky-migration.mdis the verbatim content-repo source🤖 Generated with Claude Code