Feature-Request: Server-rendered Markdown on page routes (SEO-friendly, not full SPA SSR)

[jochumdev]

Related to #67, scoped to page rendering only. Right now GET /some/page returns the SPA shell <div id="root"></div>, and content loads via a JS fetch to the API afterward. That works well for browsers, but leaves search crawlers, link previews, LLM agents, and curl looking at an empty page.

Proposal: have the existing page route render the page's Markdown to HTML server-side and include it directly in the response body, alongside the current shell. The SPA boots and takes over exactly as it does today.
Along with it:

• sitemap.xml generated from tree.json
• per-page <title> / in the initial HTML
• robots.txt check for page paths
• canonical URL + OpenGraph tags

I'm happy to implement that.

[perber]

Hi @jochumdev,

Thanks for your request. I haven’t implemented it yet because I didn’t want to maintain two separate templates.

A page-specific title should already be included. At least Google is able to crawl the page, but I agree that this is not optimal from an SEO perspective.

For agents and LLM workflows, an API should also be available later on.

Could you provide me with a plan outlining how you would approach your points?

[jochumdev]

I'm in the process of releasing the v1.0 of a project of mine.

The biggest thing was/is render the Markdown as HTML into each GET so when you curl https://docs.incus-compose.org/home for example you get the full Markdown as HTML. It must NOT be pixel perfect or perfect in any sense as React will replace the div.

That would mean, human as in JavaScript viewers would see the renderered react page while Bots see the HTML.
I thought about <noscript> for that, but Claude told me that noscript and Search Engines are no friends cause of abuse in the past.

[perber]

Maybe hidden content? A full hydration would be too much work and maintenance.

[jochumdev]

No, just the skeleton of the html generated by goldmark, no need for any css or other eye things as that gets fully replaced by React.

EDIT: Hidden content gives us again SEO troubles I'm sure.

[jochumdev]

I added the new sitemap.xml to google search console and my site https://docs.incus-compose.org vanished from google on keywords it was 2nd before ("incus compose" and "incus-compose").

I found that google is not happy with "/" being redirected to "/home" via JavaScript, so I fixed that.

Another issue I'm investigating is the flickering that still is here cause I thought google dislikes my page cause of hidden text, after it works without the redirect I'll test again with hidden fonts.

ETA: Around 2-3 Weeks cause I have to wait on google to re-index on each change.

[jochumdev]

An LLM told me to use this as the cleanest non-intrusive fix for the flickering:

#app-root { opacity: 0; transition: opacity 120ms ease; }
#app-root.mounted { opacity: 1; }

// after React's initial render callback / useEffect on app mount
document.getElementById('app-root').classList.add('mounted');

@perber what do you think about this? You have more experience with React than I do.

Kind regards,
René