Developer suggestion: Build a static HTML preview of the documentation in our CI/CD

[choldgraf]

Currently there is no way to easily preview the changes of a PR from within GitHub's workflow. I think this is because the mystmd.org site is built with some complex magic over at curvenote/mystmd.org. However, it helps a lot when developers can preview the consequences of a PR in a "production-like" environment.

Proposal

I think we should add a job to our CI/CD that uses readthedocs.org or a GitHub pages preview to give a link to a static HTML version of the documentation.

Alternatively, it'd be great if we could do this with the full server-style method, but I have no idea how to do that or if it's possible so I think static HTML is a good start.

[LecrisUT]

That should be doable with RTD. The build paths might have to be adjusted, but we should be able to do custom builds there. And we get version controlled documentation for free-ish :D

version: 2
build:
  os: ubuntu-22.04
  tools:
    nodejs: "20"
  commands:
    - npm install
    - npm run build
    - myst build --html
    - cp -r _build/html $READTHEDOCS_OUTPUT/html

Reference:

• https://docs.readthedocs.io/en/stable/config-file/v2.html#build-commands

[choldgraf]

I'm reopening this, because we have run into a bunch of issues building a PR preview on ReadTheDocs:

• The landing page of the preview is broken, due to some kind of javascript error.
• The navigation links aren't able to re-direct to .html versions of themselves and so they are broken.
• We needed to write custom robots.txt to get this to not be indexed publicly

This seems to be because of some behind-the-scenes HTML and JS rewriting that ReadTheDocs was doing, which was conflicting with the way that MyST builds its own links and assets.

So I'm going to close the RTD execution PR:

• Add execution to our documentation and test RTD previews #1121

and revert the RTD PR in another follow up:

• 📖 Add ReadTheDocs automation #982

Next steps

We're going to try the same workflow but with Netlify in order to determine if this is something unique to ReadTheDocs, or something more general for static HTML sites. I hope that this will work with Netlify, because I've used MyST static HTML with GitHub pages with no problem.

[humitos]

Hi 👋. Let me know if I can be helpful with anything related to Read the Docs. I haven't dug too much in this issue yet but I'll take a look at it tomorrow and provide any feedback I may have.

[rowanc1]

Thanks @humitos - portions of these are related to some of the other issues around hydration (#997) and absolute/relative links when changing between folders. The main RTD issue we ran into with the PR previews on RTD is not loading rtd.org/en/latest/slug when slug.html exists. Ideally there could be a way to toggle on either a redirect (slug.html --> slug) or loading from either route. Maybe this is a configuration we are missing?

[choldgraf]

OK this is now fixed for now with some netlify configuration:

• 📖 Remove RTD and add Netlify #1124

Would still love to get this working on ReadTheDocs in general! Considering this issue resolved for now though

[humitos]

Thanks @rowanc1 for the details.

portions of these are related to some of the other issues around hydration (#997)

We are tracking this internally at readthedocs/addons#278 but I'm still not sure what we can do to solve it. I need to understand a little more how hydration works.

and absolute/relative links when changing between folders. The main RTD issue we ran into with the PR previews on RTD is not loading rtd.org/en/latest/slug when slug.html exists

Would you mind opening an issue on Read the Docs explaining this case with more details and a reproducible example so we can debug further here? 🙏🏼

[LecrisUT]

portions of these are related to some of the other issues around hydration (#997)

We are tracking this internally at readthedocs/addons#278 but I'm still not sure what we can do to solve it. I need to understand a little more how hydration works.

This is also likely a mystmd issue. Here is another issue to follow and the links within. This would mostly only allow to insert head contents, but maybe there will be a workable solution with that.

Would you mind opening an issue on Read the Docs explaining this case with more details and a reproducible example so we can debug further here? 🙏🏼

It's already tracked at: readthedocs/readthedocs.org#11220

---

The issue of RTD compatibility is still very important since it is very likely that the user of mystmd would already have an RTD setup for the main code, but they want to supplement it with academic papers (related to readthedocs/readthedocs.org#11238). Support and implementation with Netlify would not be a replacement for this, but 👍 for supporting as many hosting providers as possible out-of-the-box.

[rowanc1]

@humitos gave a brief description of my findings on the differences in how netlify shows their version/collaboration bar without issue (readthedocs/addons#278).

Thanks @LecrisUT for reopening the PR, we will continue to chip away on this on our side.

[choldgraf]

I'd recommend that we open a new issue for "Add instructions and support for hosting MyST documents on ReadTheDocs", and treat this as an enhancement. We can use that to track the challenges, next steps needed, blockers, etc. @LecrisUT do you want to do that?

[LecrisUT]

I've opened the issue to track this: #1133. Feel free to expand it and add more context (still not sure what the first item on that list is, I was copying this comment)

[choldgraf]

@LecrisUT that is great, thanks! one other thought: can you just add the links to cross-reference pre-existing issues that are tracking those to-do items?

[LecrisUT]

Ok, I think I've gathered all of the items that I've posted across all the different repos.