Cross-repo documentation - Documentation portal

[ptrhck]

I am considering to use docusaurus for documenting multiple repos and been thinking about two different strategies to achieve this. I have different teams with different repos and each should document allongside their code but maintain one docs site.

First, create a docs repo that pulls the mardown files from alle repos using docusaurus-plugin-remote-content. I see a few drawbacks here:

• If the content changes in one repo, I have to trigger the build of the docs repo. As I will deploy to vercel, I will not have the preview deployment commented to the source content repo.
• A rebuild of the whole site will be triggered if only one repo changes.
• I would like to generate markdown from repository specific resources like a GraphQL schema. In that case, I would need to cross copy all those source files to the build environment

Second idea to adress all the above issues, is to deploy one docusaurus instance for each repository (If that is possible on Vercel, e.g. mydomain.com/docs1, mydomain.com/docs2 ). Here, a few questions that come up for this solution:

• Can I configure docusaurus to link between the docs from the navbar? (Same as Docs and Community on docusaurus.io)
• Will the algolia search be able to search trough all doc sites?

I am looking forward get some ideas from you!

Cheers!

[slorber]

Hi

If you have multiple repos, each with a lot of docs, it may be better to split your site into multiple subparts for build time reasons, having Vercel deploy previews etc...

It is then theoretically possible to assemble all those sites under a unique domain, each site having a subpath.

You can share the Docusaurus config/CSS across sites so that one site looks like the others

I'd suggest the following structure:

• Main, hosted at /: landing pages, blog, community doc, job offers, whatever...
• iOS, hosted at /ios
• Android, hosted at /android

With Vercel you can use rewrites (https://vercel.com/docs/cli#project-configuration/rewrites) so that one subpath is routed to a subdomain. This can allow you to assemble multiple sites/subdomains under a single domain.

Can I configure docusaurus to link between the docs from the navbar? (Same as Docs and Community on docusaurus.io)

The navbar allows you to link to any URL.

We have some components like the version dropdown that you may not be able to use: if you are on the Android website, Docusaurus doesn't know about the iOS doc versions and can't render a dropdown dynamically (but you can create it statically through config)

Will the algolia search be able to search through all doc sites?

Algolia should be able to index multiple sites assembled as a portal. The config allows you to provide start_urls and sitemaps, and the crawler probably can figure things out just by following links.

The question is also: once this data is indexed, can you query it correctly?

It may not work exactly like you want by default, but you can swizzle some parts to adapt our default logic to your need.

Particularly the "contextual search" system is opinionated and may not work for your use-case , but you can swizzle @theme/hooks/useAlgoliaContextualFacetFilters and plug your own logic so that you can decide what are the query filters (AND/OR closes)

There are questions to answer, like:

• if I'm browsing the portal homepage, should search return ios/android matches?
• if I'm browsing the ios doc, should search return android matches?

---

This use-case totally makes sense, and we can assist you with this so don't hesitate to ask questions here or on Discord.

If this is hard to set up, we'll figure out a way to make it easier.

I don't think this strategy has been used a lot so far and I'd like to have some nice "portal" sites in our showcase as a demonstration of what is technically possible.

---

Note:

• When assembling sites as a portal, the transition from one site to another is regular HTML navigation, not SPA transition. This can feel slightly less fluid, but generally not a big deal.
• The Docusaurus site itself is already a portal somehow: each locale is a distinct SPA, and we are also linking to older docs versions that are actually hosted on standalone Netlify deployments

[slorber]

Sorry but it's really hard for me to figure things out.

How many sites do you have exactly, and what is the config for each of them?

I can't help you without seeing at least the site URL, baseurl and navbar.

Also it's not clear what is the expected result: please provide a mockup of the main site navbar, and the ios site navbar (if different) (including link URLs)

[ptrhck]

I created a monorepo https://github.com/ptrhck/docusaurus-portal and deployed each repo to Vercel (I hope the non-existence of a custom domain with subdomains is no issue here):

• https://main-portal.vercel.app/
• https://android-portal.vercel.app/
• https://ios-portal.vercel.app/

My goal is to access each documentation site from the main branch domain from the navbar, which is https://main-portal.vercel.app/ here and that routing from the other pages works under that domain as well. As you can see, I can access the ios docu site through the navbar. However, it currently routes to the landing page and not to the documentation itself. But I can access it manually, e.g. https://main-portal.vercel.app/ios/docs/intro. Here, other docs pages of this dedicated deployment are displayed in the sitebar, however, they route back to the main repo deployment. To give another specific example:
From visiting Tutorial - Basics, I would expect to be routed to https://main-portal.vercel.app/ios/docs/tutorial-basics/create-a-page but I get routed to https://main-portal.vercel.app/docs/tutorial-basics/create-a-page instead.

Does that make sense?

As far as I can see, the intended navigation is similiar to the one on docusuarus.io itself with the difference that each docu category is an independent deployment.

[slorber]

However, it currently routes to the landing page and not to the documentation itself

This is because your ios/android deployments both have their own landing page. It's up to you to have an extra landing page or not.

We provide a docs-only mode in case you only want a single landing page for your whole portal: https://docusaurus.io/docs/docs-introduction#docs-only-mode

You should be able to test this locally:

Here, other docs pages of this dedicated deployment are displayed in the sitebar, however, they route back to the main repo deployment.

This is because each "sub-site" should use something like baseUrl: 'ios'.

Here you end up in a very weird state where the JS of a subsite tries to load the JS of the main site instead of loading the bundles hosted under /ios/

If you look at https://main-portal.vercel.app/ios/docs/intro, you'll notice that nothing works, React does not hydrate, the theme toggle can't be clicked etc ;)

I suggested some edits here: ptrhck/docusaurus-portal#2

---

Note: if you need to make sure that the subsites work on both the main domain and their own subdomain, I'd recommend to always force a baseurl/subpath in all cases, and make sure Vercel serves a deployment with a subfolder.

Your vercel command could include cp -R build ios/build, and your site would work with these URLs:

• https://main-portal.vercel.app/ios
• https://ios-portal.vercel.app/ios

[ptrhck]

Regarding your last note on subfolder deployments. I do not care too much about the prod subdomains as the access point would always be the main domain. Of course I would like to make sure that the preview deployments work. In that case, would I also need that subdir build folder?

Shouldn't this vercel option be used instad of copying in the build command?

[slorber]

Regarding your last note on subfolder deployments. I do not care too much about the prod subdomains as the access point would always be the main domain. Of course I would like to make sure that the preview deployments work. In that case, would I also need that subdir build folder?

It is probably possible to detect a deploy preview thanks to Vercel env variables

And I think you can use different configs (output dir, build command...) for each deployment kind (not sure how though, I know it's possible with Netlify at least)

Shouldn't this vercel option be used instad of copying in the build command?

If you want files to be served at /ios, you need to build the static files in /build/ios AND serve the build folder. If you served build/ios, your files would be served directly from / (ie /index.html would be routed to build/ios/index.html while you want /ios/index.html would be routed to build/ios/index.html).

(this applies only if you want to use a baseUrl for your deploy preview deployments 🤪 )

[ptrhck]

Many thanks for your helpful ideas and suggestions! I will look into it more deeply and give it a try. I am very sure I will have further questions upcoming.

One is that we might have to add authentication. With Next.js it would be fairly simple to add some basic auth, e.g. using middleware. It seems one could solve it using AWS Amplify. I have no clear idea yet how to add it to such portal architecture using different Vercel deployments. Do you?

[slorber]

👍

Docusaurus does not provide any server-side runtime atm, so you can't add authentication to Docusaurus.

At most you could have a /admin/index.html static page that would render an empty layout, call authenticated API routes on the client, and eventually present a login screen to users so that the next API calls would work.

To add authentication outside of Docusaurus, you can deploy edge serverless functions on top of a static deployment, intercept requests there and handle the auth on top, from the outside. This should be possible with Vercel Edge Functions, Netlify Edge Handlers and Cloudflare Workers

https://developers.cloudflare.com/workers/examples/basic-auth

Not sure why but it seems the Vercel middleware implementation is coupled to Next.js 🤷‍♂️ it shouldn't be, maybe it's just a docs / example issue

I don't know much about Amplify, but it seems to basically do the same with a UI

[ptrhck]

Yes, using _middleware on Vercel should be possible according to this issue vercel/examples#50 and I will give it a try.

[ptrhck]

Now, that the deployment was successfull I am wondering how to distribute my docusaurus configs and other assets like custom.css oder favicon across the repos. Regarding the docusaurus config, I should be able to create one universal config that just depends on certain environment variables that need to be added to each dedicated deployment.

Do you have an idea how to maintain this centrally and push to repos on new changes?

[ptrhck]

The problem is, that it actually is not a monorepo but distributed across repos. That is why I wanted a portal solution in the first place. I cannot see how yarn workspaces can help here?

[slorber]

This is a bit out of the scope of Docusaurus, and follows the same rules for sharing any code in the JS ecosystem.

You do whatever you want:

• publish a shared config to npm (private or public repo)
• load shared git repository as part of your build process
• use git submodules
• write a script to download a json/js config from a shared folder
• copy-paste the shared config everywhere
• ...

I can't really advise you here, it's up to you to find out the best way to share this config according to your org practices

[ptrhck]

I am currently working on this topic to create a universal configuration file and struggle to create an autogenerated sidebar that also loads items from another json file:

// @ts-check

const usesPydoc = process.env.USES_PYDOC === "true"

/** @type {import('@docusaurus/plugin-content-docs').SidebarsConfig} */
const sidebars = {
  // By default, Docusaurus generates a sidebar from the docs folder structure
  sidebar: [{ type: "autogenerated", dirName: "." }],
  ...(usesPydoc && {
    "API Documentation": [require("./docs/reference/sidebar.json")],
  }),
}

module.exports = sidebars

So I want autogeneration except for the reference folder. How can this be achieved?

[slorber]

The code looks fine to me, and I can't tell what's wrong without seeing the errors you are facing.

As long as you export a valid sidebar json, it should work, and the code that produces this export is just node, so it's your responsibility to write it correctly :)

You might need an upcoming feature to exclude reference from being used in the autogenerated sidebar slice (#5689) otherwise it might appear twice in your sidebar.

Until then, 2 workarounds:

• autogenerate from multiple subfolders (instead of the root)
• filter the unwanted "reference" category using a custom sidebar generator (https://docusaurus.io/docs/sidebar#customize-the-sidebar-items-generator)

[ptrhck]

I just removed the "API Documentation" from sidebar.js and renamed the reference folder through the sidebarItemsGenerator. Thanks for your hint!

[jsphstls]

A colleague of mine discovered that the Escaping from Spa Redirects functionality allows us to serve multiple Docusaurus builds by putting the build content into subdirectories of the portal's Docusaurus static folder. We could automate the portal build by gathering artifacts from the other repo pipelines.

Are there any downsides to using the static folder in this way? Does this strategy rely on any behavior that is subject to change in future releases?

[slorber]

@jsphstls I don't really see value in using the /static folder to add subsite builds. You could as well add the /build of subsites to the /build of the portal. The importance is that the final static deployment still makes sense.

Note that you can also assemble the portal at the CDN level (for example using Netlify _redirect file)

• Deploy your subsite at ios.mydoc.com/ios, and ask your CDN to serve it from mydoc.com/ios
• Deploy your subsite at android.mydoc.com/android, and ask your CDN to serve it from mydoc.com/android
• Deploy the portal to mydoc.com and link to mydoc.com/<productName>

This way users/SEO only see a single domain, and yet you can update each subsite independently

[K0rdan]

Just saying thank you @slorber and @ptrhck for your discussion and the example you worked on the repo https://github.com/ptrhck/docusaurus-portal, it helps me a lot with the new Turborepo features.

[slorber]

Awesome 👍 Can you explain how it's related to Turborepo?

[Nhollas]

Did you get a live working example of this portal? I'm Looking to build a similar site that pulls Markdown from all our different Azure DevOps Projects/Repos.