Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docusaurus 2 #789

Open
yangshun opened this issue Jun 20, 2018 · 42 comments
Labels
RFC

Comments

@yangshun
Copy link
Member

@yangshun yangshun commented Jun 20, 2018

Update 03/19/19

We have started a Roadmap/Feature requests page thanks to the awesome folks at Canny.io. Have a go at it on our WIP v2: https://v2.docusaurus.io/feedback/


These are some of the problems I'm seeing in Docusaurus now and also how we can address them in v2. A number of the ideas here were inspired by VuePress and other static site generators.

In the current JavaScript-based static site generators ecosystem, there are the following libraries:

Useful libraries

My proposal for D2 is that Docusaurus be responsible for documentation content, routing, translation and versioning, leaving layout and styling to the end user. Docusaurus would provide a default theme and a few common layouts that the user can use, or ignore if they want to build their layout from scratch. It's not a good idea for Docusaurus to maintain the entire layout and styling as these stuff are hard to make improvements without breaking existing users' code.

A Google Doc where we go into more details can be found here

D2 Design Principles

  • Easy to learn but most things are still achievable by users, even if it takes them more code and more time to write. No abstractions beat bad abstractions, and we don't want users to have to hack around the wrong abstractions. Mandatory talk: Minimal API Surface Area
  • Users won't feel overwhelmed when looking at the directory of a D2 project. It should look familiar to users familiar building web apps and websites and is easy to build on top of.
  • We should not lock-in our users to use our plugins or our CSS. Certain lower-level infra level stuff like React Loadable, React Router are fine, but not higher level ones, such as choice of Markdown engines, CSS frameworks, CSS methodology.
  • The separations of concerns between each layer of our stack should be clear - well-abstracted and modular.

D1 Problems and Proposed Solutions

Infra

Current Problems

  • Routing logic is separated and resides within both server and generation code, leading to duplicated logic and harder to ensure that code working in server mode also work in generation mode.
  • Routing is not declarative. Hard to understand what possible routes there are in Docusaurus.
  • Build pipeline is imperative and quite tangled.
  • Does not allow adding local vendor CSS and JS (#534)
  • New features are mostly enabled via siteConfig - siteConfig will bloat over time and become unmaintainble.
  • Current architecture is a bit tangled and makes it hard to introduce plugins.
  • Many components relies on process.cwd() too much (mostly for siteConfig), making it very hard to test.
  • LiveReload does a full-page reload, and page reloads for changes of all files.

Suggested improvements

  • Use a module bundler like webpack (or Metro if possible, since it's also by Facebook).
  • Static pages can be generated using (https://github.com/markdalgleish/static-site-generator-webpack-plugin)
  • Assets should be fingerprinted (should be trivial with webpack)
  • Use webpack-dev-server during development which enables hot reloading - only reload the modules which changed.
  • Rearchitect architecture to introduce hooks into the development and build phase so that a plugin system is possible.
  • Keys casing are not consistent - camelCase in siteConfig and blog front matter but snake_case in docs front matter.

HTML/CSS

Current Problems

  • Raw CSS without any preprocessor. Hacky variable substitution approach using regexp.
  • Docusaurus controls much of the styles and layout, which makes it prone to breaking end users code who customize.
  • All CSS is compiled into one file.
  • HTML/CSS is self-written and doesn't follow CSS framework best practices. Typography is not great (margins and paddings are not consistent)/. Partially fixed in #757.
  • Hard to do theming on top of current CSS structure. Limited to changing of colors and fonts.
  • Site layout is not customizable - EDIT: Just found out it is (#128).
  • Docs/blog layout is not customizable.
  • No 404 page.

Suggested improvements

  • Layout and styling should be controlled by theme and user. Docusaurus should just provide the data (content, translation, versioning), props and a default theme. This is the approach taken by many static site generators as well, such as Jekyll, Gatsby and VuePress. VuePress allows ejecting of the default theme for further layout customization.
  • Follow CSS frameworks best practices - Use rem for units, allow specifying margin width, etc.
  • Allow user to select their own CSS preprocesser.
  • Allow setting styles and scripts for specific pages only. Use case - some sites like Babel, Prettier and Reason have playgrounds where they only want to load specific scripts and styles.
  • Make Docusaurus pages/components more customizable (#249)
  • Add a 404 page.

JavaScript

Current Problems

  • JS code in pages is currently not being transpiled by Babel. We have to write ES5 in code that will be sent to browsers.
  • React is used as a templating engine for static content; we're not using any of its view library features. Interactivity has to be added through script tags and dangerouslySetInnerHTML.

Possible improvements

  • If the user adds interactivity to their React component/page, we should include React library on the page and hydrate the component after the page has been rendered.

API

Current Problems

  • CLI commands should be using docusaurus <command> (instead of docusaurus-command) which is more conventional (see Yarn and Git).
  • Docusaurus-provided components are required using a relative path which looks a bit odd and may break if we change the library's internal directory structure.

Possible improvements

  • Rework the CLI commands. Should be just a matter of renaming and shifting them into docusaurus core. This also solves the problem of having to publish docusaurus-init as a separate package.
  • Export docusaurus components on the main npm file itself. Add even more common components (we can consider creating pages as components as well).

Blog

Current Problems

  • NA

Possible improvements

  • Allow for commenting support (via a plugin and not siteConfig as how it is currently done)
  • Allow customizing of date.
  • Draft mode for posts
  • Tags, Categories

Markdown

Current Problems

  • Only reads YAML, does not support different kinds of front matter.
  • Unable to embed content from other Markdown files.

Possible improvements

  • Add reference style linking.
  • Add page tabbing (#45), probably via a Remarkable plugin.
  • Allow importing markdowns in other markdowns.
  • Line highlighting within code blocks.
  • Support embedding React components in markdown.
  • Emoji support.
  • Check that internal page links and anchors work fine.
  • Support special block tags like in VuePress for tip/warning/danger markup.

Search

Current Problems

  • We're using Algolia as the only form of search now. Algolia search is a third-party service that requires internet and doesn't work behind VPN/offline.

Possible improvements

  • I really like Algolia search as it's useful and so simple to use. But we could also come up with alternative offline search mechanisms to get around the limitations mentioned above. At my previous company, I added offline search for a static site via Lunr.js. It works by looking through the markdown files and generated a static index of the content during build time. The searching and rendering of search results is purely done on the client side. It roughly works - https://engineering.grab.com/search.html?q=react

These are my two cents. Feel free to chime in (:

@endiliey endiliey added the RFC label Jun 20, 2018
@ahmadalfy

This comment has been minimized.

Copy link
Contributor

@ahmadalfy ahmadalfy commented Jun 20, 2018

+100 for moving the layout and styling outside Docusaurus' scope.

@JoelMarcey

This comment has been minimized.

Copy link
Contributor

@JoelMarcey JoelMarcey commented Jun 20, 2018

Thanks so much for staring the planning process here @yangshun. These are wonderful.

I will say a request for a local/offline search option should be considered for v2 as well -- we have had that request multiple times as well.

Also, I feel it is important to reiterate this -- no matter what features we add for v2, we always want to make sure we adhere to the overall mantra of Docusaurus which is "easy to create a documentation website", with the emphasis on "easy". It is always a tricky balance to add a bunch of features and keep things as simple as possible.

@endiliey

This comment has been minimized.

Copy link
Collaborator

@endiliey endiliey commented Jun 20, 2018

This is exciting. webpack 🎉.

Extra things that I think will be great to have

  • Easy way to support redirect urls (301 Redirect, Not 404)
/blog/my-old-title   /blog/an-even-better-title
/docs/en/android-setup /docs/en/getting-started
  • Easy way to import from other documentation sites generator (e.g: gitbook)
  • User can download documents (maybe in pdf) 😄

Questions

  • Are we going only accept bug fixes for Docusaurus v1 ? Any PRs opened against v1 that are not bug fixes will be closed and features will only be accepted for v2
  • Given that this will be a lot of work. When ? 😃
@yangshun

This comment has been minimized.

Copy link
Member Author

@yangshun yangshun commented Jun 20, 2018

I will say a request for a local/offline search option should be considered for v2 as well -- we have had that request multiple times as well.

Added a section about search.

Also, I feel it is important to reiterate this -- no matter what features we add for v2, we always want to make sure we adhere to the overall mantra of Docusaurus which is "easy to create a documentation website", with the emphasis on "easy". It is always a tricky balance to add a bunch of features and keep things as simple as possible.

Yes! No worries we'll stick to this. It might just be troublesome for v1 users to migrate to v2 due to some potentially backward-incompatible changes we're making.

@JoelMarcey

This comment has been minimized.

Copy link
Contributor

@JoelMarcey JoelMarcey commented Jun 20, 2018

Whether we do this for 1.3+ or v2, we should also considering standardizing on one or both of the following:

  • yarn or npm in the documentation. Not both.
  • Depending on whether we do the above, remove the other's .lock file. Decide if we want .lock files at all.
@caabernathy

This comment has been minimized.

Copy link
Contributor

@caabernathy caabernathy commented Jul 5, 2018

Let's also think about integrating autogen systems into an upcoming version - see #78

@JoelMarcey

This comment has been minimized.

Copy link
Contributor

@JoelMarcey JoelMarcey commented Jul 26, 2018

Adding Docusaurus format support to hh-apidoc is not a Docusaurus v2 feature necessarily, but we can consider as something to help grow the use of it.

hhvm/hh-apidoc#29

@fredemmott

This comment has been minimized.

Copy link

@fredemmott fredemmott commented Jul 26, 2018

Ultimately I think it should be hh-apidoc's responsibility to provide documentation/metadata in a good format for docusuarus when wanted - but for user experience, it would be good if docusaurus were aware of it and would suggest/configure running it as part of the docusaurus build process rather than needing a separate predecessor step

@prakhar-goel

This comment has been minimized.

Copy link

@prakhar-goel prakhar-goel commented Aug 22, 2018

+1 for offline search using lunrjs or similar. We are using Docusaurus for internal company documentation and therefore cannot use default Algolia search. Ability to search through markdown files would be a great add on to the current version.

@kevinbarabash

This comment has been minimized.

Copy link

@kevinbarabash kevinbarabash commented Sep 4, 2018

It would be nice if #852 could be addressed as part of v2.

@chenglou

This comment has been minimized.

Copy link
Contributor

@chenglou chenglou commented Sep 16, 2018

I really hope that Docusaurus can keep its initial simplicity; v2 sounds nice but there are several things I'm slightly worried about. By simplicity, I mean both the interface and implementation (I'll explain why I care about the latter, aside from being an occasional contributor/debugger myself, at the end). Docusaurus' rather solid, that and the crowdin support are the two reason why we switched away from other ultra ambitious generators with a complex webpack pipeline that failed to handle three digits number of markdown files...

Specifically:

Routing logic is separated and resides within both server and generation code, leading to duplicated logic and harder to ensure that code working in server mode also work in generation mode.

Yeah this is definitely a pain point we felt on the user side! I've noticed/filed/fixed a few of these.

Routing is not declarative. Hard to understand what possible routes there are in Docusaurus.

Not too sure what this means; isn't it glanceable in sidebars.json?

Current architecture is a bit tangled and makes it hard to introduce plugins.

Making it pluggable seems to always come with a tradeoff (relevant). I'm imagining that when I read the supposedly new siteConfig, I might see a list of clean but super opaque require calls instead of a js object config? This is what happened with other static generators. The complexity is hidden by all the third-party plugin requires but there's always some that behave weirdly and try to over-compose, like webpack configs.

LiveReload does a full-page reload, and page reloads for changes of all files

It's been quite fast from my experience; it's less of our business from the user side, but I'd love to see the Docusaurus codebase staying simple. It would be nice if siteConfig.js's changes are picked up though!

Use a module bundler like webpack

We switched to Docusaurus to avoid some of the more hardcore webpack and ecosystem problems. At the worst time, I was cherry-picking and deploying things manually each push. cc @rickyvetter who'd concur.

Actually, the watcher mode and the startup time has regressed quite a bit these days, but I didn't have the heart to point it out because Docusaurus has been serving us well 😅. It'd be nice to have it fast again... in fact, that's probably one of my biggest request. And I don't mean overengineer the pipeline to get a fast lazy startup; I mean the "good old days" where the startup was simple enough to stay within 1-2s. Webpack will blow past that budget.

Raw CSS without any preprocessor. Hacky variable substitution approach using regexp.

Most other CSS preprocessors aren't much of a step up from basically regexes lol. Given CSS' syntax, a single regex pass isn't actually that hacky; you might replace some words in comments but that's pretty much it. The one from Docusaurus has been blazing fast so far, so that's nice. But given that a preprocessor is opt-in, and given its limited scope, I think letting userland customize it sounds fine. Aka we don't pay the price for not using them, aside from maybe one or two checks in Docusaurus codebase.

Docusaurus controls much of the styles and layout, which makes it prone to breaking end users code who customize.

We're such folks, for all 3 sites. But I'd rather keeping up with the very occasional small layout breaking changes (very quick to check the css changes from changelog and fix them) than having them overly customizable (the point being that such customizability will probably affect Docusaurus' current setups).

All CSS is compiled into one file.

That has been very fine so far! Kinda like how non-nonsense it is.

HTML/CSS is self-written and doesn't follow CSS framework best practices. Typography is not great (margins and paddings are not consistent)

Yeah fixing those would be great!

No 404 page.

Oh yeah, a custom URL redirect json config would be sweet. We made our own for our sites by essentially putting dummy placeholder redirect pages.

Anyway, the gist is that, we really appreciated Docusaurus' non-nonsense toolchain, and whatever depth-related improvements (like even better page layout) are super welcome. Reaching too much for breath and becoming essentially another "website compiler" seems a bit much; after all, we do want to just focus on the documentation, and no matter how hard folks try to compiler-fy websites to regain the ease of use, simplicity and performance, we've not seen such effort that hasn't leaked hard into userland so far. Won't point fingers though. Also, sorry if some of the spirit I've summarized here wasn't part of Docusaurus' initial goal; we might happen to have piggy backed on top of it.

Thanks a lot for your continued work 😃

@muuvmuuv

This comment has been minimized.

Copy link
Contributor

@muuvmuuv muuvmuuv commented Sep 19, 2018

+1 for Gatsby.js. This would make Docusaurus so flexible and modular

@sunnylqm

This comment has been minimized.

Copy link
Contributor

@sunnylqm sunnylqm commented Sep 20, 2018

Can we have a roadmap for v2 like this? react-navigation/react-navigation#4434 @yangshun

@JoelMarcey

This comment has been minimized.

Copy link
Contributor

@JoelMarcey JoelMarcey commented Oct 3, 2018

@chenglou Thank you so much for your thoughtful feedback. I am in 100% agreement with your overall premise that Docusaurus needs to be true to its original charter of "easy and simple" for open source websites. I believe the v2 leads, @endiliey and @yangshun, understand that and believe that too.

The v2 folder has been added to the repo so we can keep tabs on the progress of the implementation.

@yangshun

This comment has been minimized.

Copy link
Member Author

@yangshun yangshun commented Feb 15, 2019

A more detailed v2 document can be found here. Comments are welcome!

Would like to give interested parties a heads up that v2's progress is delayed due to manpower constraints on our end (no one full-time working on the project). We hope to have a beta ready by mid 2019, dogfood on Facebook's Docusaurus sites, and launch in Q4 2019.

@JoelMarcey

This comment has been minimized.

Copy link
Contributor

@JoelMarcey JoelMarcey commented Feb 15, 2019

That said, if anyone is interested in joining our effort as a direct contributor to v2 (or maintainer of v1), let us know. We would love the help.

@ymschaap

This comment has been minimized.

Copy link

@ymschaap ymschaap commented Mar 3, 2019

@yangshun @JoelMarcey I've been following the project for a while now. Have 10+ year experience in front-end development, lots of React+webpack work and would be able to commit time to join the effort on getting 2.0 launched.

@JoelMarcey

This comment has been minimized.

Copy link
Contributor

@JoelMarcey JoelMarcey commented Mar 4, 2019

@yangshun, @endiliey -- what is the best way for @ymschaap and @HenryStevens (who I chatted with offline) to get started helping with v2?

@endiliey

This comment has been minimized.

Copy link
Collaborator

@endiliey endiliey commented Mar 4, 2019

This might need more discussion. But one of the useful thing that can be done for those who wanted to help with v2 is to actually test & play around with the on-going v2 implementation

There is an instruction on how to run it locally.

Edit: Alternatively, join us on discord

@endiliey endiliey pinned this issue Mar 6, 2019
@emanic

This comment has been minimized.

Copy link

@emanic emanic commented Mar 26, 2019

Are there any plans to support variables and templating a la Liquid/Jekyll, go/Hugo? I did not see any mention of adding this in D2, but it would be extremely useful.

@yangshun

This comment has been minimized.

Copy link
Member Author

@yangshun yangshun commented Mar 26, 2019

Are there any plans to support variables and templating a la Liquid/Jekyll, go/Hugo? I did not see any mention of adding this in D2, but it would be extremely useful.

@emanic React is used as a templating library in D1, this is supported out of the box in D1 and will continue to be a thing in D2.

@nebrelbug

This comment has been minimized.

Copy link

@nebrelbug nebrelbug commented Mar 26, 2019

@yangshun it would be very nice to allow customizing template engines as well.

@yangshun

This comment has been minimized.

Copy link
Member Author

@yangshun yangshun commented Mar 26, 2019

@nebrelbug That'll be a little hard because the whole framework will be built upon the React ecosystem as we use React Router, React Helmet. Docusaurus' direction isn't to be so customizable at such a low level. That said, you can still use React as a shell to render your static HTML that's rendered using another templating engine, but I can't think of good reasons to do that.

@endiliey

This comment has been minimized.

Copy link
Collaborator

@endiliey endiliey commented Mar 26, 2019

About the template engines, I think that can be left as a plugin.

Possible example: let’s say we want to create a template that allows you to import code snippet from a file in markdown. That means we’re replacing all the “template syntax” string occurence in markdown with the actual code snippet. This can be done by overriding the webpack markdown-loader.

@nebrelbug

This comment has been minimized.

Copy link

@nebrelbug nebrelbug commented Mar 26, 2019

@yangshun ah, that makes sense.

@endiliey good idea!

@emanic

This comment has been minimized.

Copy link

@emanic emanic commented Apr 11, 2019

Are there any plans to support interactive SVGs, such as one that displays tooltips on mouse hover? I have found that none of the common methods of making the SVG interactive work in the Docusaurus site. For example, let's take the simplest method, using the <title> tag. I can add the <title> tag and I see the tooltip when I open the SVG outside of my Docusaurus website, but when I display the same SVG inside of the Docusaurus site, no joy. Of course, I would ultimately like to do something more sophisticated than just using the <title> tag.

Update: I have found that if I use <embed src="file.svg" type="image/svg+xml" />, the <title> tooltip shows up. This requires some HTML, but it seems to work.

@yangshun yangshun changed the title [RFC] Docusaurus v2 Docusaurus 2 Apr 26, 2019
@ahmadawais

This comment has been minimized.

Copy link

@ahmadawais ahmadawais commented May 1, 2019

I'd appreciate the ability to have multiple links forwarded/redirected to the same document. Iterating on what @endiliey said:

---
title: Getting Started
Links: getting-started, start, go
---

Now with that, I could use short link like docs.mysite.com/go and that goes to docs.mysite.com/getting-started the first link. But also docs.mysite.com/start goes there.

@blling

This comment has been minimized.

Copy link

@blling blling commented May 5, 2019

How to init a Docusaurus v2 project ?

@endiliey

This comment has been minimized.

Copy link
Collaborator

@endiliey endiliey commented May 5, 2019

@lovoop currently we haven't implemented the init part of Docusaurus v2 yet because there are several areas still in progress (e.g: responsive styling, theming, plugins, etc) and we don't want people to actually install it yet/ even using it in production yet.

Edit: If you wanted to give a taste of docusaurus 2. Read this short unofficial blog post of mine https://endiliey.com/2019/05/16/docusaurus-2-technical-preview/

@ericnakagawa

This comment has been minimized.

Copy link
Contributor

@ericnakagawa ericnakagawa commented May 7, 2019

I'd appreciate the ability to have multiple links forwarded/redirected to the same document. Iterating on what @endiliey said:

---
title: Getting Started
Links: getting-started, start, go
---

Now with that, I could use short link like docs.mysite.com/go and that goes to docs.mysite.com/getting-started the first link. But also docs.mysite.com/start goes there.

This would affect how sitemaps are generated, also relates to 301 redirects.

Would the first item in the list be the canonical example that the others are redirected to or should we be generate multiple redundant files?

@afrankel-sfdo

This comment has been minimized.

Copy link

@afrankel-sfdo afrankel-sfdo commented May 8, 2019

didn't see this on the list: per language assets. useful when images contain information that needs to be translated.

@emanic

This comment has been minimized.

Copy link

@emanic emanic commented Jun 17, 2019

Perhaps I can explain more specifically what I am looking to do and you can see if this is something you may want to add to v2, but more importantly I would appreciate some guidance on how I might accomplish it with v1. We have versioned docs and need to display different content to users based on which version of the docs they are looking at. We want to be able to identify what should be replaced in the markdown using something like this {{variable}}. In the immediate term, we need this for download paths inside of code blocks so that they can copy and paste code that gets them the correct client for their version. Longer term, we want to show different chunks of texts and even different navigation based on the version. I saw the discussion here #395, unfortunately, this does not satisfy my use case. I understand that some of this would be in a remarkable plugin (though none exist at present that we can use), but also, we need to be able to set the strings, navigation, content chunks via React. Any tips would be appreciated.

@CGick

This comment has been minimized.

Copy link

@CGick CGick commented Jul 29, 2019

I am migrating my companies product documentation to Docusaurus, and a lot of our pages are referenced by other pages. It would be nice to have a reference mapping file in which I can add all of the link ids and the link address they map to, then in my docs pages just enter the link id and have it pull the link address from the mapping file. If this is currently possible could someone please let me know how this is done.

@joetidee

This comment has been minimized.

Copy link

@joetidee joetidee commented Aug 2, 2019

Is there a rough timeline for the beta and full releases of v2?

@Berkmann18

This comment has been minimized.

Copy link

@Berkmann18 Berkmann18 commented Oct 11, 2019

Is there any estimated release time or something?

@iamandrewluca

This comment has been minimized.

Copy link

@iamandrewluca iamandrewluca commented Oct 14, 2019

Proposal: Evergreen documentation

I don't think this should be implemented into Docusaurus, but to keep in mind.

From times to times it happens that docs remain behind code.
Would be great to have a validator that will test if docs reflect latest code changes. I think it could be used as a prebuild step.

This package allows to validate documentation files
https://github.com/Originate/text-runner

Are there other alternatives?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.