feat(v2): core v2 i18n support + Docusaurus site Crowdin integration

[slorber]

Motivation

Initial i18n POC

See RFC #3317

---

Current WIP state:

Markdown translation

Markdown translations work for docs (including versioning), blog, pages.

Localized markdown files should be put in ./i18n/<locale>/<pluginName><-pluginId>/**/*.md

If the markdown exist at the correct localized location, it will be used in priority over the original document.

Note: some questions asked to Crowdin: https://gist.github.com/slorber/30643299196c7efa77084eec10c1c609

Key/values translations

Key/value translations are helpful to translate the plugin's data (content plugins) and UIs (theme plugins).

Key/value translations use Chrome i18n JSON format:

• it allows adding context/description to messages, useful for translators
• it's widely supported by many SaaS

SaaS documentation:

• https://developer.chrome.com/apps/i18n-messages
• https://support.crowdin.com/file-formats/chrome-json/
• https://www.applanga.com/docs/formats/chrome_i18n_json
• https://docs.transifex.com/formats/chrome-json
• https://help.phrase.com/help/chrome-json-messages

You can "init" the localized JSON files by using the cli command docusaurus write-translations --locales all.

Then you need to modify the JSON files in ./i18n/<locale>/<pluginName><-pluginId>/**/*.json

Code translations

Code translations is a subset of key/value translations that are found in UI / JSX, using the Docusaurus i18n runtime.

You can use translations in pages and components, with a new client API:

import Translate, { translate } from "@docusaurus/Translate";

const Examples = {
  // component API: useful for regular JSX translations
  content: (
    <Translate
      id="homepage.quotes.christopher-chedeau"
      description="Quote of Christopher Chedeau on the home page"
    >
      Default quote message of Christopher Chedeau
    </Translate>
  ),

  // imperative API: useful for placeholders, alt or regular string props...
  title: translate({
    id: "homepage.quotes.christopher-chedeau.title",
    message: "Lead Prettier Developer",
    description: "Title of quote of Christopher Chedeau on the home page"
  })
};

ìdanddescription` are optional args.

Code translations are extracted with the docusaurus write-translations cli at ./i18n/<locale>/code.json

Note: it's not possible to use dynamic translation values, because translations are handled through Babel, the id/message/description must be statically evaluable.

Site config

The initial config is small, but we'll add more customization options later (locale route paths, fs paths, ltr, charset encoding...)

export default {
  i18n: {
    defaultLocale: "en",
    locales: ["en", "fr"]
  }
};

CLI

Note: each locale leads to a single independent SPA site. It's not one SPA site containing all locales, that's why you can't start a Docusaurus site with all locales at once (but we may add this later).

• docusaurus start: start your site on the default locale

• docusaurus start --locale fr: start your site on the french locale

• docusaurus build: will build your site in all available locales

• docusaurus build --locale fr: will only build the french site

• docusaurus write-translations: will extract translations from code and plugins

Use --help on cli for some extra doc.

Plugin lifecycles for localization

As Docusaurus v2 is modular, each plugin is responsible:

• to declare its own translations (ie, write JSON files to FS at the correct location with cli command docusaurus write-translations)
• to read those JS files from FS
• use the FS translations to translate the loaded content (plugin data), or themeConfig (

type Plugin = {
  // return a descriptor of all files that will be written by cli docusaurus write-translations
  getTranslationFiles?(): Promise<TranslationFiles>;

  // will be called after "plugin.loadContent()" with the localized translation files of this plugin
  // you should return the localized content
  // the localized content will be passed to "plugin.contentLoaded()"
  // Usecase: translate docs titles, sidebars, version names...
  translateContent?({
    content,
    translationFiles
  }: {
    content: T; // the content loaded by this plugin instance
    translationFiles: TranslationFiles;
  }): T;

  // Translate the themeConfig
  // Usecase: translate themes like the classic theme (navbar, footer etc...)
  translateThemeConfig?({
    themeConfig,
    translationFiles
  }: {
    themeConfig: ThemeConfig;
    translationFiles: TranslationFiles;
  }): ThemeConfig;
};

Test deployments:

Markdown translation examples (Markdown data from Crowdin):

• https://deploy-preview-3325--docusaurus-2.netlify.app/classic/fr/docs/
• https://deploy-preview-3325--docusaurus-2.netlify.app/classic/fr/blog/2020/01/07/tribute-to-endi
• https://deploy-preview-3325--docusaurus-2.netlify.app/classic/fr/examples/markdownpageexample/

One domain for all locales:

• https://deploy-preview-3325--docusaurus-2.netlify.app/classic/fr/
• https://deploy-preview-3325--docusaurus-2.netlify.app/classic/

One domain per locale:

• https://docusaurus-en.netlify.app/
• https://docusaurus-fr.netlify.app/

[docusaurus-bot]

Deploy preview for docusaurus-2 ready!

Built with commit f7ba5f3

https://deploy-preview-3325--docusaurus-2.netlify.app

[slorber]

Hi everyone.

This PR is now merged, but it's not the end of the story.
We'll test this code on Docusaurus + Jest, before advertising and documenting the i18n support.

In the meantime, you can still use the doc of this PR to figure out how it works, but keep in mind that until the official announcement, it's likely there will be a few breaking changes.

I'll probably look soon to onboard some of you privately before the official i18n release.
If you are interested, please let me know some details to see if you are a good candidate:

• Is your site public, and easy to contribute to? (Github repo + convenient hosting with deploy previews)
• Where do you want to store the translations? ( Git / Git submodule / Crowdin / Other ?)
• Your Discord username?

[krillboi]

Hi @slorber, awesome news!

We'd love to be onboarded as we are ready to translate our whole site (https://support.touchgfx.com/) to Chinese and just need to get started with the integration between Docusaurus and Crowdin.

• Our site is public but not open for contribution.
• We'd like to use Crowdin as translation tool.
• KrillBoi#9646

Looking forward to it!

[jcayzac]

@slorber from the changes it's clear how to provide translations for documents, but what about the website's footer / navbar / title / tagline ?

[jcayzac]

With this in my site config:

    i18n: {
      defaultLocale: 'en',
      locales: ['en', 'ja',],
    },

When I run write-translations -l ja, I get the following error:

Error: Can't write-translation for locale that is not in the locale configuration file.
Unknown locale=[ja].
Available locales=[en]
    at writeTranslations (/Users/julien.cayzac/rakuten/rpay/docs/node_modules/@docusaurus/core/lib/commands/writeTranslations.js:41:15)

(with 2.0.0-alpha.bd62be93d)

[jcayzac]

Nevermind, I'm stupid, I had the i18 object in themeConfig……… 🤦🏼

[clairefro]

Hi @slorber this looks amazing!

Could I please hop on the onboarding train for using i18n with Docusaurus 2?

I am helping JAM stack framework RedwoodJS (https://redwoodjs.com/) start a new site for localizing it's documentation. Docusaurus 2 + Crowdin looks like a stellar combo for us.

• Is your site public, and easy to contribute to? (Github repo + convenient hosting with deploy previews)
  Yes - our new site "Learn RedwoodJS" would be in the same spirit as redwoodjs.com here. Our repo would be public where anyone can add EN content, and translations would be crowdsourced via Crowdin.

• Where do you want to store the translations? ( Git / Git submodule / Crowdin / Other ?)
  Github

• Your Discord username?
  Claire Froelich#3472 <-- hope that works

[slorber]

Hi there

I posted an update of i18n progress here:
#3317 (comment)

There are still a few things to change in Docusaurus core to be able to make Jest production-ready, but this is progressing: https://jest-v2.netlify.app/

[LoicPoullain]

@slorber the translation dropdown seems broken on jest. When I select another language, I'm redirected to the home page with still English as language. Tested on Chrome and Firefox.

PS: Docusaurus is really cool and you're doing a great job. Keep it up! 👍

[slorber]

@LoicPoullain thanks

The locale dropdown is hardcoded for now, will implement this very soon and make Jest production ready :)

[clairefro]

@slorber This may be intentional behavior but just dropping this here as a suggestion

Localized docs based on source docs that point to static assets (like ./static/img/logo.svg) break if the static assets are are not also defined in the localized directory.

Example error for localized version of shipped sample doc1 :

./i18n/fr/docusaurus-plugin-content-docs/current/doc1.md
Error: Image i18n/fr/docusaurus-plugin-content-docs/static/img/logo.svg used in i18n/fr/docusaurus-plugin-content-docs/current/doc1.md not found.
    at async Promise.all (index 1)

While this is a chance to optionally localize images that contain diagrams/text, it would be great if it were not required to copy assets to all locales.

Instead the i18n asset paths could resolve to the source assets by default, just like the i18n markdown files default to the source markdowns if no locale counterpart is found.

[slorber]

hi @clairefro

If an asset is colocated, this means the asset will be duplicated for both versioning and localization, allowing you to use a different asset per version/locale (some images might contain localized text).

The localized doc folder should rather contain a full copy of the upstream doc folder, so that all relative imports keep working, or you should avoid using relative imports/ asset paths.

Note: if you don't need to version/localize an asset, you can put your svg in static and import it with @site/static/img/logo.svg

In the future, I'd like to allow "partial" localized doc sets more easily, and fallback to the original upstream doc if localized doc/asset does not exist, but it requires that I figure out how to do this with Webpack module resolution systems. This will not be part of the initial i18n release but will come after.

This will fallback as you expect:

• upstream doc exist but localized doc does not exist
• localized doc 1 link to ./doc2.md but doc2 localized doc does not exist

This won't fallback (as of today):

• doc1 is localized and use a relative image path that does not exist: [](./colocatedImage.png)
• doc1 is localized and import a mdx partial that is not localized: import MDXPartial from "./partials/_mdxPartial.mdx"

[krillboi]

Hi @slorber,

I am starting out with trying to use the i18n feature and looked at the Docusaurus v2 project on Crowdin. I noticed that compared to Docusaurus v1, the source files are a lot more messy to look at and understand for translators due to import statements and markdown/html cluttering up the view.

Take a look at: https://crowdin.com/translate/docusaurus-v2/128/en-fr?filter=basic&value=0

Compared to: https://crowdin.com/translate/docusaurus/2524/en-fr?filter=basic&value=0

Links in the v1 document are evaluated as markdown while v2 is just raw text (for example a lot of divs). In our docs, we use a lot of different React components, so this seems like it might be difficult for translators without a technical background to read and translate.

Is this something that can be fixed?

[slorber]

Hi @krillboi

The i18n doc will soon be available.

I noticed that compared to Docusaurus v1, the source files are a lot more messy to look at and understand for translators due to import statements and markdown/html cluttering up the view.

You mean due to the fs organization?

Or just the embedded React components?

How is it more annoying than embedded HTML?

Links in the v1 document are evaluated as markdown while v2 is just raw text (for example a lot of divs).

That seems related to using the .mdx extension instead of .md, and how it is interpreted by Crowdin.
This v2 doc seems to work better for you no?
https://crowdin.com/translate/docusaurus-v2/126/en-fr?filter=basic&value=0

I see advantages of using the MDX extension, as IDE's can understand the embedded React code, Prettier can format it etc...
However, there's no first-class support for MDX in Crowdin.

You can "force" any extension to be interpreted as markdown in crowdin.yaml:
https://support.crowdin.com/configuration-file/#uploading-files-to-specified-path-with-specified-type

Their customer support can also enable custom processors too (not possible through the UI afaik).

Also, if Crowdin ask translators to translate strings that should rather not be translated (like mdx import statements), it is possible for you to use the "hide strings" feature so that only the crowdin admins can translate these strings

---

Is this something that can be fixed?

I'm in touch with Crowdin to see how to fix these problems, but keep in mind that Crowdin is an external SaaS software on which I have absolutely 0 control myself, and it's also not the one and only solution that you can use to translate a docusaurus project.

The Docusaurus v2 i18n is not coupled to any SaaS or Crowdin, and it just requires you to put files at the correct path.

We will try to document Crowdin usage the best we can, but in the end, if you want to use Crowdin to translate your project, you should understand how it works and read its doc, and also get in touch with their support to figure things out if needed.

In my opinion, Crowdin is far from perfect, but works not too badly either. There might be better translation SaaS alternatives that I'm not aware of, unfortunately don't have time to explore those, but hopefully some community contributors will be able to provide feedbacks and write documentation about other possible solutions?

Treat i18n in the same way as deployment doc: we historically document well the GitHub pages deployment, but many other options are possible. And in the end, you have to read the doc and understand the tradeoffs of the underlying solution.

[krillboi]

@slorber ah I understand, specifying the uploaded type as .md is a good first step I think and then a custom processor .mdx later maybe would make sense.

No worries, was just curious if anything was on the horizon 👍

[slorber]

I'm constantly giving feedback to Crowdin support so that they can improve their product.

Here's just a few of my initial questions, but had many more recent ones 😉
https://gist.github.com/slorber/30643299196c7efa77084eec10c1c609

And they may implement some of my suggestions (they already did) but it may take time and I don't know when they will be released

[slorber]

Hi there.

You'll find a i18n documentation preview on #4014 at https://deploy-preview-4014--docusaurus-2.netlify.app/classic/docs/next/i18n/introduction

Please give me feedbacks on the documentation PR if some things are not clear enough