Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(v2): core v2 i18n support + Docusaurus site Crowdin integration #3325

Merged
merged 130 commits into from Nov 26, 2020
Merged
Show file tree
Hide file tree
Changes from 98 commits
Commits
Show all changes
130 commits
Select commit Hold shift + click to select a range
c9ec765
docs i18n initial poc
slorber Aug 21, 2020
9d215e6
docs i18n initial poc
slorber Aug 21, 2020
5e33dad
docs i18n initial poc
slorber Aug 21, 2020
d7cc391
docs i18n initial poc
slorber Aug 21, 2020
0eeea32
merge from master
slorber Sep 3, 2020
ff35e45
Merge branch 'master' into slorber/i18n
slorber Sep 3, 2020
30b6039
crowdin-v2 attempt
slorber Sep 3, 2020
382a0df
fix source
slorber Sep 3, 2020
ded8168
Merge branch 'master' into slorber/i18n
slorber Sep 4, 2020
5d97601
use crowdin env variable
slorber Sep 4, 2020
e698e73
try to install crowdin on netlify
slorber Sep 4, 2020
2ec2075
try to install crowdin on netlify
slorber Sep 4, 2020
6270093
try to use crowdin jar directly
slorber Sep 4, 2020
cea9b42
try to curl the crowdin jar
slorber Sep 4, 2020
ab98435
add java version cmd
slorber Sep 4, 2020
08c8ba3
try to run crowdin jar in netlify
slorber Sep 7, 2020
3fede62
fix translatedDocsDirPath
slorber Sep 7, 2020
1c7a8e8
fix loadContext issue due to site baseUrl not being modified in gener…
slorber Sep 7, 2020
54746e5
real validateLocalesFile
slorber Sep 7, 2020
951e713
add locale option to deploy command
slorber Sep 7, 2020
8615d2f
better LocalizationFile type
slorber Sep 7, 2020
21a4cec
create util getPluginI18nPath
slorber Sep 7, 2020
d00a124
better core localization context loading code
slorber Sep 7, 2020
c3ea993
More explicit VersionMetadata type for localized docs folders
slorber Sep 7, 2020
14212ed
Ability to translate blog posts with Crowdin!
slorber Sep 7, 2020
f61da9e
blog: refactor markdown loader + report broken links + try to get lin…
slorber Sep 8, 2020
590bbdd
upgrade crowdin config to upload all docs folder files except source …
slorber Sep 8, 2020
ab15901
try to support translated pages
slorber Sep 8, 2020
18cbe9d
make markdown pages translation work
slorber Sep 8, 2020
b54b6a6
Merge branch 'master' into slorber/i18n
slorber Oct 20, 2020
15a7fb1
Merge branch 'master' into slorber/i18n
slorber Oct 21, 2020
ed626fb
add write-translations cli command template
slorber Oct 21, 2020
834d65f
fix site not reloaded with correct options
slorber Oct 22, 2020
5cc0c2e
refactor a bit the read/write of @generated/i18n.json file
slorber Oct 22, 2020
8038448
Add <Translate> + translate() API + use it on the docusaurus homepage
slorber Oct 23, 2020
96b6e04
watch locale translation dir
slorber Oct 23, 2020
4e1a072
early POC of adding babel parsing for translation extraction
slorber Oct 23, 2020
cfcf7e5
Merge branch 'master' into slorber/i18n
slorber Oct 26, 2020
0163f91
fs.stat => pathExists
slorber Oct 26, 2020
8b7e0b8
Merge branch 'master' into slorber/i18n
slorber Oct 27, 2020
b870b14
add install:fast script
slorber Oct 27, 2020
b44cb62
TSC: noUnusedLocals false as it's already checked by eslint
slorber Oct 27, 2020
72193a5
POC of extracting translations from source code
slorber Oct 27, 2020
0c71e7c
Merge branch 'master' into slorber/i18n
slorber Nov 3, 2020
d49376d
minor typo
slorber Nov 3, 2020
4affc2f
fix extracted key to code
slorber Nov 3, 2020
4c29a93
initial docs extracted translations
slorber Nov 3, 2020
ee5de79
stable plugin translations POC
slorber Nov 4, 2020
fb8ce3c
add crowdin commands
slorber Nov 5, 2020
5c0f72a
Merge branch 'master' into slorber/i18n
slorber Nov 5, 2020
4d28880
quickfix for i18n deployment
slorber Nov 5, 2020
2bdb9b8
Merge branch 'master' into slorber/i18n
slorber Nov 5, 2020
c733d33
POC of themeConfig translation
slorber Nov 6, 2020
4f1cbee
add ability to have localized site without path prefix
slorber Nov 6, 2020
5f1e97d
Merge branch 'master' into slorber/i18n
slorber Nov 9, 2020
b378467
sidebar typo
slorber Nov 10, 2020
97b08cf
refactor translation system to output multiple translation files
slorber Nov 11, 2020
3d78a4e
translate properly the docs plugin
slorber Nov 11, 2020
7151661
improve theme classic translation
slorber Nov 11, 2020
bacf709
rework translation extractor to handle new Chrome I18n JSON format (i…
slorber Nov 11, 2020
1375d84
writeTranslations: allow to pass locales cli arg
slorber Nov 11, 2020
af51ef3
fix ThemeConfig TS issues
slorber Nov 11, 2020
f457f03
fix localizePath errors
slorber Nov 11, 2020
be70337
temporary add write-translations to netlify deploy preview
slorber Nov 11, 2020
829a33c
Merge branch 'master' into slorber/i18n
slorber Nov 12, 2020
f9d1614
complete example of french translated folder
slorber Nov 12, 2020
e2a6576
Merge branch 'master' into slorber/i18n
slorber Nov 13, 2020
b8162fa
Merge branch 'master' into slorber/i18n
slorber Nov 13, 2020
6148000
update fr folder
slorber Nov 13, 2020
5d602d2
remove all translations from repo
slorber Nov 13, 2020
8e828fd
minor translation refactors
slorber Nov 13, 2020
b9109fd
fix all docs-related tests
slorber Nov 13, 2020
bc859bd
fix blog feed tests
slorber Nov 13, 2020
ac34b90
fix last blog tests
slorber Nov 13, 2020
2b02129
refactor i18n context a bit, extract codeTranslations in an extra gen…
slorber Nov 13, 2020
8a2c5b6
improve @generated/i18n type
slorber Nov 13, 2020
e118561
fix some i18n todos
slorber Nov 13, 2020
2d5d767
minor refactor
slorber Nov 13, 2020
3b5238f
Merge branch 'master' into slorber/i18n
slorber Nov 16, 2020
4de5c7a
Merge branch 'master' into slorber/i18n
slorber Nov 17, 2020
61bc977
fix logo typing issue after merge
slorber Nov 17, 2020
3fc4703
move i18n.json to siteConfig instead
slorber Nov 17, 2020
9c967e8
try to fix windows CI build
slorber Nov 17, 2020
18ca731
fix config test
slorber Nov 17, 2020
14819cc
attempt to fix windows non-posix path
slorber Nov 17, 2020
3b0200e
increase v1 minify css jest timeout due to flaky test
slorber Nov 17, 2020
7ac7f39
proper support for localizePath on windows
slorber Nov 17, 2020
ee296ed
remove non-functional install:fast
slorber Nov 17, 2020
5291bd5
docs, fix docsDirPathLocalized
slorber Nov 17, 2020
baecbd4
fix Docs i18n / md linkify issues
slorber Nov 17, 2020
a0be4e4
ensure theme-classic swizzling will use "nextjs" sources (transpiled …
slorber Nov 17, 2020
d6cf657
fix some snapshots
slorber Nov 17, 2020
c489aa7
improve themeConfig translation code
slorber Nov 17, 2020
efcb0e9
refactor a bit getPluginI18nPath
slorber Nov 17, 2020
8423f0f
readTranslationFileContent => ensure files are valid, fail fast
slorber Nov 17, 2020
ad77266
fix versions tests
slorber Nov 17, 2020
315f937
add extractSourceCodeAstTranslations comments/resource links
slorber Nov 17, 2020
b245f24
ignore eslint: packages/docusaurus-theme-classic/lib-next/
slorber Nov 17, 2020
839ecb4
fix windows CI with cross-env
slorber Nov 18, 2020
99f9cca
Merge branch 'master' into slorber/i18n
slorber Nov 19, 2020
1b6365a
crowdin ignore .DS_Store
slorber Nov 19, 2020
9595c7d
improve writeTranslations + add exhaustive tests for translations.ts
slorber Nov 20, 2020
ae247dc
remove typo
slorber Nov 20, 2020
bed2997
Wire currentLocale to algolia search
slorber Nov 20, 2020
1e40135
improve i18n locale error
slorber Nov 20, 2020
7939f74
Add tests for translationsExtractor.ts
slorber Nov 20, 2020
e464ccc
better code translation extraction regarding statically evaluable code
slorber Nov 20, 2020
21cfa87
Merge branch 'master' into slorber/i18n
slorber Nov 24, 2020
0213bf9
fix typo
slorber Nov 24, 2020
346bb8b
fix typo
slorber Nov 24, 2020
40859e1
improve theme-classic transpilation
slorber Nov 24, 2020
0a98764
refactor + add i18n tests
slorber Nov 24, 2020
ce80a4b
typo
slorber Nov 24, 2020
ee0271c
test new utils
slorber Nov 24, 2020
5bafd66
add missing snapshots
slorber Nov 24, 2020
eeeb08b
fix snapshot
slorber Nov 24, 2020
ea6aa2a
blog onBrokenMarkdownLink
slorber Nov 24, 2020
25b0395
add sidebars tests
slorber Nov 24, 2020
c1f433f
theme-classic index should now use ES modules
slorber Nov 24, 2020
df82b1a
tests for theme-classic translations
slorber Nov 25, 2020
12c2ad7
useless comment
slorber Nov 25, 2020
81c3176
add more translation tests
slorber Nov 25, 2020
fc2584a
simplify/cleanup writeTranslations
slorber Nov 25, 2020
a2298d0
try to fix Netlify fr deployment
slorber Nov 25, 2020
d2eeb2a
blog: test translated md is used during feed generation
slorber Nov 25, 2020
d80b1dd
blog: better i18n tests regarding editUrl + md translation application
slorber Nov 25, 2020
6494c95
more i18n tests for docs plugin
slorber Nov 25, 2020
34c5fcb
more i18n tests for docs plugin
slorber Nov 25, 2020
5ffa942
Add tests for pages i18n
slorber Nov 25, 2020
f7ba5f3
polish docusaurus build i18n logs
slorber Nov 25, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1 change: 1 addition & 0 deletions .eslintignore
Expand Up @@ -24,6 +24,7 @@ packages/docusaurus-plugin-sitemap/lib/
packages/docusaurus-plugin-ideal-image/lib/
packages/docusaurus-plugin-ideal-image/copyUntypedFiles.js
packages/docusaurus-theme-classic/lib/
packages/docusaurus-theme-classic/lib-next/
packages/docusaurus-theme-bootstrap/lib/
packages/docusaurus-migrate/lib/

Expand Down
7 changes: 6 additions & 1 deletion .gitignore
Expand Up @@ -28,11 +28,16 @@ packages/docusaurus-plugin-debug/lib/
packages/docusaurus-plugin-sitemap/lib/
packages/docusaurus-plugin-ideal-image/lib/
packages/docusaurus-theme-classic/lib/
packages/docusaurus-theme-classic/lib-next/
packages/docusaurus-theme-bootstrap/lib/
packages/docusaurus-migrate/lib/

website/netlifyDeployPreview
website/netlifyDeployPreview/*
!website/netlifyDeployPreview/index.html
!website/netlifyDeployPreview/_redirects

website-1.x-migrated

website/i18n/**/*
#!website/i18n/fr
#!website/i18n/fr/**/*
142 changes: 142 additions & 0 deletions crowdin-v2.yaml
@@ -0,0 +1,142 @@
#
# Your Crowdin credentials
#
'project_id': '416738'
'api_token_env': 'CROWDIN_PERSONAL_TOKEN'
'base_path': '.'
'base_url': 'https://api.crowdin.com'

#
# Choose file structure in Crowdin
# e.g. true or false
#
'preserve_hierarchy': true

#
# Files configuration
#
files:
[
{
'source': '/website/i18n/en/**/*',
'translation': '/website/i18n/%two_letters_code%/**/%original_file_name%',
},
{
'source': '/website/docs/**/*',
'translation': '/website/i18n/%two_letters_code%/docusaurus-plugin-content-docs/current/**/%original_file_name%',
},
{
'source': '/website/community/**/*',
'translation': '/website/i18n/%two_letters_code%/docusaurus-plugin-content-docs-community/current/**/%original_file_name%',
},
{
'source': '/website/versioned_docs/**/*',
'translation': '/website/i18n/%two_letters_code%/docusaurus-plugin-content-docs/**/%original_file_name%',
},
{
'source': '/website-1.x/blog/**/*',
'translation': '/website/i18n/%two_letters_code%/docusaurus-plugin-content-blog/**/%original_file_name%',
},
{
'source': '/website/src/pages/**/*',
'translation': '/website/i18n/%two_letters_code%/docusaurus-plugin-content-pages/**/%original_file_name%',
'ignore': ['/**/*.js', '/**/*.jsx', '/**/*.ts', '/**/*.tsx', '/**/*.css'],
},
]
#
# Source files filter
# e.g. "/resources/en/*.json"
#
#"source" : "/website/docs/**/*.md",
#
# Where translations will be placed
# e.g. "/resources/docs/%two_letters_code%/%original_file_name%"
#
#"translation" : "/website/i18n/%language%/docs/current/%original_file_name%",
#
# Files or directories for ignore
# e.g. ["/**/?.txt", "/**/[0-9].txt", "/**/*\?*.txt"]
#
#"ignore" : [],
#
# The dest allows you to specify a file name in Crowdin
# e.g. "/messages.json"
#
#"dest" : "",
#
# File type
# e.g. "json"
#
#"type" : "",
#
# The parameter "update_option" is optional. If it is not set, after the files update the translations for changed strings will be removed. Use to fix typos and for minor changes in the source strings
# e.g. "update_as_unapproved" or "update_without_changes"
#
#"update_option" : "",
#
# Start block (for XML only)
#
#
# Defines whether to translate tags attributes.
# e.g. 0 or 1 (Default is 1)
#
# "translate_attributes" : 1,
#
# Defines whether to translate texts placed inside the tags.
# e.g. 0 or 1 (Default is 1)
#
# "translate_content" : 1,
#
# This is an array of strings, where each item is the XPaths to DOM element that should be imported
# e.g. ["/content/text", "/content/text[@value]"]
#
# "translatable_elements" : [],
#
# Defines whether to split long texts into smaller text segments
# e.g. 0 or 1 (Default is 1)
#
# "content_segmentation" : 1,
#
# End block (for XML only)
#
#
# Start .properties block
#
#
# Defines whether single quote should be escaped by another single quote or backslash in exported translations
# e.g. 0 or 1 or 2 or 3 (Default is 3)
# 0 - do not escape single quote;
# 1 - escape single quote by another single quote;
# 2 - escape single quote by backslash;
# 3 - escape single quote by another single quote only in strings containing variables ( {0} ).
#
# "escape_quotes" : 3,
#
# Defines whether any special characters (=, :, ! and #) should be escaped by backslash in exported translations.
# e.g. 0 or 1 (Default is 0)
# 0 - do not escape special characters
# 1 - escape special characters by a backslash
#
# "escape_special_characters": 0
#
#
# End .properties block
#
#
# Often software projects have custom names for the directories where translations are placed. crowdin-cli allows you to map your own languages to be understandable by Crowdin.
#
#"languages_mapping" : {
# "two_letters_code" : {
# "crowdin_language_code" : "local_name"
# }
#},
#
# Does the first line contain header?
# e.g. true or false
#
#"first_line_contains_header" : true,
#
# for spreadsheets
# e.g. "identifier,source_phrase,context,uk,ru,fr"
#
# "scheme" : "",
11 changes: 5 additions & 6 deletions examples/bootstrap/docs/doc1.md
Expand Up @@ -29,9 +29,9 @@ To serve as an example page when styling markdown based Docusaurus sites.

## Emphasis

Emphasis, aka italics, with *asterisks* or _underscores_.
Emphasis, aka italics, with _asterisks_ or _underscores_.

Strong emphasis, aka bold, with **asterisks** or __underscores__.
Strong emphasis, aka bold, with **asterisks** or **underscores**.

Combined emphasis with **asterisks and _underscores_**.

Expand All @@ -48,11 +48,11 @@ Strikethrough uses two tildes. ~~Scratch this.~~
1. Ordered sub-list
1. And another item.

* Unordered list can use asterisks
- Unordered list can use asterisks

- Or minuses
* Or minuses

+ Or pluses
- Or pluses

---

Expand Down Expand Up @@ -92,7 +92,6 @@ Images from any folder can be used by providing path to file. Path should be rel

![img](../static/img/logo.svg)


---

## Code
Expand Down
10 changes: 5 additions & 5 deletions examples/classic/docs/doc1.md
Expand Up @@ -29,9 +29,9 @@ To serve as an example page when styling markdown based Docusaurus sites.

## Emphasis

Emphasis, aka italics, with *asterisks* or _underscores_.
Emphasis, aka italics, with _asterisks_ or _underscores_.

Strong emphasis, aka bold, with **asterisks** or __underscores__.
Strong emphasis, aka bold, with **asterisks** or **underscores**.

Combined emphasis with **asterisks and _underscores_**.

Expand All @@ -48,11 +48,11 @@ Strikethrough uses two tildes. ~~Scratch this.~~
1. Ordered sub-list
1. And another item.

* Unordered list can use asterisks
- Unordered list can use asterisks

- Or minuses
* Or minuses

+ Or pluses
- Or pluses

---

Expand Down
10 changes: 5 additions & 5 deletions examples/facebook/docs/doc1.md
Expand Up @@ -29,9 +29,9 @@ To serve as an example page when styling markdown based Docusaurus sites.

## Emphasis

Emphasis, aka italics, with *asterisks* or _underscores_.
Emphasis, aka italics, with _asterisks_ or _underscores_.

Strong emphasis, aka bold, with **asterisks** or __underscores__.
Strong emphasis, aka bold, with **asterisks** or **underscores**.

Combined emphasis with **asterisks and _underscores_**.

Expand All @@ -48,11 +48,11 @@ Strikethrough uses two tildes. ~~Scratch this.~~
1. Ordered sub-list
1. And another item.

* Unordered list can use asterisks
- Unordered list can use asterisks

- Or minuses
* Or minuses

+ Or pluses
- Or pluses

---

Expand Down
2 changes: 2 additions & 0 deletions package.json
Expand Up @@ -29,6 +29,8 @@
"serve:v2:ssl:gencert": "openssl req -x509 -nodes -days 365 -newkey rsa:4096 -subj \"/C=US/ST=Docusaurus/L=Anywhere/O=Dis/CN=localhost\" -keyout ./website/.docusaurus/selfsigned.key -out ./website/.docusaurus/selfsigned.crt",
"serve:v2:ssl:message": "echo '\n\n\nServing Docusaurus with HTTPS on localhost requires to disable the Chrome security: chrome://flags/#allow-insecure-localhost\n\n\n'",
"serve:v2:ssl:serve": "serve website/build --ssl-cert ./website/.docusaurus/selfsigned.crt --ssl-key ./website/.docusaurus/selfsigned.key",
"crowdin:upload:v2": "crowdin upload sources --config ./crowdin-v2.yaml",
"crowdin:download:v2": "crowdin download --config ./crowdin-v2.yaml",
"changelog": "lerna-changelog",
"postinstall": "yarn lock:update && yarn build:packages",
"prettier": "prettier --config .prettierrc --write \"**/*.{js,ts}\"",
Expand Down
2 changes: 1 addition & 1 deletion packages/docusaurus-1.x/lib/server/__tests__/utils.test.js
Expand Up @@ -40,7 +40,7 @@ describe('server utils', () => {
expect(css).toMatchSnapshot();

await expect(utils.minifyCss(notCss)).rejects.toMatchSnapshot();
});
}, 10000);

test('autoprefix css', async () => {
const testCss = fs.readFileSync(
Expand Down
26 changes: 26 additions & 0 deletions packages/docusaurus-module-type-aliases/src/index.d.ts
Expand Up @@ -47,6 +47,20 @@ declare module '@generated/globalData' {
export default globalData;
}

declare module '@generated/i18n' {
const i18n: {
defaultLocale: string;
locales: [string, ...string[]];
currentLocale: string;
};
export default i18n;
}

declare module '@generated/codeTranslations' {
const codeTranslations: Record<string, string>;
export default codeTranslations;
}

declare module '@theme/*';

declare module '@theme-original/*';
Expand All @@ -69,6 +83,18 @@ declare module '@docusaurus/Link' {
export default Link;
}

declare module '@docusaurus/Translate' {
type TranslateProps = {children: string; id?: string; description?: string};
const Translate: (props: TranslateProps) => JSX.Element;
export default Translate;

export function translate(param: {
message: string;
id?: string;
description?: string;
}): string;
}

declare module '@docusaurus/router' {
export const Redirect: (props: {to: string}) => import('react').Component;
export function matchPath(
Expand Down
4 changes: 2 additions & 2 deletions packages/docusaurus-plugin-client-redirects/src/index.ts
Expand Up @@ -14,7 +14,7 @@ import writeRedirectFiles, {
toRedirectFilesMetadata,
RedirectFileMetadata,
} from './writeRedirectFiles';
import {removePrefix} from '@docusaurus/utils';
import {removePrefix, addLeadingSlash} from '@docusaurus/utils';

export default function pluginClientRedirectsPages(
_context: LoadContext,
Expand All @@ -27,7 +27,7 @@ export default function pluginClientRedirectsPages(
async postBuild(props: Props) {
const pluginContext: PluginContext = {
relativeRoutesPaths: props.routesPaths.map(
(path) => `/${removePrefix(path, props.baseUrl)}`,
(path) => `${addLeadingSlash(removePrefix(path, props.baseUrl))}`,
),
baseUrl: props.baseUrl,
outDir: props.outDir,
Expand Down
@@ -0,0 +1,11 @@
---
title: This post links to another one!
---

[Good link 1](2018-12-14-Happy-First-Birthday-Slash.md)

[Good link 2](./2018-12-14-Happy-First-Birthday-Slash.md)

[Bad link 1](postNotExist1.md)

[Bad link 1](./postNotExist2.mdx)
@@ -1,5 +1,20 @@
// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`report broken markdown links 1`] = `
"---
title: This post links to another one!
---

[Good link 1](/blog/2018/12/14/Happy-First-Birthday-Slash)

[Good link 2](/blog/2018/12/14/Happy-First-Birthday-Slash)

[Bad link 1](postNotExist1.md)

[Bad link 1](./postNotExist2.mdx)
"
`;

exports[`transform to correct link 1`] = `
"---
title: This post links to another one!
Expand Down