i18n overhaul #365
i18n overhaul #365
Conversation
✅ Deploy Preview for astro-docs-2 ready!Built without sensitive environment variables
To edit notification comments on pull requests, go to your Netlify site settings. |
Decision made based on this high tech, large sample size polling data: https://discord.com/channels/830184174198718474/867764597838184478/966981090831126559
Relies only on the existence of `.md` files to decide how to generate fallback pages rather than relying on the navigation contents as previously. This is possible now that slugs are consistent across locales and helps for cases where a nav link isn’t for an actual page (“Installation” currently hits a redirect route for example).
Brings in some fixes for Markdown Content component
Something about how components are rendered goes wrong on build (not during dev) if we pass `Content` out of `getStaticProps` via props. Explicitly globbing the content again inside the page itself seems to fix this 🤷
We tag navigation sidebar entries as “fallback” content based on if the slug exists for a given language. Previously I did this using Vite’s `import.meta.glob('../pages/**/*.md)` to grab the URLs of all pages. Although this shouldn’t load content (you need `globEager` for that), this still seemed to trigger Astro processing *all* Markdown pages. Unsurprisingly, this was slow — HMR of ~10s for a single page edit 😬
I’ve replaced the glob by recursively reading the pages directory when needed using `fs.readdir`. Avoiding globbing also means we can target *only* the relevant language instead of all pages. That brings reloads back down to 100–200ms in dev.
There is a disadvantage: if new pages are added while running `dev`, the navbar may be cached for previously visited pages and because we’re not using `glob`, Vite/Astro don’t know we’re depending on files, so won’t invalidate that cache. Not sure if there’s a workaround? But still, this solution seems preferable to waiting 10 seconds each time you make an edit.
|
Woo! I went through and tested everything listed in the preview, looked at the new file structure and all that seems to be working well. 🥳 Pretty slick! To address the points raised in "areas for review" and "constraints" here are my thoughts/observations! Key Areas For Review
(There were a couple of sentences ending in quotation marks where I'd tuck the punctuation in, but there's nothing blocking here.)
Contraints
|
| } | ||
|
|
||
| const { fallback } = Astro.params | ||
| const englishPages = await Astro.glob('../en/**/*.md'); |
There was a problem hiding this comment.
FYI you should be able to do this:
const { fallback } = Astro.params
const foo = await import(`../en/${fallback}.md`);
however, it looks like its not returning the right thing due to a Vite bug, so Astro.glob() is fine until that gets resolved.
There was a problem hiding this comment.
@delucis this looks great, thank you for tackling something so huge in scope with such an attention to detail! Those two things are usually not easy to do together :)
I've given it a look and I don't have too many comments! Given the scope I'd prefer to merge this sooner rather than later to prevent merge conflicts.
I want to take a look at some of the perf issues you've run into, and maybe suggest a couple of follow up PRs. Sadly, some of them may be in Vite instead of Astro, and how they eagerly scan/build files in a project.
|
Suggested follow-ups, based on the convo above:
RE project layout: At this level of complexity, I would suggest that we begin to look into moving all markdown content out of |
This PR takes most of the structural steps to get #359 off the ground.
Summary
It’s a big PR and touches a lot, so here’s a summary of what’s going on:
All non-English pages except for “Getting Started” have been removed. (They’re archived in
old-translations/for now for reference, but we can delete them at some point in the future.)Every language now has its own directory in
src/i18n. This includes translations for UI strings as well as the navigation sidebar (which used to live insrc/config.ts).Every language now uses the same navigation sidebar structure as English. If translations are found in
src/i18n/{lang}/nav.ts, these will be used instead of the English label. If a page is not yet available in this language, the page is still generated, but using fallback content from the English version. These are marked with an EN suffix in the sidebar.For example, here’s
/de/editor-setup/. Its sidebar text is translated — “Editor-Einrichtung” — but the page is not and shows a notice explaining this. This notice can be translated insrc/i18n/{lang}/ui.ts.src/i18n/README.mdcontains a guide on how to contribute translationsA new script provides automated scaffolding of all of the required files for a new language:
pnpm run add-languageadd-language.mov
Smaller details
404 pages are now translatable in the same way as other UI strings and are generated automatically for each language
Bare language URLs redirect to Getting Started, e.g.
/de→/de/getting-startedFollowing feedback on Discord, this system uses consistent slugs across locales
The language switcher now preserves the current page when the language changes instead of always redirecting to “Getting Started”
Had to upgrade to the latest Astro beta to get some bug fixes for the markdown
Contentcomponent.SEO: added
hreflangalternate links to connect up translations across languages and made sure that pages displaying fallback content use the English URL as their canonical.Key areas for review
Evaluate
src/pages/[lang]/[...fallback].astro. Is this approach OK for rendering the fallback pages? Any improvements possible?I ran into a few performance hurdles, see
adb545dfor one detailed explanation.What is still needed in
src/i18n/README.md? We need a list of supported languages obviously, but otherwise?Should this PR also prune the number of supported languages or will we do that later? Or, in the other direction, should this PR preserve potentially out-of-date translations for now instead of moving them to
old-translations/?How does the translation file structure in
src/i18nfeel? German (de) is a good language to check as it has bits of everything translated.Constraints introduced
The fallback content system relies on
Astro.glob, so all pages need to be.mdfiles rather than.astro. Might be possible once the metadata returned byAstro.globfor Astro files is improved to expand that, but for now we’ll need to stick to Markdown.Because slugs are shared across locales, Markdown files need to have the same name across the different language directories and these need to match the slugs in
src/i18n/en/nav.ts. I haven't added any tooling around that, but we could.