New i18n process

[delucis]

Opening an issue to keep track of the things we need to get the new i18n process ready. This is partly for myself as I can tackle a bunch of this, but also so others can point out other improvements and requirements or generally give feedback — or even jump in and help out! The more the merrier.

To-do

• Ping relevant roles in #docs-i18n on Discord every “Translation Tuesday”. #1057

• Track UI translation progress in the Translation Status Overview #1058

• Add "In Progress" section to the Translation Overview Status #1059

• Display the "freshness"/status of a translated page #1481

• Can we manually “clear” a language from the overview issue if a contributor confirms it is actually up-to-date?

• Improve and highlight the i18n Guide #1060

  Hippo re: keywords:

  if anyone wonders what needs to happen for a previously translated page to switch to the "needs updating" state: If changes are committed to the English source page after the translation got committed, the new commits are parsed and if there is a commit that doesn't include any of the terms (minor|typo|broken link|i18nReady), this commit is considered "major" and therefore triggers the state change of the translated versions.

  Hippo re: “clearing”:

  Similar to the explanation in the script in lines 271-272, you can achieve this by editing the summary issue and setting the respective translation’s UTC date in the state data to the current date.

Done

Done in #365

• Write a guide on contributing translations.

  • Cover the two sources of translations: pages and UI strings.
  • Explain the new Astro i18n policy posted on Discord.
  • Instructions on keeping forks updated. App option recommended by @sarah11918: https://github.com/wei/pull

• Add a fallback content system to surface English content for languages that aren’t fully translated.

  • Fetch English pages and inject their content where it is missing in another language
  • Add a translatable notice explaining that this page isn’t yet translated and that’s why you’re seeing the English version.
  • Add a link to instructions on how to get involved translating content to the notice.
  • Side quest: this would make all pages reliably “available” across languages. We should update the language switcher select menu to preserve the current page when switching between languages (currently it always links to Getting Started).

• Sidebars! (related to above, see Discord thread).

  • Include all pages in sidebar even if they’re not translated and will show fallback content (seems to be the consensus on Discord).
  • Indicate in the sidebar that pages are in the fallback language, e.g. with a [EN] suffix or something.

• Remove all current non-English pages except for Getting Started (keep them in the repo for reference but don’t publish them, potentially using the drafts option). All other pages would use the fallback content for now.

Done in #431

• Automate documentation updates. A CI job checks if any English pages were updated more recently than non-English pages and generates an issue listing out-of-date pages.
• Can the CI job estimate how out of date a non-English doc is? If a single line changed in the English because there was a typo, a non-English doc being 2 months behind is no big deal, but if a lot of content changed, it would be good to indicate that in the to-do list. If a large amount of content becomes out of date, we might want to consider unpublishing the translation in favour of the English fallback. That should be a manual decision I think — it would be very hard to automate reliably.
• Related: if we only fixed a typo in English, the other languages likely don’t need updating. Can we mark typo fixes in some way to be ignored by this automation?

Done in #439

• Link to the auto-generated to-do lists from the translation how-to guide.

Done in #446

• Add a “Create Page” link to the overview issue for missing pages (see Feat: Add translation status script and workflow #431 (comment))

---

Comment below if you think something is missing!

[sarah11918]

Under "Write a guide" - would it be helpful if we created/kept a list of terms not to be translated (ie. use the English word or phrase in all cases)

• some to come from Astro core (whatever they want kept in English, for whatever reason)
• many to come from our translators, who have experience knowing what they'd reasonably expect to be in English in tech docs, or what might end up being confusing if translate

OR, is it just assumed that there's a set or rules that is generally understood e.g. "proper names (languages, Frameworks..) don't get translated" and this isn't something we need to explicitly state and worry about?

[delucis]

@sarah11918 Sounds like a good idea to me. I would imagine this would be a fairly short list concentrated on Astro-specific terms. That would help ensure that for support cases there’s a shared vocabulary. For other terms, I’d leave it to the translators’ discretion as languages can differ significantly in how normal it is to integrate foreign terms.

[hippotastic]

Great summary, @delucis! Thank you for writing this down. It's very good to have all related todos here in a central location.

Regarding the CI job, the idea is to have that job run automatically after any push to the main branch using GitHub Actions. This would ensure that the language-specific lists of pending translations are always up to date.

If we're able to agree on a common format for commit messages, it would be no problem to ignore minor changes like typo fixes when determining if a translated document is out of date or not. For example, the CI job could ignore any commits that start with fix or typo.

I'm sure it's also possible to automatically determine how much of a document has changed since it was translated, thanks to the commit history and git diffs. The only requirement I see to make this "level of outdatedness" calculation work reliably is to have fairly stable document URLs, which will hopefully be the case after the 1.0 release.

[sarah11918]

Alight @delucis & @hippotastic , with the main site merged, you're up on automation! (I promise I will try not to make this difficult for you.) 😅

Can you START with a system for:

• /en/ pages to be marked as READY TO TRANSLATE (I don't know whether a yes/no or a green/yellow/red is more helpful. So only green are DONE for translating, as much as anything is ever done. But if you're "working ahead" and need some direction, yellow pages are a much safer bet than red pages.)

• This information to be displayed somewhere that translators can go to check
  These two things are the bare minimum we need to get started.

• BONUS: when an /en/ page is marked yes/green, there's some kind of notification that goes... somewhere? to someone?

Note that these are "one off" notifications, because it's just changing the state of the page to, OK, you can start translating now. It's not going to go back and forth after that. When this page later changes, it will go into the OTHER automated system indicating that this page has been updated. (And, that will probably happen multiple times over the page's lifespan.)

[sarah11918]

(Follow up, I know you might tell me that, "This is actually the same system!" and it might end up that way... but this first has a smaller scope and simpler problem to solve. So sure, if you want to treat this as an experimental playground for the MAIN system that we'll need, going forward, that's fine. BUT, I would rather think of it as 2 different systems if it means that we get a nice, simple solution that works well for translators who will be off working in big batch translations rather than responding quickly to change, at first.)

[delucis]

Closing in favour of the separate issues!