Infrastructure for reading TTW in different languages

[da5nsy]

Summary

In the Collab Cafe on Aug 2nd 2023 there was a conversation about how there is currently no way to read TTW in different languages (spinning off on the conversation about #3213), despite extensive effort to translate TTW. Writing this issue to add notes from that discussion, and hopefully to push forwards some development towards the goal of changing that!

Goal(s):

• When someone views TTW website, there is a button to switch between languages.
• (stretch goal) A dynamic 404 page that says something like "this content is not available in your language, but it is available in X, Y, and Z languages, in case that language is accessible to you (and you can help with translation via LINK!)".

What is the current translation infrastructure? This (and the surrounding pages) is currently the best reference for the current state of affairs (I think?). Adding crumbs to #2961 where this info was added.

What needs to be done?

Explore the options

@sgibson91 has strong feelings that this should be done at the Sphinx level. @sgibson91 - add more info here please?

Starting links:

• https://jupyterlab.readthedocs.io/en/stable/developer/internationalization.html
• https://www.sphinx-doc.org/en/master/usage/advanced/intl.html
• https://docs.readthedocs.io/en/stable/guides/manage-translations-sphinx.html
• Multilingual books executablebooks/jupyter-book#223

Notes: (@AndreaSanchezTapia - can you check this please?)

• The translation team moved away from "Transifex" to "Crowdin", because Transifex does not support RTL languages.
• Currently, there are different branches for different languages, and they use .md files rather than the .po files that Transifex uses

@aleesteele: How can we decenter English as the main language? How do we enable/encourage people to contribute in the language(s) that are accessible to them? (@aleesteele - please update this if you can clarify/expand!)

Who can help?

We'll need an active collaboration/bridge between the Infrastructure and Translation teams to make this happen!

• Who are the key people? Who within the translation team knows most about the infrastructure currently used?

---

Updates

[aleesteele]

Hi @da5nsy - a full week later, I'm adding in some notes and links from our previous discussion at the Infrastructure WG meeting later today. Thanks so much for making this issue, and hope I can field more support before Book Dash as well, as there are a few moving parts before this happens. 😄

I want to flag this for @the-turing-way/translation - @BatoolMM, @melibleq, and @AndreaSanchezTapia to ask if it'd be possible to give access to @da5nsy and other members of the infrastructure team who might be working through this at Book Dash (@JimMadge? @sgibson91? @bsipocz?). This might make it easier to scope out what might be needed on their end. I'm particularly thinking of:

• Crowdin: https://turingway.crowdin.com/turing-way
• TWTranslation: https://github.com/TWTranslation

Adding some more notes today, publishing so that I don't lose the draft!

[JimMadge]

How can we decenter English as the main language? How do we enable/encourage people to contribute in the language(s) that are accessible to them?

This is a really interesting, and important question.

I think often in software you will have a "default" language (probably English?) and ask for translations to other languages to be added. The .po system works quite well here, where you can enumerate all of the text and provide suitable translations.

That isn't really the situation here though. I imagine translating prose is much more involved than the kinds of simple prompts that would be common in software. Also, we don't have/want such a strong impression that "contributions are in English, other languages are translations".

@aleesteele How does this work on Wikipedia? Are pages in different languages translations of each other or is each language of the site more like an independent project?

[JimMadge]

I didn't apply for the book dash (expecting I wouldn't have enough time), but I'm definitely interested in exploring what we can do collaboratively 👍

[BatoolMM]

@the-turing-way/infrastructure do you have any regular meetings that I and @RealRichi3 can join to next week to brief you about the progress we made with the multi-deployments?

[sgibson91]

@BatoolMM We have a Working Group meeting on Thursday this week. @aleesteele or someone else in the WG, can you provide details please? I am having a travel nightmare and need to swap accommodation.

[da5nsy]

Today in the collab cafe, @JimMadge, @sgibson91 and I explored whether we could do this with RTD.

Notes:

• The path forwards with readthedocs we'd imaged is a no-go because one version of repo - so we'd need to do a multi-repo approach (just in case, I've asked RTD whether they'd consider enabling repo-duplication).
  • Need to co-ordinate with localisation team
  • Not technical issue, just organizational
    • Though we might run into limit of how many RTD projects we're allowed (for free)
  • To avoid duplicating images we should/could spin those out into separate repo
    • Could we link direct to zenodo?
  • If not readthedocs, self-hosting:
    • We'd need css to create language dropdown
    • Where are we hosting (£?)
    • Then it would be a matrix/CI job
• Problems we've faced with RTD
  • Would need multiple config files - that requires specifying non-standard config paths, which is listed as "experimental"
  • Once you've imported a repo once you can't import it again (to create another language from within the same repo)

[jcolomb]

some thoughts as it is late and I feel creative:

• can one fork the repo as being a mirror, and upload that mirrored repo into RTD (readthedoc)

• for pages created in another language, make a specific PR workflow that make sure to add an (empty) page in the english version. Maybe even better, get a specific branch that will have only empty pages so the "default" version is actually the empty-content one.

I think this would mean that if the browser default language is not one of the language of the landing page, that page will also show the available language for that page (?)

[bsipocz]

The path forwards with readthedocs we'd imaged is a no-go because one version of repo

OK, let me then try the GHA way to go fwd. At least in that way we can have full control over the build, and I do have some experience in pulling a couple of repos together when building the docs/website.
However, I'm travelling in the coming weeks, so won't have time to even do simple tries before June.

[JimMadge]

Once you've imported a repo once you can't import it again (to create another language from within the same repo)

Thanks for @da5nsy for pointing out that you can "manually import" a repo to bypass this problem. I didn't see the big button when we were co-working 🤦.