Add list of supported languages; disable unsupported languages

[eloquence]

SecureDrop Client translation is currently unofficially supported via Weblate (it has not been announced/publicized yet); as a consequence, some volunteers have started translating the client into other languages. We do even have some 100% completions already:

https://weblate.securedrop.org/projects/securedrop/securedrop-client/

The way it works right now, however, you can run the app in a partially translated language. For example, as of this writing, Catalan is at 69%, and you'll see a partially translated UI if you launch it with LANG=ca.

Ideally, we should only enable languages in the app once they've reached full completion, and only disable them once they fall below a certain threshold (e.g., 80%). To do so, we need to be able to specify a list of languages that are supported, vs. ones that are currently unsupported. This should not prevent translation in Weblate, it should only prevent the language from being made available in the app.

If we're unable to initially support certain languages for technical reasons (e.g., RTL languages, see #1406), we can also use that capability to delay enabling those languages until technical foundations are in place.

[cfm]

make supported-languages currently writes a file l10n.txt that lists languages at 100% translation in the securedrop-client component in Weblate:

(.venv) user@sd-dev:~/securedrop-client$ git diff
diff --git a/Makefile b/Makefile
index e7a7a9c..f5ea434 100644
--- a/Makefile
+++ b/Makefile
@@ -236,6 +236,6 @@ supported-languages:
                --format json \
                --url ${WEBLATE_API} \
                list-translations \
-               | jq -r 'map(select(.component.slug == "${WEBLATE_COMPONENT}" and .translated_percent == 100)) | map("* \(.language.name|tostring) (``\(.translated_percent|tostring)``)") | join("\n")' \
+               | jq -r 'map(select(.component.slug == "${WEBLATE_COMPONENT}" and .translated_percent == 100)) | map("* \(.language.name|tostring) (``\(.language.code|tostring)``)") | join("\n")' \
                > ${SUPPORTED_LOCALES_LIST}
        @git add ${SUPPORTED_LOCALES_LIST}
(.venv) user@sd-dev:~/securedrop-client$ make supported-languages
(.venv) user@sd-dev:~/securedrop-client$ git diff --cached
diff --git a/l10n.txt b/l10n.txt
new file mode 100644
index 0000000..f7d72ee
--- /dev/null
+++ b/l10n.txt
@@ -0,0 +1,7 @@
+* English (``en``)
+* German (``de``)
+* Spanish (``es``)
+* Icelandic (``is``)
+* Turkish (``tr``)
+* Chinese (Simplified) (``zh_Hans``)
+* Russian (``ru``)

I added this target for parity with securedrop's i18n_tool.py --update-docs, from which docs/includes/l10n.txt is pulled into the documentation to list supported languages.

Whether or not we want to keep this human-readable l10n.txt (with the above patch!) here, we could use the JSON response from the Weblate API in either of two ways:

1. Save it to the repository as part of Weblate's incoming translation PR; then have the Client read it for a list of language codes for which it's permitted to load a gettext catalog if defined in $LANG. The repository and packages would still contain catalogs for all languages.
   • Con: Adds more machinery to the continuous-translation workflow.
2. Use it in the (Python, probably, rather than Debian) packaging logic to filter catalogs for 100% translation. The repository would still contain catalogs for all languages, but only supported languages would be packaged and loadable in a production installation.
   • Con: python setup.py sdist would have to make a call to the Weblate API.

The caveat to both of these approaches is that they ask what languages are 100% translated in Weblate, not in (1) the package from which the Client is being run or (2) the ref from which a package is being built. To do that, we'd have to:

3. Use Babel in setup.py to read the actual .po files present and filter them for 100% translation. The repository would still contain catalogs for all languages, but only supported languages would be packaged and loadable in a production installation.

I'm leaning towards (3), because it (a) adds no new dependencies or calls and (b) has the least risk of surprise to packagers or users. What do you think among these options, @eloquence?

[eloquence]

So, I'm not sure we'll immediately want to "unsupport" a language if it drops below 100% - there's probably a threshold where we give translators some time to catch up in case they miss a few strings between releases. Otherwise we run the risk that the software as a whole falls back to English just because a couple of strings didn't make it.

Similarly, 100% translation doesn't necessarily mean that a language actually works as well, as the RTL example illustrates.

I'm not sure how that influences the choice between different technical approaches, but it's one reason I might want to lean towards the decision of what languages to be enabled to be made by humans looking at statistics and considering other factors. So a simple list of language codes in setup.py or elsewhere (which possibly could be overridden via an env var if you want to preview an incomplete language) might be the way to go. Does that make sense?

[cfm]

@eloquence in #1438 (comment):

[...] I might want to lean towards the decision of what languages to be enabled to be made by humans looking at statistics and considering other factors. So a simple list of language codes in setup.py or elsewhere (which possibly could be overridden via an env var if you want to preview an incomplete language) might be the way to go. Does that make sense?

freedomofpress/securedrop#6156 has convinced me of this approach, which I've recorded there in freedomofpress/securedrop#6387. Specifically, for each project:

1. All languages present in Weblate would receive source-string updates.
2. Each language present in Weblate would appear in the repository in the form of a messages.{po,mo} catalog.
3. Only languages listed as supported in (e.g.) setup.py would have their catalogs packaged.
4. Only packaged catalogs would be loadable at runtime.