Implement multilanguage support

[nicopace]

Migrate to nikola: https://getnikola.com
This is to support multilanguage properly.

[gmarcos87]

I propose @rtroyan to work in this issue and collaborate with translation to portuguese language. I think that is not necessary migrate to another static site generator. Exist plugins for Jekyll like https://github.com/Anthony-Gaudino/jekyll-multiple-languages-plugin or https://github.com/contentascode/fixture-jekyll-transifex

[nicopace]

(updated the title to reflect about the task)

[ilario]

This was implemented by @digitigrafo in #28, I suggest to close this issue and #23.

[patogit]

Reposting this here from #28 at @ilario 's suggestion (keep things organized 👍 )

(In case you don't know me: I'm coordinating or advancing the translation of the software and documentation for the LibreRouter project, with the goal of making it easy for people to translate the project into many languages, and for people the use and contribute to the project in many languages.)

I see that the content of the Documentation page at LibreMesh.org is available in Italian, but the sidebar and menus remain in English (and the dark grey header is larger on that page, some CSS error I imagine). Based on the experience setting up the Italian page, do we want to use the same method to translate the site into other languages? Is Jekyll still meeting our needs for LibreMesh.org?

The file layout of the website content is more like documentation than like a software interface. For translating the LibreRouter documentation, we haven't chosen a platform or workflow yet. Options include:

• TranslateWiki (what we plan to use for software interfaces),
• Zanata (another online platform, it supports more file formats than TranslateWiki. A test I did with some documentation),
• desktop software like OmegaT, or
• just doing it by hand.
• (@digitigrafo mentioned Weblate. Weblate didn't seem to work very well last time I tried it, but maybe it's an option.)

To use TranslateWiki, we might have to make a parser-generator for asciidoc (based on SimpleFFS), and doing that might be useful for other LibreRouter documentation and other projects.

[ilario]

Is the Jekyll + TranslateWiki way practically doable?

[patogit]

Doable, yes. Practical, I don't know. Immediate and easy and online would be Zanata, or offline with OmegaT or OmegaT+.

Here's why using TranslateWiki to translate documents requires extra effort:

TranslateWiki is designed to translate translation files for software interfaces. In those files, there are pairs of keys and translations (the key might be called the Message ID). For example, in the Spanish translation file for LiMeApp, there's a key align_11050992 followed by the Spanish version of that message, Alineación. Or in the Spanish translation file for LuCI, there's the key Channel 5 GHz with the translation Canal 5GHz. In the file, it looks like this:
"align_11050992" : "Alineación",
and
msgid "Channel 5 GHz"
msgstr "Canal 5GHz"

The process for translation goes something like this:

1. @gmarcos87 puts the LiMeApp code in a GitHub repo, and @Nikerabbit (of TranslateWiki) configures TranslateWiki to ingest the translation files,
2. a parser-generator written in PHP reads that translation file,
3. TranslateWiki software makes the messages available in the translators' interface (screenshot of translators interface),
4. someone translates the message,
5. the PHP parser-generator creates a new version of the translation file,
6. @Nikerabbit commits that file to the LiMeApp repo.

The tricky bit with the LibreMesh.org web pages is that they don't follow the key/message pair format. There are ways around this: Zanata has a parser-generator that creates a translation file from a text file by making each paragraph a message and generating a random key for each message. The TranslateSVG extension for MediaWiki works similarly.

So in order to use TranslateWiki for asciidoc files (like the LibreMesh.org pages), we need an asciidoc parser-generator. The same for MarkDown files (such as the Readme file in each repo), or ODT files (the current format for LibreRouter documentation). The current parser-generators are at https://github.com/wikimedia/mediawiki-extensions-Translate/tree/master/ffs -- in that list, SimpleFFS.php is a template for making a new parser-generator, and FFS.php is special but I don't understand what it does.

[patogit]

For example, I put a page from lime-web in Zanata: screenshots of stable editor and of alpha editor, or go see it in action at the Zanata project I made for testing.

[patogit]

It turns out Zanata doesn't handle all the file formats that TranslateWiki handles. Here's the text from the "file upload" dialog on Zanata:

Maximum file size is 200MB
Maximum number of files is 100
Accepted:

• XML_DOCUMENT_TYPE_DEFINITION[dtd],
• PLAIN_TEXT[txt]
• IDML[idml]
• HTML[htm,html]
• OPEN_DOCUMENT_TEXT[odt]
• OPEN_DOCUMENT_PRESENTATION[odp]
• OPEN_DOCUMENT_GRAPHICS[odg]
• OPEN_DOCUMENT_SPREADSHEET[ods]
• SUBTITLE[sub,srt,sbt,vtt]
• GETTEXT[pot] # I tried uploading a .po file from LuCI, and it wouldn't accept it... only .pot
• PROPERTIES[properties]
• PROPERTIES_UTF8[properties]
• XLIFF[xml]

[patogit]

@nicopace mentioned Nikola instead of Jekyll. This wouldn't necessarily make the pages easier to translate with TranslateWiki, but it might (because it would let us use file formats besides asciidoc, some of which might be easier to make compatible with TranslateWiki), and it might make it easier to manage the multilingual site.

[patogit]

Instead of writing a parser-generator in PHP, we could write a script in whatever language we prefer to convert an asciidoc file into a JSON file, creating a random ID for each paragraph. Maybe that would be a lot easier. Then on the reverse end another script would change the JSON back into asciidoc. Or maybe someone has already written asciidoc-JSON conversion scripts. Maybe this is within my coding ability... it seems pretty simple, and I'm willing to try, but I don't know what programming language would be the best for it to be automated... bash?

Or maybe that's a waste of time if we can just write the parser-generator in PHP.

If we don't want to write any parsers, let's just use Zanata.

[nicopace]

Would recommend to go with the later :) we already have a very active community on the website for translation, right?

[patogit]

As far as I know there's no automated connection between Zanata and GitHub, so really people can translate however they want to and then commit to the repo here on GitHub. If they want to use Zanata (I like the interface, I think it's useful), they can go to https://translate.zanata.org/iteration/view/lime-web/lime-web-1/documents , make an account, and go from there. The automated connection between GitHub and TranslateWiki is what makes TranslateWiki a more accessible tool for translators who don't want to mess with GitHub.

[patogit]

Maybe some automation would be possible using the Zanata CLI client. I haven't read enough of the documentation yet to understand: http://docs.zanata.org/en/release/client/commands/push/

[nicopace]

The main problem right now is how to track the versioning of the translation efforts, cause it is in a document based approach more that in a string based approach.

[nemobis]

In case it helps:
https://docs.google.com/spreadsheets/d/1t6Zo9OmlRCH3N9dOumnWn0w8QrgI9Is2zQXSJB7K-QI/edit

[patogit]

@nemobis , that's a nice spreadsheet that gathers a lot of info. One of our criteria is that we don't want to host anything, and if possible we'd rather not pay for hosting because then we need to keep finding funding for as long as we want to keep the translations up to date.

Mozilla Pontoon looks really useful and I hadn't heard of it before your spreadsheet, but we would have to host it ourselves.

So for now, TranslateWiki is our choice, with Zanata as backup.

[patogit]

Henrique has set up a space on Zanata to translate the website content files, and I just found out that we can translate content pages on TranslateWiki too, so we'll probably go with that in order to do all the translation on one platform.

This means that if we want to change the way we implement the multilingual function on the website, maybe it would make sense to change before we get a lot more multilingual content.

But given priorities and the limited attention available, I'm guessing we'll leave the website function as it is for now.

I'll post again when the space on TranslateWiki is ready.

[ilario]

Currently, the website supports basic multilanguage but it was not used much.
I would close this issue and consider it again if we see that interest is raising again (and possibly moving to a completely different website type like a wiki).