i18n (non-english) builds and deployment on readthedocs

[rufuspollock]

readthedocs appears to have some intention to support different language builds of material as it puts 'en' into all urls :-)

However, I can't see from the readthedocs docs or the dashboard how to enable i18n builds or how to get deployed.

As a concrete use case we are building Open Data Manual in both english and other languages off of sphinx rest stored in github (see https://github.com/okfn/opendatamanual -- whose README details how we prep and build the i18n material):

• http://opendatamanual.org/en/ - english (we don't currently add the version in the url but that would be no problem to change)
• http://opendatamanual.org/it/ - italian

We'd love to use readthedocs for this but need some way to have, for example, the 'it' version built and deployed. (We considered a hack approach by using the subproject feature but we imagined this would lead to urls like {subproject}/en/latest).

NB: i18n until recently was only in head for sphinx but as of v1.1 is now in main release.

Thanks again for creating such a great tool.

[ericholscher]

Neat. I have been thinking about how to provide internationalization support. It isn't baked in anywhere on the backend, I just future-proofed the URLs because I didn't want to have to change them later :)

I have some thoughts on how to do this, mainly it just involves modeling the relation from a master project to another project, and saying that it is the "it" translation of that. If you have any other thoughts on how to support this at a lower level, or with sphinx, I'd be curious.

[rufuspollock]

I'd suggest supporting this with sphinx since as mentioned sphinx now supports i18n "out of the box" (see http://sphinx.pocoo.org/intl.html).

Of course a little bit of work is needed here but the good news is it's trivial and we worked most of this out for the Open Data Manual. Specifically from RTD point of view we need to:

• Configure what languages should be built (probably just a setting on the core project)
• Build each language

The second item is pretty simple and is as follows (The following is just extracted from "Building a translation of the manual" in https://github.com/okfn/opendatamanual/blob/master/README.rst and associated make file):

Let's assume we agreed translations always live in (this would either be hard convention of configurable in project settings):

{docs-dir}/_translation/{lang}/LC_MESSAGES/

Then all that would happen to build lang X would be:

1. Build all mo files (or we could omit this if you just required people to build and commit the mo file themselves)

   msgfmt *.po -o *.mo
   

2. Change your html build command to include -D option:

    -D language=$(lang)
   

   And to include ${lang} in output directory path (you are probably already doing this for your default en build)

Offer: if you point me in the right direction in the repo I could take a stab at coding some of this up ...

[ericholscher]

Thanks for the info. This sounds pretty easy to accomplish. There are a couple problems that I'll probably need to deal with on my side, but it sounds like this shouldn't be too hard.

The main problem is that the files are stored on disk without the translation in the directory hierarchy. This should be pretty easy to change, but will take a bit of time and scripting on our side. We namespaced the URLs, but not the filesystem, at least that's easier to change after the fact :)

The sphinx builder is here: https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/doc_builder/backends/sphinx.py - This and the base builder (in the same dir) are where most of that logic happens. It should just be figuring out how to get the language that is being built into the builder, and then making sure it ends up in the right directory on the file servers.

I'd be willing to take a stab at helping get this done over the weekend, as it should be pretty simple, just a bit annoying.

Cheers,
Eric

[rufuspollock]

@ericholscher great to hear this is possible. As you say once you introduce i18n you do get output namespaced with langs e.g. build/html/en/ build/html/it etc. If there is anything I can do let me know.

[papaeye]

Any update on this?
I tried this at https://github.com/papaeye/readthedocs.org/tree/i18n-html-build-spike.

I confirmed with my gntplib package in a local environment (gunicorn).
The following URLs work well.

http://localhost:8000/docs/gntplib/en/latest/
http://localhost:8000/docs/gntplib/ja/latest/

[rufuspollock]

@papaeye can you submit a pull request with the patch? I imagine that would make it much easier for @ericholscher to review and incorporate.

[papaeye]

@rgrp Thank you for your advice. Is this OK? #193

[zupo]

Anything new on this one? We're now starting to localize an API for Plone (github.com/plone/plone.api) and would love to have all languages on RTD. Is there anything I can help out with?

[ericholscher]

Re: that PR -- I have merged it, but it doesn't address building python3 applications without a virtualenv. I am looking into how to support this, and will enable it on the site when it's done.

[ericholscher]

Looking into it, when I install sphinx using python3 it doesn't install it as a seperate binary than with python2, so AFAICT I can only have 1 sphinx install (at least accessible by standard PATH commands).

[ericholscher]

Believe this should be functioning now. We currently have people using multiple languages on RTD. Please file a new issue if there are outstanding issues.

[myroslav]

On standalone Sphinx installations I'm building .mo files with:

sphinx-intl build -c docs/source/conf.py

in #193 there was msgfmt used. But I cannot find that in RTD nevertheless it is reported to be merged (see #139 (comment) above). There is recommendation avoid putting .mo files in git repo due to rather strait-forward way to generate them and ugly hacks to resolve merge conflicts if one still does.

What is the status of ability to have non-English RTD translation project and still have only .po in git repo? Should I put that as request as separate issue?

[techtonik]

@myroslav have you created the new issue?