Skip to content

i18n (non-english) builds and deployment on readthedocs #139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rufuspollock opened this issue Nov 13, 2011 · 13 comments
Closed

i18n (non-english) builds and deployment on readthedocs #139

rufuspollock opened this issue Nov 13, 2011 · 13 comments

Comments

@rufuspollock
Copy link

readthedocs appears to have some intention to support different language builds of material as it puts 'en' into all urls :-)

However, I can't see from the readthedocs docs or the dashboard how to enable i18n builds or how to get deployed.

As a concrete use case we are building Open Data Manual in both english and other languages off of sphinx rest stored in github (see https://github.com/okfn/opendatamanual -- whose README details how we prep and build the i18n material):

We'd love to use readthedocs for this but need some way to have, for example, the 'it' version built and deployed. (We considered a hack approach by using the subproject feature but we imagined this would lead to urls like {subproject}/en/latest).

NB: i18n until recently was only in head for sphinx but as of v1.1 is now in main release.

Thanks again for creating such a great tool.

@ericholscher
Copy link
Member

Neat. I have been thinking about how to provide internationalization support. It isn't baked in anywhere on the backend, I just future-proofed the URLs because I didn't want to have to change them later :)

I have some thoughts on how to do this, mainly it just involves modeling the relation from a master project to another project, and saying that it is the "it" translation of that. If you have any other thoughts on how to support this at a lower level, or with sphinx, I'd be curious.

@rufuspollock
Copy link
Author

I'd suggest supporting this with sphinx since as mentioned sphinx now supports i18n "out of the box" (see http://sphinx.pocoo.org/intl.html).

Of course a little bit of work is needed here but the good news is it's trivial and we worked most of this out for the Open Data Manual. Specifically from RTD point of view we need to:

  • Configure what languages should be built (probably just a setting on the core project)
  • Build each language

The second item is pretty simple and is as follows (The following is just extracted from "Building a translation of the manual" in https://github.com/okfn/opendatamanual/blob/master/README.rst and associated make file):

Let's assume we agreed translations always live in (this would either be hard convention of configurable in project settings):

{docs-dir}/_translation/{lang}/LC_MESSAGES/

Then all that would happen to build lang X would be:

  1. Build all mo files (or we could omit this if you just required people to build and commit the mo file themselves)

    msgfmt *.po -o *.mo
    
  2. Change your html build command to include -D option:

     -D language=$(lang)
    

    And to include ${lang} in output directory path (you are probably already doing this for your default en build)

Offer: if you point me in the right direction in the repo I could take a stab at coding some of this up ...

@ericholscher
Copy link
Member

Thanks for the info. This sounds pretty easy to accomplish. There are a couple problems that I'll probably need to deal with on my side, but it sounds like this shouldn't be too hard.

The main problem is that the files are stored on disk without the translation in the directory hierarchy. This should be pretty easy to change, but will take a bit of time and scripting on our side. We namespaced the URLs, but not the filesystem, at least that's easier to change after the fact :)

The sphinx builder is here: https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/doc_builder/backends/sphinx.py - This and the base builder (in the same dir) are where most of that logic happens. It should just be figuring out how to get the language that is being built into the builder, and then making sure it ends up in the right directory on the file servers.

I'd be willing to take a stab at helping get this done over the weekend, as it should be pretty simple, just a bit annoying.

Cheers,
Eric

@rufuspollock
Copy link
Author

@ericholscher great to hear this is possible. As you say once you introduce i18n you do get output namespaced with langs e.g. build/html/en/ build/html/it etc. If there is anything I can do let me know.

@papaeye
Copy link

papaeye commented Mar 24, 2012

Any update on this?
I tried this at https://github.com/papaeye/readthedocs.org/tree/i18n-html-build-spike.

I confirmed with my gntplib package in a local environment (gunicorn).
The following URLs work well.

http://localhost:8000/docs/gntplib/en/latest/
http://localhost:8000/docs/gntplib/ja/latest/

@rufuspollock
Copy link
Author

@papaeye can you submit a pull request with the patch? I imagine that would make it much easier for @ericholscher to review and incorporate.

@papaeye
Copy link

papaeye commented Mar 24, 2012

@rgrp Thank you for your advice. Is this OK? #193

@zupo
Copy link

zupo commented Aug 25, 2012

Anything new on this one? We're now starting to localize an API for Plone (github.com/plone/plone.api) and would love to have all languages on RTD. Is there anything I can help out with?

@ericholscher
Copy link
Member

Re: that PR -- I have merged it, but it doesn't address building python3 applications without a virtualenv. I am looking into how to support this, and will enable it on the site when it's done.

@ericholscher
Copy link
Member

Looking into it, when I install sphinx using python3 it doesn't install it as a seperate binary than with python2, so AFAICT I can only have 1 sphinx install (at least accessible by standard PATH commands).

@ericholscher
Copy link
Member

Believe this should be functioning now. We currently have people using multiple languages on RTD. Please file a new issue if there are outstanding issues.

@myroslav
Copy link

On standalone Sphinx installations I'm building .mo files with:

sphinx-intl build -c docs/source/conf.py

in #193 there was msgfmt used. But I cannot find that in RTD nevertheless it is reported to be merged (see #139 (comment) above). There is recommendation avoid putting .mo files in git repo due to rather strait-forward way to generate them and ugly hacks to resolve merge conflicts if one still does.

What is the status of ability to have non-English RTD translation project and still have only .po in git repo? Should I put that as request as separate issue?

@techtonik
Copy link
Contributor

@myroslav have you created the new issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants