Internationalization for Wagtail content

[shezi]

As noted in #23 and other places, it would be great if Wagtail could support content in and for different locales.
The main effort about this is probably specification, so I'd like to get the discussion going here about the direction i18n for content should move.

One important question about the issue is whether i18n should be enabled all the time or optional. Depending on this, the changes can either go into the models directly or into model subclasses.

[umazalakain]

I would suggest using https://github.com/deschler/django-modeltranslation. It uses a registration approach similar to django's admin for content i18n and it's currently the most developed option.

[shezi]

That looks nice indeed. We'd have to look at how that integrates into the Wagtail Admin, though.

[tomdyson]

https://github.com/edoburu/django-parler/ looks interesting, too.

[helderco]

This is actually a deal breaker. I so wanted to build my next project with wagtail, which is very simple, though it needs to be bilingual. Any update on this? How can I help?

[tomdyson]

Hi @helderco. As noted in #172, Wagtail does support bilingual sites - we've nearly finished building one with it - but it doesn't do anything clever with translator workflow, for example, so you currently have to link different language versions of the same pages manually. If you need more than that, the best way you could help now would be by adding some specifications to this issue, detailing how you think it should work.

Thanks!

[helderco]

@tomdyson Ok, how do I do that? You create different pages, and then "link" them?

I don't need anything fancy, but I like how Drupal works in this regard. You create a page targeting a language and then you have a button or link to add a translation. This fills the create new page form with content from the source page, then you specify your new language (it's just one of the fields) and proceed to translate your texts in place. In the end they become associated with each other, so you can easily have a language switcher with links to each translated version of the page.

Furthermore, there are certain fields that are common to all languages and thus don't need translation (e.g. a photo, otherwise you'd need to re-upload the same photo to all languages).

The process of creating a new page and associating with a source page is one way to do it, but on it's next version (D8), the community found it's easier and more flexible to just create one page and translate the fields instead. The workflow is the same to the editor, but internally only one "page object" gets created with fields dependent on language or not. This, opposed to have a the page object be dependent on language, with fields that are not i18n aware.

I manage a site with 6 languages so that's where I'm basing off. For this next project of mine, it's just 2 languages, I'd just care that the workflow is easy for the editor, at least better then Wordpress.

Are there any docs or steps I can follow to try it your way?

Thanks.

[sixpearls]

What about creating a TranslatableField model, perhaps with a reference to a language model? It could be abstract, and the concrete models could add the appropriate CharField, TextField, or RichTextField. For any field that needs translation, create a concrete model with a foreign key reference to the page model that gets the field. On the page model, use an inline panel to modify each translation. The page model would get the non-translatable field definitions.

This flow would work if you prefer translating fields instead of pages. I like the idea of having different (perhaps child) pages, as well, but not sure how that would work.

I think both would need a custom routing system to pick up the language.

[GabLeRoux]

Deal breaker over here too.
I would have loved to use wagtail for my next project, but after trying a bit to use wagtail with other apps for translations, I found out it will be a lot easier to go with django-cms this time. I like how django-cms manages multiple languages but I hope wagtail gets a good internationalization system without creating same page more than once.

I came from wordpress and I was using wpml, overkill but it was getting the job done by giving access to translators wihtout having to give access to entire admin.

[alej0varas]

Hi. I'm working on this here. I'm using django-i18n-model to create translation models without touching wagtail code. They are editable using "/django-admin/". HomePage's title and body can be translated, you need to set the language in translate tag inside "home_page.html" template to see it working.

What is missing is been able to edit translations through "/admin/". I've tried adding a panel but that haven't worked.

Update

I've added a language chooser in the base template to ease testing.

How to try it. Go to "/django-admin/demo/homepage/" and edit "Home page", add a translation for Spanish. Go to "/", change language to Spanish and you'll see how the body changes to the Spanish version.

[edrex]

@alej0varas did you want to open a pull request to get some eyes on your branch?

[alej0varas]

@edrex I've been busy on other aspects of the project and I've left this on hold until I get enough time to debug what's happening with panels that don't show up in the admin that as far as I investigated is related to how django-modelcluster lookup related fields. I didn't want to pull request something I consider broken.

[lemieux]

Any update on this issue?

[rmartins90]

https://github.com/infoportugal/wagtail-modeltranslation

2015-05-19 17:33 GMT+01:00 Marc-Antoine Lemieux notifications@github.com:

Any update on this issue?

Reply to this email directly or view it on GitHub
torchbox#96 (comment).

[rmartins90]

Still in development. Maybe you can contribute.

Thanks in advance

2015-05-19 18:02 GMT+01:00 Rui Martins rmartins16@gmail.com:

https://github.com/infoportugal/wagtail-modeltranslation

2015-05-19 17:33 GMT+01:00 Marc-Antoine Lemieux notifications@github.com
:

Any update on this issue?

Reply to this email directly or view it on GitHub
torchbox#96 (comment).

[SalahAdDin]

Have not you thought about doing it like they do in django cms? To create "copies" of the model fields for each language supported, and that the user fills these fields with the appropriate translations? I think it would be easier that way.

[SalahAdDin]

wagtail-modeltranslation work like django-cms, i think that is a very good idea become a official wagtail feautre, and put each language in a content tab.

[merqurio]

👍 @SalahAdDin That would be awesome

[nek4life]

👍 Looking into using wagtail for a project and this is probably going to end up being a deal breaker. I looked at the current suggested approach, but it seems like that would expect all pages to be translated and duplicated. There are also fields that do not need translating that would be duplicated.

[tomdyson]

@nek4life we've built multi-language Wagtail sites that don't expect all pages to be translated, e.g. Global Witness:

https://www.globalwitness.org/ (see 'Languages' at the top)

and some that do, e.g.

http://www.coebank.org/

and one where every field is translated:

http://refugee-photo-project.unhcr.org/en/

Better i18n / multi-language support is a priority for upcoming Wagtail versions, but it's certainly possible now.

[tomdyson]

P.S. @nek4life which model do you need? We want to make sure we can accommodate most of them :)

[rmartins90]

We, at InfoPortugal, developed an isolated app that integrates django-modeltranslation with Wagtail CMS. It's pretty stable. We are using it on several projects and soon one of them will be available on production environment. It's been tested on pre-production environments.

https://github.com/infoportugal/wagtail-modeltranslation

[kaedroho]

The docs for multilingual sites are a bit old (#612) and there's now better options available.

If I get time, I'll have a go at rewriting them over the weekend.

[tomdyson]

@rmartins90 this is great to read! Is the production environment running on Wagtail 1.0?

[rmartins90]

The pre-production environment is running on Wagtail 1.0 and Django 1.7.8. And is being used by clients as well. Until now it's working just fine.

[rmartins90]

We intend to make some serious modifications on v1.0 as:

• drop django-modeltranslation dependency;
• use translated tabs instead of repeating fields along the tab. This should not contain non translatable fields;
• better documentation;

All help is welcome.

[tomdyson]

Thanks, @rmartins90, that's a useful roadmap. We (Torchbox) can certainly help. Shall we have a Hangout or similar to discuss, early next week? Does anyone else want to be involved?

[J0hn5mith]

Hi

My company is also figuring out how we can use wagtail with several languages. I already approached you but we could not find a date.
I could help with some ideas for the concept. But I don't have much time for actual coding.

[nek4life]

Hey @tomdyson, so I actually need to talk with the owner of this particular project to see what their exact needs are, but I do have some general thoughts on translation support.

Front end. I'd like the language in the path /en /fr-CA /fr . I would like the paths to remain the same to child pages even if they are not translated, i.e. the default language the content is generated is the fall back, but the url does not change only the fields that are translated. Half a page could potentially be translated for example.

Admin side: One page with translatable fields. No pages are duplicated just to translate them. All fields could be translated, but not necessarily think something like images. I would also like the translations to not be inline with the original text for content creators so it's not in the way of getting work done. A translation tab that shows the original content next to the selected translated language would be great way to handle this.

I've built a custom admin like this before using flask, but not using Django. It's the only site I've done i18n on so it may or may not be the best approach, but just some ideas of how I was thinking about the problem.

[Proper-Job]

@nek4life @GabLeRoux I'd just like to put in my 10 cents for what it's worth.
I've done a lot of multilingual work in Django (though not in Wagtail) and in my experience there is no one-size-fits-all translation paradigm. Different CMS have different approaches that serve certain use cases well and others less so.
A common way to translate pages in Magnolia CMS for instance is to duplicate each and every page for every language similar how you would do it now in Wagtail. Wagtail's approach is by no means an oddball approach.
I would encourage you to think about your translation needs and how you can adapt Wagtail to fit your needs.
I've written Django backends that deal with up to 8 languages. One way of effectively handling that many languages (that also scales well for a REST API for instance) is to have the page contain the non-translatable content and add translations as inlines.
I guess what I'm trying to say is that there is usually no drop-in solution that makes all your problems go away. In my experience you always have to accept some trade-offs with regards to multilingual content, be it usability or scalability or what have you.

[Proper-Job]

@rmartins90 @tomdyson Yes I'd like be in the loop. I'm going to start coding on a new wagtail project in August. I might be able to help out.

[nek4life]

@tomdyson @alp-phone Some more about the use case I'm considering using Wagtail for and the needs for translations. I'm looking into using Wagtail as the backend to supply the content for an E-Commerce platform. The E-Com platform has terrible content handling, so I would like to provide a better experience for content authors. For this particular project I would not be designing a front-end using Wagtail at all. I would be doing either one of three things. Exporting content as an xml feed. Pushing content to the other system using API's provided from the E-Com platform or potentially developing a REST API with Wagtail that we could pull content from depending on how well Wagtail can scale up. The content in the E-Com system ends up as one entity with translated fields so if we were to do separate pages we'd have to merge them back down somehow, so that is a big reason I would rather not use separate pages, although I'm sure we could find a work around if we had to. Just wanted to give some more insight into my particular use case.

[Proper-Job]

@nek4life How many languages do you support? Are you using StreamField? if not, you could duplicate model fields with a language postfix. That's not very elegant, but it gets the job done, i.e.: product_name_en, product_name_fr, product_name_it.

[nek4life]

@alp-phone We're currently supporting 9 languages. That is definitely one approach to consider, perhaps there's even a way to store those fields in a separate tab when editing. It's definitely not elegant as you say, but it also doesn't require any joins. I've also set it up before with two tables one with the standard fields with a locale_id field and also another table with post_id and locale_id as the foreign keys along with the translation fields although that wasn't using Django and I wrote a custom admin for that.

[SalahAdDin]

👍

[jorge-marques]

👍

Although two tables would incur a join, I would rather have a separate table for translations, similar to how django-parler does it. I'd like to avoid migrations when adding a new language. Maybe it could be leveraged for wagtail?

[helderco]

How about having a db column for the language code, and another for a translation group? Then you could just add another record for the translation. You would group by the translation group id (tid), and get all records from the same table for each language.

[jorge-marques]

@helderco I may be misunderstanding but wouldn't you end up with duplicate image fields and other potentially non translatable stuff?

[helderco]

@jorge-marques yes, in order to do this, you would have only translated fields in this table and common fields in another, tied to the translation group.

[Proper-Job]

The django-parler approach (even though it seems quite clever) wouldn't work on Streamfield content, would it?

[Proper-Job]

Actually, I take that back. Django-parler appears to have an exquisite design. I think it could work with StreamField.

[keimlink]

@rmartins90 Your integration of django-modeltranslation with Wagtail CMS looks very promising. Thanks for publishing your work! Have you thought about adding some tests to your app to make the functionality better testable?

[rmartins90]

Hey. We've been very busy, sorry for the late reply.
We've just released a new version with support for StreamFields.
Until the next month we'll have no time to work on tests or anything to this app, however we'll be watching for any contributions or reports.

[kaedroho]

I've just pushed some updates to the i18n docs to this PR #1630.

[SalahAdDin]

wagtail-modeltranslation have many problems with migrations :(

[agelinas]

SalahAdDin,

Just curious, are the migrations problems encountered for new projects or one where migrations have been done before implementing wagtail-modeltranslation?

My limited experience is that I had no problems using wagtail-modeltranslation in new projects. I do like using wagtail-modeltranslation and hope that the community can wrinkle out any problems it may have.

[SalahAdDin]

The problem is when you migrate a project that haven't implemented wagtail-modeltranslation before.
I'm working in this project since wagtail 0.8.4 and i had tried implement modeltranslation and i have these problems.

[ratson]

I am looking for a way to integrate django-parlerTranslatedFields to Wagtail.

Does anyone have a success story about it?

[akhayyat]

It's been a while since there were any updates on this issue. Any progress? Is there a plan to tackle it?

It'd be great to have built-in support for model translations. For one thing, multilingual websites would be able to upgrade to new wagtail releases without having to wait for third-party translation apps to support them.

Although wagtail-modeltranslation did work with wagtail 1.3.1, it still does not work with the later wagtail releases, and we have to upgrade due to critical-for-us features and fixes in these newer releases.

[benebun]

@akhayyat: As far as I know the current plan is to use translatable page models which should arrive in Wagtail 1.5 in April. I think there is no documentation yet but there are two blog posts about the sprints where Torchbox & Co tackled this topic:
https://wagtail.io/blog/cape-town-retrospective/
http://www.springload.co.nz/blog/wagtail-cape-town-sprint/

In the first post Tom linked to the Takeflight's fork for the translation work:
https://github.com/takeflight/wagtail/tree/feature/page-translations

Not sure what that means for wagtail-modeltranslations though…

[timheap]

Sorry everyone, I have been very slack about getting this translation work finished. I'll try and get some more polish on this, as well as tests and documentation when I can, but life has been hectic ever since I did the initial work, and life will not become calm again any time soon. I'll make an effort to work on this when ever I get a chance to do some Wagtail work though.

[JoshBarr]

@timheap is there an RFC or spec that the wider Wagtail community could follow to help get it finished?

[benebun]

Yes that would be helpful. We're just starting the next Wagtail project here, so maybe we can help.

[rmartins90]

While Wagtail team don't release a version with support for multiple languages site we've just released an alpha version of wagtail-modeltranslation (0.5.0a) supporting Wagtail 1.4: https://github.com/infoportugal/wagtail-modeltranslation

[SalahAdDin]

@rmartins90 , i support you, but, i think that you documentation are incomplete.

[akhayyat]

Any updates on when to expect built-in translation support?

The blog posts:
https://wagtail.io/blog/cape-town-retrospective/
http://www.springload.co.nz/blog/wagtail-cape-town-sprint/
suggested that this should be available sometime in April. On the other hand, the branch where this development is supposedly taking place has not received any updates since February!
https://github.com/takeflight/wagtail/tree/feature/page-translations

[gasman]

@akhayyat No updates right now. Work on this feature is currently dependent on volunteered time, which (as per @timheap's message above) is in short supply at the moment.

[akhayyat]

@gasman How can we volunteer? Are there any specs?

[fdemmer]

adding to what @akhayyat wrote: i looked over that sprint branch and there were a few exceptions right from the start, so it is hard to see what is supposed to be finished and what is actually part of translation or just ui changes (like the new button/language selector widget).

[gasman]

There's now a draft spec for this feature at torchbox/wagtail-rfcs#9 - feedback welcome.

[robmoorman]

Please see if https://github.com/LUKKIEN/wagtailtrans fits your requirements, this should be a pretty good implemetation of the wagtail/wagtail-rfcs#9 purposal

[SalahAdDin]

@robmoorman is better than wagtail-modeltranslation?

[mikedingjan]

@SalahAdDin, some advantages of wagtailtrans over wagtail-modeltranslation..

• No database migrations, in model translation you’ll get a extra column in your DB for every language / translatable field. With wagtailtrans there is a relation with a Language object, it will create a new page for the language.
• Separate Page trees per language, allows you to have a translator (role) per language
• With settings, languages / tree structures can differ per language.

[SalahAdDin]

Sounds good, but, how manage this package the urls routers? Are there any way to manage all translations of a page in the same page? Changing between translations using some button or combo box?

[mikedingjan]

@SalahAdDin URL routing is done via your page structure, it creates trees per language, where you can handle your urls yourself.. The TranslatableSiteRootPage makes sure the user gets redirected to the right language..

Managing all translations on one page isn't possible, since they're actually different pages. We can't assume that content is the same for all languages. All pages do have a relation with their canonical page (language), so creating a link to other pages (languages) isn't that hard to do..

More information can be found in the docs: http://wagtailtrans.readthedocs.io/en/latest/

[SalahAdDin]

@mikedingjan I understand all, but you din't to me, i mean, in the interface, not in the background, in the interface, is possible change between languages in the page? I'm explain me: You have root page, instead to have in the HUD three different page, one for each language, have the single page, go into this page and with a botton change between languages. Had i explained me good?

[mikedingjan]

I understand what you mean.. Short answer: No

[SalahAdDin]

@mikedingjan Ok, Is easy make this way? By other i think this is a better alternative to moderltranslation, that now have many problems, and is not usable.

[rristow]

@ratson - some news about the wagtail-parler integration? For me also seems to be the best approach.

[SalahAdDin]

Yes, wagtail-modeltrandlation is super useless.

[kaedroho]

@SalahAdDin please keep it constructive.

[nifel87]

Hi everyone! I recently started to use wagtail for my projects and it's an amazing CMS. Are there any news for the internationalization support? Will it be included (natively) in the upcoming 2.0 version of Wagtail?

[SalahAdDin]

@nifel87 same question here.

[dmarcelino]

Hi everyone,

Wagtail is a great project which I've started using earlier this year. Ease of use, clean integration with Django and support for internationalisation (according to the docs I read back then) were among the reasons that convinced me.

Fast-forward some 10 months, with lots of content created using Wagtail it came the time to translate it and suddenly we started hitting some wagtail-modeltranslation bugs. I've raised and fixed some issues (as you can see in https://github.com/infoportugal/wagtail-modeltranslation/issues?utf8=%E2%9C%93&q=author%3Admarcelino+) and was invited as a project collaborator (thanks @DiogoMarques29!).

For the project I'm working on good i18n support is key and since we are committed to Wagtail I'm willing to spend more time until the end of the year looking to improve wagtail-modeltranslation as best as I can. After looking at the different alternatives for translating Django (django-modeltranslation, django-hvad and django-parler) django-modeltranslation is still my favourite approach (mostly because the translation fields live in the same tables as the original ones) hence I believe wagtail-modeltranslation is the best bet for translating Wagtail.

Having said all this, I think wagtail-modeltranslation needs some breaking changes and I've created a POC outlining them on infoportugal/wagtail-modeltranslation#150. I'd appreciate if @kaedroho and @gasman could look at the POC and give some feedback. I'll try to accommodate your opinions as best as I can since wagtail-modeltranslation should align with Wagtail.

If we combine our efforts I'm sure we can give Wagtail the i18n support it deserves.

Thanks and Merry Xmas!

[mikedingjan]

Hi @dmarcelino ,

Did you ever take a look al wagtailtrans, I personally like that approach better because, as you said, translations live in the same table as the original ones. But even better, adding a new language doesn't involve migrating you complete database. It also allows you to have different page structures per language (if you want)..

I'm not a big fan of monkeypatching Wagtail for adding translations, as you've done in your PR with adding the title_xx and slug_xx to the Page table. This because changes in for example wagtail can break the functionality.

Having said all that, I don't think there is a holy grail for good i18n support, since there are a lot of different ways of doing that and everybody has their own preference.

One thing I would like for wagtail is a "easier" way of integrating i18n support, perhaps we can have a discussion about that after the holidays? Packages as wagtail-personalisation and wagtail-experiments could really benefit from this as well.

Happy Holidays!

[SalahAdDin]

@mikedingjan Personally i did try to use these package and wagtail-modeltranslation, and both have a terrible performance with last versions. I tried with the other package, it was better but still it doesn't represent a ideal option.

[mikedingjan]

Performance shouldn’t be less with wagtailtrans, since it doesn’t change anything to wagtails way of selecting / showing pages, etc. If you really encounter heavy performance issues I would rather think you’ve configured something wrong..

Can you provide a detailed report of the issues you encounter? Also include your configuration. If I can reproduce it I (we) can have a look at it and maybe resolve the issues..

[dmarcelino]

Hi @mikedingjan, thanks for your feedback.

Yes, I looked at wagtailtrans and browsed through the docs, I did not install it though. Correct me if I'm wrong but my understanding is that wagtailtrans duplicates pages and all their fields without an option to translate specific fields only. For example, if there is an Event model with date and body it would be up to the translator to keep date in sync across languages if it were to change, right? In my project I have Wagtail models which only require partial translation.

The apps I've mentioned, including django-modeltranslation, allow the developer to choose which fields to translate and only those require any translator intervention. django-modeltranslation also has nice fallback settings where a missing language for a field can be replaced by another. For example, Brazilian Portuguese can fallback to European Portuguese and finally to English. This can be configured on a general basis or on field-by-field basis.

I confess I'm not a fan of monkey patching either, it was kind of last resort. If Wagtail supported swappable models things would become much simpler. Having said this, I believe patching Page to be a calculated risk, it's not likely that Wagtail authors will add slug_xx, title_xx, etc_xx fields to Page unless they are adding their own i18n app (which would fix all our issues 🙂).

I wholly agree Wagtail should be easier to add i18n to, perhaps #3282 can be a way to reach that goal.

@SalahAdDin, infoportugal/wagtail-modeltranslation#150 will enable the removal of most (perhaps all) calls to Page.specific and that should improve performance significantly.

Happy New Year!

[MechanisM]

It is possible to add support for django-nece and django-modeltrans as well? These apps for PostgreSQL only, but so awesome.

[mikedingjan]

@dmarcelino Partial translation isn't something that's supported by wagtailtrans indeed, however it shouldn't be to hard to implement yourself. For example with syncing "some" fields upon save, each page keeps a relation to it's translations, which makes that easy.

I've looked at swappable models as well, will solve a lot of use cases but will also bring more risk of breaking things.

@MechanisM Wagtail supports more than only PostgreSQL so adding support for something that only woks with PostgreSQL isn't something that's likely going to happen. If you want to use those packages with Wagtail, you can create your own "Wagtail" add-on.

[dmarcelino]

@mikedingjan, I would certainly do that if I had no alternative but I see 2 issues with that approach:

1. Not very DRY: fields like images, names, dates, addresses, numbers, etc will be repeated for each language;
2. I don't think the user of a translation app should have the burden to override Wagtail's save to achieve this. The translation app should handle this scenario seamlessly, requiring only the appropriate configuration (perhaps you can look at it as a feature suggestion for wagtailtrans).