Generate English Backdrop language files regularly

[olafgrabienski]

To set up the translation server for Backdrop language files, we need to upload an English language source file to the server.

While there is probably a workaround for testing purposes, this issue is about the question how to generate such files regularly and reliably for each Backdrop release on a production translation server.

[olafgrabienski]

Gábor Hojtsy, http://hojtsy.hu/blog/2010-may-05/using-drupal-collaborative-software-translation-tool:

How you generate .po files from your software depends totally on your API.

Is there such an API for Backdrop, or could someone with coding skills look for a way to generate the English .po file of Backdrop releases with the help of a script?

[olafgrabienski]

Just discovered: for Drupal modules and themes, there is the Translation template extractor module which extracts translatable strings via the interface or command line. (It's even part of our Drupal based translation server.)

We need something similar which extracts translatable strings from Backdrop core.

[olafgrabienski]

Workaround to generate an English Source file for translations:

• On a Backdrop site, go to admin/config/regional/translate/export.
• Under Export template, export the `.pot* (language template) file of the site.

NB: I noticed that the number of language strings is quite low on a new site. As if the system needs some usage to build complete information about the language strings.

[quicksketch]

We can possibly build the exporting of strings as part of the BackdropCMS.org packager. Right now we have the packager running to update the .info files on every release. We could additionally add a step to automate the string extractor. Though once we have the strings, I'm not sure yet how we would "push" that information to the translation server. Perhaps we could simply publish a feed of some kind on BackdropCMS.org and then have the translation server consume it.

[olafgrabienski]

@quicksketch From the Connectors page of our Drupal translation server:

Localization server connectors handle data from different locations, parsing source files to make them available for translation.

There are four "Connectors" in a list of which at the moment only the last one is enabled:

• Drupal packages from the file system
• Drupal packages from file uploads
• Drupal.org packages from the drupal.org REST API
• Gettext files from file uploads

Maybe this information helps at some point automating the process of getting the English language files.

[olafgrabienski]

Just wanted to add the last two Backdrop releases to the translation server. To get the English language template (translation source) file of 1.6.3 I tried the workaround mentioned in #6 (comment) on one of my test sites, but the number of language strings is far too low (for the reason see "NB" in the linked comment). It's only 208 KB but should be about 530 KB or more.

So, the workaround seems to be not reliable, and we shouldn't use it to build the Backdrop language source files on the translation server, but without the source files we're blocked in building the server.

As long-term alternative see quicksketch's comment above.

But we need also a short-term solution. @quicksketch or @serundeputy Do you think it's possible to port Translation template extractor to Backdrop? Or to build a similar tool to the "Command line extraction" mentioned on the Translation template extractor project page?

[serundeputy]

@olafgrabienski give this a go when you get a chance; backdrop-contrib/potx#1 ;)

[olafgrabienski]

Short report here: backdrop-contrib/potx#1 (comment)

Help by @quicksketch needed!

[quicksketch]

backdrop-contrib/potx#1 is fixed. potx module is now all set to go.

[olafgrabienski]

I'm reopening this issue and assigning it to @serundeputy.

While the potx module is working, it's a short-term solution (which is great) but the original intent of this issue is about the generation of language files "for each Backdrop release on a production translation server" (from the issue description).

For a long-term solution approach see @quicksketch's thoughts in #6 (comment).

[olafgrabienski]

From #6 (comment):

Though once we have the strings, I'm not sure yet how we would "push" that information to the translation server. Perhaps we could simply publish a feed of some kind on BackdropCMS.org and then have the translation server consume it.

At the moment, the localization server is configured to manage translatable strings with Gettext standard input/output, i.e. manually uploading .pot files. There are however two submodules which might help in automatization:

• Localization server for Drupal (REST API) -- Fetch Drupal projects from drupal.org.
• Localization server for Drupal -- Extracts translatable strings from Drupal packages.

[serundeputy]

I've added this repo: https://github.com/serundeputy/autogen_translation_releases

tl;dr

A collection of scripts that are intended to run on cron on the localization server. they will

• download backdrop latest release
• download and run potx
• create a release on the localization server corresponding the latest download
  • TODO: upload and parse the po file
• there is a cleanUp.sh script to clean up the tmp things

details in the README.md

Perhaps we should move the repo to backdrop-contrib It is probably not useful for anyone but us, but in the name transparency it is probably the right place for it or maybe backdrop-ops.

[olafgrabienski]

Yesterday I suggested in the forum to discuss the topic in the weekly Dev meeting:

The site provides translations of Backdrop core but so far not for contributed projects. While it is possible to add contrib projects to the site, we don't have a good way to add English language source files to their releases. I do that manually for Backdrop core minor releases, but won't be able to do it for more projects and releases.

Questions:

• How can we make progress regarding the automatic generation and distribution of language files, ideally not only for core but also for contrib projects?
• What is a good approach?
• Who wants to and is able to help?
• Should we spend some money for this infrastructure task?

Below a link to the video of yesterday's meeting, starting where the topic was discussed. Thanks for the encouraging feedback!

https://www.youtube.com/watch?v=EysmdI3XIlQ&t=190s