Use Weblate for translations

[agaida]

hi - we have our weblate instance up and running - so i would like to suggest using it. With weblate we should move back our translations back from lxqt-l10 to the repositories they belong to - there are several reasons for it:

• small components are easy handled with weblate
• the chance of conflicting commits raise with the amount of components in one repository - a pita to maintain
• we can move step by step, starting with the leaf packages that don't have a dependency to liblxqt

We tested the weblate instance with featherpad and it was fun to work with - the only thing missed right now in weblate are automatic translations and suggestions - will implement it with the upcoming new version of weblate.

@ALL - opinions?

[PCMan]

@agaida How does this compare to Pootle?
We tried Pootle in the past but it does not seem to work well with Qt *.ts files.
(Also we tried transifex, which is no longer free, long time ago.)

[agaida]

@PCMan basicly it is the same transition engine as in pootle with a new interface. As far as i can tell it works nice with *.ts. A very big plus it the git orientated workflow. We learned about weblate a few years ago, but it seems nobody was able/brave/crazy enough to set it up production ready. So i finally bite the bullet and took the old installation, took it to our new machine into a LXD container, update it and reconfigured it from the scratch.

Time is the key - meanwhle weblate is mature and fun to setup and to work with. In the end it took some days to make it work like i want it to. That means: Auto-import of new changes from the repos, semi-automatic sending pull requests to github - right now it works robust and it is easy to maintain.

I also maintain the pootle installation for LXDE - and to be honest, the implementation quality of weblate is on a much higher level - it might be our fault because the pootle guys has changed the preferred database to my/mariasql - we run postgres and basic database things don't worked with - it should, but doesn't - so i had to fix some constraints, keys etc manually. No of the problems with weblate. So i would give it a go.

[agaida]

i forgot - one could have a look into https://github.com/tsujan/FeatherPad - tsujan/FeatherPad@bbc9c5a as an example. What i like most is the PR thing (achived with hub - a go-based git wrapper esp. for github. So no hard commits from weblate into the project - i love it.

[agaida]

Disclaimer - i'm not much into translations, nearly no clue, not the python guy, nearly no clue, not the postgres guy, nearly no clue, i don't know linguist well - same for containers, ip-routing and python hosting - if a project based on these requirements runs stable and reliable some people have done a great job. 😎

[PCMan]

@agaida My vote is +1 for this one. Sounds awesome! The current lxqt-l10n repo is not easy to use for ordinary translators who don't know programming & git.
@luis-pereira What do you think? Since you are the master of *.ts and Qt linguist and does most of the LXQtTranslate stuff, your opinion is quite important here.

[yan12125]

Hi @agaida, I tried to login to https://weblate.lxqt.org/ with GitHub OAuth2 but it says:

已停用新註冊！

(New registrations are disabled!)

Is it still under construction?

[agaida]

Right now there are three projects - featherpad, screengrab and qps - ir acked i can enable the registration thing - and yes, i don't wanted new registrations in this service right now. Doesn't make sense. An open registration would make think people that the instance is up and ready for production - and that was not the case right now.

If we decide to go with weblate i will import some leaf packages first and enable registrations. If one want an account right now - drop me a mail via the contact form.

[agaida]

@luis-pereira - dunno if it is possible - but i played a bit with not so common git commands in preparation of putting back the translations with history to the repos they belong to. Would it be much effort if we place the translations in /translations/l10n and leave the possible desktop files in /translations - or is it to much work and /translation/$component would be better?

[yarons]

You can create a different branch for that so it won't clog your logs if that bothers you.

[agaida]

@yarons: but one has to maintain the translation branch - right now it is fine as it is, we should only place our translations again in a sane place :) - second thing about the translation process: we use hub for pushing the translations as pull requests - we could even squash things, but that would need a bit of change at the weblate side - forcefully pull the original sources if needed.

EDIT: But such things will come later - first of all we should have a working and sane base configuration.

[yarons]

@agaida So you're suggesting keeping it in the same repo?
It's also a possibility, maybe @nijel can advise you as well 😄.

[agaida]

i really want it in the same repo - hup helps a lot - so we (will) get pull requests from weblate - right now i see no reason to split the translation process out. My main concern right now is to make the process clean on our site - weblate is up and running (latest 2.xx, pip installation in a container only for weblate). So no problems right now.

One remark: We really want the 3.0 asap - but i'm not desperate enough for checking out git 😎

[yarons]

@agaida including the latest docker changes? 😀
https://blog.cihar.com/archives/2018/05/29/improved-docker-container-weblate/?utm_source=rss2

[agaida]

@yarons - we don't use docker 😄

[nijel]

Weblate 3.0 will be out on Friday, probably with no code changes compared to current git...

[agaida]

Cool, really great news - so i will upgrade when it is available via pip

[agaida]

@nijel and @yarons - thanks for the heads up - as i wrote earlier, weblate is pure fun (as far as translations can be fun) to work with. The only thing is our mentioned translation repo. Kind of works, but a horrible base for any webbased translation process and maintenance - so i guess we should change it first. I've learned the needed git magic two days ago - so that should be no problem at all.

[nijel]

Indeed it's usually better to keep translations with the code. In case you want more complex git workflow from Weblate side, you can use wlc to eg. reset the Weblate repository to upstream one...

[tsujan]

Indeed it's usually better to keep translations with the code.

I completely agree. Translators need to know about the context.

[agaida]

ludi incipiant

% pip search weblate
weblate-tools (0.1.0)  - weblate文本处理工具
Weblate (3.0)          - A web-based translation tool with tight version control integration
wlc (0.8)              - A command line utility for Weblate, translation tool with tight version control integration

@nijel - that was fast

[agaida]

@luis-pereira @tsujan @palinek - i have played a bit with merging back translations into the repos they belong to - the script works fine: https://github.com/4sid/translation-move/blob/master/move-translations

but the result looks a bit brain damaged - so i would like to change the structure to
https://github.com/4sid/translation-move/commit/3d9a20221fcb5bf03d13700b487b6de2085ac6ac?diff=split

[agaida]

Weblate upgraded to 3.0 - that was easy and pure fun. So i guess we should work on the things to translate and migrate asap.

[agaida]

@luis-pereira @tsujan @palinek @yan12125 - finished the translation move script - the nice thing about is: it just works™ - the not so nice thing is that we need to modify lxqt-build-tools and the CMakeLists.txt's a bit. No big deal, just removing the git pulls and targeting the new translation dirs.

One thing i really like to have - it is implemented in the script now: uniform translation locations. So instead of /$foo/translations/$component or /$foo/translations/$component/$subcomponent it should just be $foo/translations/l10n - will prepare the needed PRs

Can also be filed under "filigree work with a chain saw"

[ghost]

Hello !
What a nice move ! Thank you ! <3
Waiting for the regisrations to open up :)
Closing lxqt/lxqt-l10n-deprecated#345

Best, regards !

[tsujan]

Very good!

[stefonarch]

Going through the files, it's not so bad, but found some weird things (example https://github.com/lxqt/lxqt-notificationd/blob/master/config/translations/lxqt-config-notificationd_pl.desktop); committed to the branch
Done until lxqt-panel.

[yan12125]

Thank you very much for the efforts! After some minor fixes, it works 👍 lxqt/qterminal@40a2cef

found some weird things (example https://github.com/lxqt/lxqt-notificationd/blob/master/config/translations/lxqt-config-notificationd_pl.desktop)

Looks lilke a copy-and-paste error lol

[stefonarch]

Ok, most of *.desktop.yaml and *.directory.yaml files are imported in Weblate now, except some components (most of panel, session) which need fixes.

For future memory the import goes like that:

Add new translation component → Import from existing component (second tab) → select component → select matching *.yaml file

• Monolingual base language file: path/translations/component.desktop.yaml
• Template for new translations: path/translations/component.desktop.yaml (weblate still pushes immediately after creation the empty template)
• Edit base file: no (not done everywhere but it looks like weblate marks all existing translations orange when source file changes)

[yan12125]

From #1869 (comment)

Maybe there is reason that adding translation is disabled bu default, will think about it.

Contacting admins should be required, IMHO.

I'm +1 for requiring admins' involvement for adding new languages. Many of recent rebase failures on Weblate is related to new languages. Also, new languages should be pushed immediately to GitHub after adding it from Weblate to reduce the possibility of rebase failures.

[stefonarch]

I'm +1 for requiring admins' involvement for adding new languages.

I spent one hour yesterday evening, with no result. Will test further.
With so many components now this would mean much work too, didn't see a batch function to add a language to the entire project.

• settings and permissions as before (create new template) results in "insufficient permissions to see this page" when clicking the + button, Docs say this should work.
• setting "contact maintainers" when adding new language does just nothing
• then I tested restrict group access to only choosen languages (without pt_PT...) and even this didn't work, no idea why.
• there is an add-on "language consistency" which creates for one repo all language files in every language already present. Not sure about that.
• One option would be remove all seldom used languages and stuff like pt_PT | ar_* de_*, but I saw things like pavucontrol-qt translated 99% in Odia. Also not sure for arabic and spanish (there is already much for Venezuela)

Also, new languages should be pushed immediately to GitHub after adding it from Weblate to reduce the possibility of rebase failures.

This is the case, but until now I waited with merging until weblate committed also the content. It worked fine until this pt_PT flooding. The 2 featherpad locks were other reasons I think.
Atm I reactivated "immediate nofification" when adding new languages.

[tsujan]

Also, new languages should be pushed immediately to GitHub...

If you mean without a PR, I disagree. IMHO, every change to the sources should be made through PRs.

[stefonarch]

I don't think this was a problem until yesterday. There were many new .yaml files, empty and complete and no rebase failure.

Came out that language restriction led to disabled upload (for finnish which was permitted)

I'm thinking about an message on signup and/or an infobox about asking admins for adding new languages: this would mean login in django admin panel and add the given user to the "Create new translation" group. Looks like this is the less painful approach.

[stefonarch]

I've implemented this approach now:
https://translate.lxqt-project.org/projects/lxqt-desktop/#information

Welcome to the LXQt-Weblate Platform!
For adding language files that are not already present please contact administrators.

This means adding the user to "Create New Translation" group, besides "users,viewers".

[tsujan]

I've implemented this approach now:

For a newbie like me, it's clear enough.

[yan12125]

The 2 featherpad locks were other reasons I think.

If I understand the situation correctly, here is the timeline related to the conflicting bg.ts in FeatherPad:

1. An empty bg.ts added and comitted in Weblate
2. The commit with the empty bg.ts is pushed to a pull request (Translations update from Weblate tsujan/FeatherPad#462)
3. Translations are added to bg.ts on Weblate, and the existing commit with an empty bg.ts on Weblate is rewritten (squashed) to include translations. However, the rewriiten commit is not pushed to the existing PR as the one-hour timer is not hit yet
4. The PR with an empty bg.ts is merged
5. git cannot rebase the rewritten commit (with translations in bg.ts) on top of the upstream commit (with an empty bg.ts)

This is the case, but until now I waited with merging until weblate committed also the content.

So yes, always waiting for translations before merging a pull request should also work.

I don't think this was a problem until yesterday. There were many new .yaml files, empty and complete and no rebase failure.

From analysis above, the only remining benefit of requiring admins for new languages is avoiding accidents like pt flooding. It is still nice to have IMO.

I've implemented this approach now:

Thanks a lot for investigating and finishing it!

If you mean without a PR, I disagree. IMHO, every change to the sources should be made through PRs.

Sure empty translation files will still be reviewed via PRs. Sorry for not being clear enough. I was annoyed by so many rebase failures in a row.

[stefonarch]

This was the message, I didn'tunderstand what "trailing whitespace" referref to.

Applying: Weblate commits
.git/rebase-apply/patch:1525: trailing whitespace.
Някои работни среди обаче може да не 
.git/rebase-apply/patch:1702: trailing whitespace.
Ако скриптът не се изпълнява в терминал, 
.git/rebase-apply/patch:1859: trailing whitespace.
отваряни файлове, които FeatherPad показва. 
.git/rebase-apply/patch:1898: trailing whitespace.
        <translation>Ако курсорът е вътре в текстовия изглед, 
warning: 4 lines add whitespace errors.
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
CONFLICT (add/add): Merge conflict in featherpad/data/translations/featherpad_bg.ts

About adding languages I don't see clear yet, I remember new *_he:* files added by @yarons, while for my testuser and 2 others this didn't work, Permissions are the same (users, viewers).

EDIT: wrong, yarons has some admin rights. So all clear.

[stefonarch]

I was annoyed by so many rebase failures in a row.

Here we have still a bus factor of 1...

[yan12125]

Here we have still a bus factor of 1...

Not that bad ;) If I can no longer fix them, you can fix it yourself or give someone else ssh access. IMO such tasks require a little git knowledge, while knowledge about Weblate is not quite relevant.

[stefonarch]

Well, I am not the great git person.

@yan12125 I added Kvantum and wanted to tell you, but I saw you had already added it to the bot. How exactly does work the configuration in weblate? In the Gui only pull URL and GitHub is set. In Featherpad in the gui there is still my repo as push so this doesn't seem to influence anything.

[yan12125]

I added Kvantum and wanted to tell you, but I saw you had already added it to the bot.

Aha, I don't remember it. Maybe done long time ago ;)

How exactly does work the configuration in weblate?

If I understand Weblate correctly, it contains special handling when GitHub is used. Only the upstream repo is required and other fields (e.g., branch name) should be left blank. Weblate will create a fork and open pull requests when needed.

[stefonarch]

But how does weblate know to push just to the Bot? Nowhere in the gui it's configured? Here it is but old stuff from my tests:
https://translate.lxqt-project.org/settings/tsujan/featherpad/#vcs

[yan12125]

But how does weblate know to push just to the Bot?

If I remember t correctly, GITHUB_USERNAME in settings.py is used. Also PRs work only(?) if the push URL and branch are left blank.

[stefonarch]

I see, thanks! Username and token is there.

PRs work only(?) if the push URL and branch are left blank.

well, in Featherpad there are still set to my repo but it looks like just doesn't matter, maybe for branch.

[stefonarch]

If we didn't miss something this lxqt/qps#267 was the last piece missing - everything is up and running in weblate.

[tsujan]

@stefonarch , @yan12125, thanks for the hard work!

[yan12125]

🎉🎉🎉

[stefonarch]

Not to forget @palinek 's yaml files!

I see now the pattern: here the list is alphabetical:
https://github.com/lxqt/lxqt#translation

In the single components it's "newest on top" I guess.

Atm there are 63 translators registered.

[stefonarch]

Any objection adding lxqt-appswitcher under "zjes" (as tsujan and cmst, for the latter I maybe should have asked too...)?
zjes/lxqt-appswitcher#20

[tsujan]

I didn't know that we had something called "lxqt-appswitcher". Any link to tell why it was abandoned?

[stefonarch]

It's 3rd party and it's not abandoned ;)
We have it mentioned in the wiki.
It integrated very well (present as a module in session settings) and has nice configuration, I'm using it for years.

[tsujan]

It's 3rd party and it's not abandoned ;)

Good! I should try it. I thought it was abandoned because its name started with "lxqt-".