-
-
Notifications
You must be signed in to change notification settings - Fork 958
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Smart merge of po files #55
Comments
I'd note the comment I just made on issue #98 here. I am currently working on 3-way merge translate toolkit. It really does not have much to do with weblate. I only use .po, but I am writing it so that it should be extensible to any other format supported by translate toolkit (e.g. xliff). |
I ran into a conflict today where the only difference was the po header timestamp. I haven't tested it yet, but looking around I found this gist which points to git-whistles. The heavy lifting by |
There is already one po merge driver in Weblate sources: https://docs.weblate.org/en/latest/admin/continuous.html#updating-repositories |
Just to document current state:
|
Well, even msgcat alone (from gettext tools) is capable to perform the merge. But the merge itself is not sufficient. Most probably, the source of the conflict is pot file change and corresponding po file edit. msgcat --use-first from_weblate.po from_upstream.po -o intermediate.po
msgmerge --previous --lang=my_language intermediate.po new_pot_file.pot -o merged.po
rm intermediate.po Here is a problem: Weblate knows where the pot file is. Git does not. gettextize/autopoint knows it as well (because it creates the infrastructure). ⇒ It would be nice to integrate this feature to gettext tools, and additionally add to a tool generating a merge script (needs access to weblate component description). Also note, that such a generic script requires pot file being merged before all po files. Note that it would be better to replace --use-first by --use-latest (using PO-Revision-Date), but it is not implemented in gettext yet. |
And I believe it shouldn't be relevant. Merge is a merge and the best approach is to do a 3-way merge. I have prototyped 3-way merge for PO files in https://github.com/jan-hudec/podiffutils. I have used it on project for some time (before changing jobs last summer) and it worked. It would deserve integrating into translate-toolkit, but it would need to be polished a bit first.
Not if you do a 3-way merge. Also, the only way to merge the pot is a 3-way merge and the algorithm is the same for pot and po, because pot is just a po with no translations filled. |
@stanislav-brabec This is what we were discussing yesterday. |
hello, thanks to your documentation and my lack of patience, I wrote this script: https://pagure.io/fedora-docs/translations-scripts/blob/master/f/solve_weblate_merge_failures.sh only the pot file path is specific to the way the localization are stored: https://pagure.io/fedora-docs/translations-scripts/blob/master/f/solve_weblate_merge_failures.sh#_43 If I understand correctly, weblate should be able to provide the upstream repository to this script (and the full path of the local git repo for the weblate's one, yes, you can git clone /any/local/path) As we already have the "new_base", exposed by the component API, I assume this should be possible to automate it? |
In my experience, one of the biggest ongoing source of git merge conflicts are the gettext fields |
I disagree. These fields are mandatory for po file format. And they are one of most important fields for human review, e. g. check of maintenance status, check that po files are in sync with pot. |
In the context of Weblate, the .po files will always be committed to
git. That means there will always be a date of the git commit. So for
.pot files, POT-Creation-Date could be derived from the last git commit
that contains a change to the .pot file. For .po files,
PO-Revision-Date could be derived from the last git commit that changed
that .po file. But that could be complicated sometimes.
I guess those dates are used in the merge/update processes? Seems to me
that PO is the only format that has them. In my experience, it would be
worth it to me to disable the features that require POT-Creation-Date
and PO-Revision-Date so that there will be fewer merge conflicts.
But maybe you have something else in mind to solve this?
|
|
If there is a standard way to manage git merge algorithms for specific
file types, and this can be included in GNU/Linux and git distros, then
I think that sounds like a good solution. Or maybe it would be enough
that the Weblate server is setup with this PO merge algorithm, but my
guess is no.
|
Git has support for custom merge strategies. https://git-scm.com/docs/git-merge#_merge_strategies |
right, I mean that if it is something that can be included when someone
does `apt-get install weblate` or something like that. I think it is
important that whatever the solution is, it is easy to deploy and
transparent to use.
|
It is not related with Weblate. But yes, ideally Weblate should install it automatically to hosted git repositories. It is easy to do such three way merge manually:
Writing a generic three way merge command working with any translation layout is more complicated. |
While I agree it's a standard I still think that
I say ,let the end-user make its own decisions and decides if he wants it in the file or not. |
Here's a more generic 3-way merge for .po files: https://stackoverflow.com/a/68799310/334451 |
Thank you for your report; the issue you have reported has just been fixed.
|
@nijel so is this script used automatically by Weblate or needs additional configuration? I can't make it out from comments, and it's not mentioned in the documentation as far as I can see.
|
Please avoid posting the same question twice, #9328 already has the answer. |
I asked here first. Since there was no response for a few days, I assumed you didn't see the notification. |
It should be possible to write smarter po file merger for git, actually using content of the files and pushing translations to new template. It might be indeed complex, but should by doable.
The text was updated successfully, but these errors were encountered: