-
Notifications
You must be signed in to change notification settings - Fork 16
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Format Markers #152
Add Format Markers #152
Conversation
Post-processing of the POT files looks like a simpler solution than extending the GNU gettext, IMHO a nice solution! 👍 Note: maybe we should support a special comment which would disable this heuristic for specific texts. I do not have any good example, but in theory it might happen that the |
That already exists: # xgettext: c-format
foo(_("...%1..."))
Edit: The script will now not touch a message that already has a format. |
Edit: After renaming the script (now without the .rb extension), this is no longer a problem since the yast-devtools only scan *.rb files. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just some minor comments
Thanks for the excellent PR description! IMHO the program name should not include the |
That's understandable, since we don't want to enlarge the build deps on a critical build path, but unfortunate. I think adding the test and skipping it if Ruby+RSpec is not available is better. I will make a patch. |
Here: 4ef3308 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Please apply my commits for the tests. Removing the .rb
extension would be nice to have.
…mat-hints Integrate unit tests into build (thanks to mvidner)
After all the changes (in particular renaming the script), I just put it through another real-world test with |
✔️ Public Jenkins job #18 successfully finished |
✔️ Internal Jenkins job #7 successfully finished |
✔️ Internal Jenkins job #8 successfully finished |
Trello
https://trello.com/c/Z4PNrMjq/
Bugzilla
https://bugzilla.suse.com/show_bug.cgi?id=954505
https://bugzilla.suse.com/show_bug.cgi?id=980329
Problem
Our translators have a hard time to always get the placeholders right in all the many messages in YaST, and we have several different types of them:
%s
,%d
, ... (and much more complex with field widths etc.)%1
,%2
, ...They use tools that are mostly beyond our control; Weblate or even some Windows programs.
First (Failed) Attempt
Extend the GNU gettext tools to get native support for Ruby in them.
This turned out to be a major task with a hand-crafted Ruby parser for the GNU gettext tools. Ruby is hard to parse, and the available tools in that GNU gettext environment are very basic: Just plain C with the ususal poor C string handling and no regexps etc.; this was abandoned. It would have been a pretty simplistic and probably bug-ridden parser. It is doubtful if that would even have been accepted upstream.
Chosen Pragmatic Approach
Rather than extend the GNU gettext tools and hope for the changes to be accepted and waiting for them to become released and distributed to the translators, this approach uses available mechanisms:
GNU gettext already supports YCP as well as C (and a number of other programming languages, but not Ruby).
You can add markers in the source code to give gettext a hint what format a message is in, and then it will be checked by tools like
msgformat
andmsgmerge
. Available translation tools tend to use that to do at least some amount of error checking..pot file generated by
xgettext
(regardless of detected programming language):Rather than relying on the extraction tools to get this right automatically or adding yet another translation tools related comment manually to the YaST source code, this pull request introduces a new tool to detect our known formats and add that marker to the .pot files after extracting the messages with
rxgettext
(the Ruby variant of xgettext from the gettext Ruby gem).This is intended to be called from
y2makepot
which is called byrake potfiles
.Usage
Supported Formats
Simple printf-like:
%s
,%d
,%x
, ...-> c-format
Complex printf-like:
%02x
,%10.4d
,%-10.4d
, ...-> c-format
printf-like with positional parameters:
%1$d
,%2$s
, ...-> c-format
sformat-like with positional parameters:
%1
,%2
, ...-> ycp-format
Notice that c-format has precedence, so
%2d
etc. has precedence over%2
.named parameters:
%{foo}
,%{bar}
-> perl-brace-format
Caveat: perl-brace-format is really only the part without the
%
, somsgfmt -c
will only check the{foo}
or{bar}
part, not the complete%{foo}
or%{bar}
. So there is still some potential for translators to get this wrong, but it's clearly much better than not checking those named parameters at all.If there are problems, this format can be disabled with the
--no-perl-brace-format
(or-p
) command line switch.Unsupported Format
Named parameters with a format spec:
%<foo>s
,%<bar>02x
We only have a handful (2-5) of those all over YaST, and AFAICS never any more complex than
%<foo>s
or%<bar>d
, so they can easily be replaced by their simpler equivalent%{foo}
or%{bar}
.How to Use It
Simply call
rake potfiles
ory2makepot
. Of course you can also call it directly.Preferably, call this on the .pot files before merging them with the last translations, so the format markers are propagated into all the individual .po files for each language.
It is also possible, of course, to call this on all the .po files, if that is desired.
This script changes the files in-place, i.e. it writes the changes back to each file directly, i.e.
modifies
myfile1.pot
andmyfile2.pot
.Invoke it with
or
to see what it would do. Or simply let it modify the files and check with
git diff
what it changed.Test
Manual tests with https://github.com/shundhammer/gettext-tester
RSpec unit tests in tests/ in this repo.
Those tests are not integrated into the Autotools build logic of this project.Edit: They are now integrated thanks to @mvidner 's patch.
Sample Output
rake pot
in yast-storage-ng (with--verbose
):Package Dependencies
Like the related
y2makepot
, this script uses rubygem-gettext. It relies on the weak dependencies in the .spec file (a suggests) and in they2makepot
invocation.The rationale is that most developers who need this
yast-devtools
package never generate .pot files, so it should not be a hard requirement.