Support translation message contexts

[nvaccessAuto]

Reported by jteh on 2011-05-26 00:47
There are necessarily a lot of short translatable strings in NVDA. However, some of these should be translated differently depending on the context. An example of this is the word "none", which is used for both symbol level and the eSpeak variant name.

Message contexts can be used to differentiate between these. In the code, we would write something like pgettext("symbol level", "none"). "symbol level" would then show up in the .po file in an msgctxt field. (See the Contexts section of the gettext manual for further details.)

Unfortunately, Python doesn't yet support this, and even when it does, it won't be in Python 2.x. However, it is fairly trivial for us to support fetching the right message in our code. The problem is that pygettext and msgfmt.py don't support this. There's a patch to update msgfmt.py, but I think we should just switch to the real gettext tools, as there are also other reasons to do this.

Notes:

• gettext 0.15 or later is required to do this. The only build of gettext 0.15 or later for Windows that I can find is at http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/
  • We need both http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/gettext-runtime_0.18.1.1-2_win32.zip and http://ftp.gnome.org/pub/gnome/binaries/win32/dependencies/gettext-tools-dev_0.18.1.1-2_win32.zip . Extract both into the same directory.
• The context is just prepended to the msgid with a \x04 separator; e.g. "context\x04message". In languageHandler, we can define pgettext() to handle this and install it into the builtin namespace.
• scons needs to be updated to use these tools.
  • Where do we tell the user to place them? We could say gettext in Program Files or we could install them somewhere in the NVDA build tree.

One major problem is that poedit doesn't support viewing contexts yet and many of our translators rely on poedit.

[nvaccessAuto]

Comment 2 by jteh on 2011-05-26 07:27
Also need to add --keyword=pgettext:1c,2 to the xgettext command line.

[nvaccessAuto]

Comment 3 by jteh on 2011-05-26 09:37
It looks like poedit doesn't support viewing msgctxt, but the bug where it stripped msgctxt fields is apparently fixed. So, although msgctxt won't show up, we can still use it safely. If poedit shows translator comments, we can rely on the comment to tell the translator what it means. Does anyone know whether translator comments are viewable (and accessible) in poedit?

[nvaccessAuto]

Comment 4 by orcauser on 2011-05-26 10:09
Personally use a simple utf aware editor, so no problems with this.
If i remember correctly from windows users, comments are visible on the screen, but i dont think they could access them easely using nvda, maybe flat review or something? If Peter doesn't chime in, I will ask for confermation elsewhere.

[nvaccessAuto]

Comment 5 by orcauser (in reply to comment 3) on 2011-11-27 15:50
Replying to jteh:

If poedit shows translator comments, we can rely on the comment to tell the translator what it means. Does anyone know whether translator comments are viewable (and accessible) in poedit?

Yes to both questions, the appmodule suggested in 1811 makes accessing these very straight forward.

I also found localization tips on the boost website [another prebuilt gettext for windows package provided by the CPPCMS project linked off the same page.

[1](1].
and): http://www.boost.org/doc/libs/1_48_0/libs/locale/doc/html/gettext_for_windows.html

[nvaccessAuto]

Comment 6 by orcauser (in reply to comment 1) on 2012-01-10 13:12
Replying to jteh:

• scons needs to be updated to use these tools.
  • Where do we tell the user to place them? We could say gettext in Program Files or we could install

them somewhere in the NVDA build tree.

I presume these are just .exe files without installers, in which case it might be least troublesom for us to add it to the nvda build tree, otherwise we may have to ask the potential developer to modify their path variable to include the gettext path in program files?

[nvaccessAuto]

Comment 7 by jteh (in reply to comment 6) on 2012-01-10 21:33
Replying to orcauser:

I presume these are just .exe files without installers, in which case it might be least troublesom for us to add it to the nvda build tree, otherwise we may have to ask the potential developer to modify their path variable to include the gettext path in program files?

They wouldn't necessarily have to modify their path variable. We could have them extract gettext into a predefined location which scons would know about; e.g. c:\program files\gettext. This is probably still a bit ugly, though, since it's not a real installer which makes this path non-standard. I'm reluctant to install non-include stuff into our build tree, not least because I'm not sure where to put it. That said, it's probably the least ugly option.

[nvaccessAuto]

Comment 8 by jteh on 2012-01-18 07:48
We now use xgettext as of b287038. The build server has been updated accordingly.

[nvaccessAuto]

Comment 9 by jteh on 2012-04-19 23:29
Tasks remaining to fix this:

1. Switch to msgfmt from GNU Gettext instead of msgfmt.py. We already use xgettext, so this isn't a major change.
2. In languageHandler, implement our own pgettext() and install it into the builtin namespace in setLanguage().
   • We might need to refactor setLanguage() a bit.
   • The context is just prepended to the msgid with a \x04 separator; e.g. "context\x04message".

[nvaccessAuto]

Comment 11 by jteh on 2012-09-13 01:26
c0c5244
Changes:
Milestone changed from near-term to 2012.3
State: closed