Skip to content

Commit

Permalink
[3.5] bpo-29755: Fixed the lgettext() family of functions in the gett…
Browse files Browse the repository at this point in the history 
…ext module. (pythonGH-2266)

They now always return bytes.

Updated the gettext documentation.
(cherry picked from commit 26cb465)
  • Loading branch information
@serhiy-storchaka
serhiy-storchaka committed Jun 20, 2017
1 parent 4108606 commit bd4baba
Show file tree
Hide file tree
Showing 4 changed files with 229 additions and 107 deletions.
161 changes: 80 additions & 81 deletions Doc/library/gettext.rst
View file
Expand Up @@ -48,9 +48,10 @@ class-based API instead.

.. function:: bind_textdomain_codeset(domain, codeset=None)

Bind the *domain* to *codeset*, changing the encoding of strings returned by the
:func:`gettext` family of functions. If *codeset* is omitted, then the current
binding is returned.
Bind the *domain* to *codeset*, changing the encoding of byte strings
returned by the :func:`lgettext`, :func:`ldgettext`, :func:`lngettext`
and :func:`ldngettext` functions.
If *codeset* is omitted, then the current binding is returned.


.. function:: textdomain(domain=None)
Expand All @@ -67,28 +68,14 @@ class-based API instead.
:func:`_` in the local namespace (see examples below).


.. function:: lgettext(message)

Equivalent to :func:`gettext`, but the translation is returned in the
preferred system encoding, if no other encoding was explicitly set with
:func:`bind_textdomain_codeset`.


.. function:: dgettext(domain, message)

Like :func:`gettext`, but look the message up in the specified *domain*.


.. function:: ldgettext(domain, message)

Equivalent to :func:`dgettext`, but the translation is returned in the
preferred system encoding, if no other encoding was explicitly set with
:func:`bind_textdomain_codeset`.
Like :func:`.gettext`, but look the message up in the specified *domain*.


.. function:: ngettext(singular, plural, n)

Like :func:`gettext`, but consider plural forms. If a translation is found,
Like :func:`.gettext`, but consider plural forms. If a translation is found,
apply the plural formula to *n*, and return the resulting message (some
languages have more than two plural forms). If no translation is found, return
*singular* if *n* is 1; return *plural* otherwise.
Expand All @@ -101,24 +88,33 @@ class-based API instead.
formulas for a variety of languages.


.. function:: lngettext(singular, plural, n)

Equivalent to :func:`ngettext`, but the translation is returned in the
preferred system encoding, if no other encoding was explicitly set with
:func:`bind_textdomain_codeset`.


.. function:: dngettext(domain, singular, plural, n)

Like :func:`ngettext`, but look the message up in the specified *domain*.


.. function:: lgettext(message)
.. function:: ldgettext(domain, message)
.. function:: lngettext(singular, plural, n)
.. function:: ldngettext(domain, singular, plural, n)

Equivalent to :func:`dngettext`, but the translation is returned in the
preferred system encoding, if no other encoding was explicitly set with
Equivalent to the corresponding functions without the ``l`` prefix
(:func:`.gettext`, :func:`dgettext`, :func:`ngettext` and :func:`dngettext`),
but the translation is returned as a byte string encoded in the preferred
system encoding if no other encoding was explicitly set with
:func:`bind_textdomain_codeset`.

.. warning::

These functions should be avoided in Python 3, because they return
encoded bytes. It's much better to use alternatives which return
Unicode strings instead, since most Python applications will want to
manipulate human readable text as strings instead of bytes. Further,
it's possible that you may get unexpected Unicode-related exceptions
if there are encoding problems with the translated strings. It is
possible that the ``l*()`` functions will be deprecated in future Python
versions due to their inherent problems and limitations.


Note that GNU :program:`gettext` also defines a :func:`dcgettext` method, but
this was deemed not useful and so it is currently unimplemented.
Expand Down Expand Up @@ -179,8 +175,9 @@ class can also install themselves in the built-in namespace as the function
names are cached. The actual class instantiated is either *class_* if
provided, otherwise :class:`GNUTranslations`. The class's constructor must
take a single :term:`file object` argument. If provided, *codeset* will change
the charset used to encode translated strings in the :meth:`lgettext` and
:meth:`lngettext` methods.
the charset used to encode translated strings in the
:meth:`~NullTranslations.lgettext` and :meth:`~NullTranslations.lngettext`
methods.

If multiple files are found, later files are used as fallbacks for earlier ones.
To allow setting the fallback, :func:`copy.copy` is used to clone each
Expand Down Expand Up @@ -250,26 +247,29 @@ are the methods of :class:`NullTranslations`:

.. method:: gettext(message)

If a fallback has been set, forward :meth:`gettext` to the fallback.
Otherwise, return the translated message. Overridden in derived classes.


.. method:: lgettext(message)

If a fallback has been set, forward :meth:`lgettext` to the fallback.
Otherwise, return the translated message. Overridden in derived classes.
If a fallback has been set, forward :meth:`.gettext` to the fallback.
Otherwise, return *message*. Overridden in derived classes.


.. method:: ngettext(singular, plural, n)

If a fallback has been set, forward :meth:`ngettext` to the fallback.
Otherwise, return the translated message. Overridden in derived classes.
Otherwise, return *singular* if *n* is 1; return *plural* otherwise.
Overridden in derived classes.


.. method:: lgettext(message)
.. method:: lngettext(singular, plural, n)

If a fallback has been set, forward :meth:`lngettext` to the fallback.
Otherwise, return the translated message. Overridden in derived classes.
Equivalent to :meth:`.gettext` and :meth:`ngettext`, but the translation
is returned as a byte string encoded in the preferred system encoding
if no encoding was explicitly set with :meth:`set_output_charset`.
Overridden in derived classes.

.. warning::

These methods should be avoided in Python 3. See the warning for the
:func:`lgettext` function.


.. method:: info()
Expand All @@ -279,32 +279,28 @@ are the methods of :class:`NullTranslations`:

.. method:: charset()

Return the "protected" :attr:`_charset` variable, which is the encoding of
the message catalog file.
Return the encoding of the message catalog file.


.. method:: output_charset()

Return the "protected" :attr:`_output_charset` variable, which defines the
encoding used to return translated messages in :meth:`lgettext` and
:meth:`lngettext`.
Return the encoding used to return translated messages in :meth:`.lgettext`
and :meth:`.lngettext`.


.. method:: set_output_charset(charset)

Change the "protected" :attr:`_output_charset` variable, which defines the
encoding used to return translated messages.
Change the encoding used to return translated messages.


.. method:: install(names=None)

This method installs :meth:`self.gettext` into the built-in namespace,
This method installs :meth:`.gettext` into the built-in namespace,
binding it to ``_``.

If the *names* parameter is given, it must be a sequence containing the
names of functions you want to install in the builtins namespace in
addition to :func:`_`. Supported names are ``'gettext'`` (bound to
:meth:`self.gettext`), ``'ngettext'`` (bound to :meth:`self.ngettext`),
addition to :func:`_`. Supported names are ``'gettext'``, ``'ngettext'``,
``'lgettext'`` and ``'lngettext'``.

Note that this is only one way, albeit the most convenient way, to make
Expand Down Expand Up @@ -349,49 +345,52 @@ If the :file:`.mo` file's magic number is invalid, the major version number is
unexpected, or if other problems occur while reading the file, instantiating a
:class:`GNUTranslations` class can raise :exc:`OSError`.

The following methods are overridden from the base class implementation:

.. class:: GNUTranslations

.. method:: GNUTranslations.gettext(message)
The following methods are overridden from the base class implementation:

Look up the *message* id in the catalog and return the corresponding message
string, as a Unicode string. If there is no entry in the catalog for the
*message* id, and a fallback has been set, the look up is forwarded to the
fallback's :meth:`gettext` method. Otherwise, the *message* id is returned.
.. method:: gettext(message)

Look up the *message* id in the catalog and return the corresponding message
string, as a Unicode string. If there is no entry in the catalog for the
*message* id, and a fallback has been set, the look up is forwarded to the
fallback's :meth:`~NullTranslations.gettext` method. Otherwise, the
*message* id is returned.

.. method:: GNUTranslations.lgettext(message)

Equivalent to :meth:`gettext`, but the translation is returned as a
bytestring encoded in the selected output charset, or in the preferred system
encoding if no encoding was explicitly set with :meth:`set_output_charset`.
.. method:: ngettext(singular, plural, n)

Do a plural-forms lookup of a message id. *singular* is used as the message id
for purposes of lookup in the catalog, while *n* is used to determine which
plural form to use. The returned message string is a Unicode string.

.. method:: GNUTranslations.ngettext(singular, plural, n)
If the message id is not found in the catalog, and a fallback is specified,
the request is forwarded to the fallback's :meth:`~NullTranslations.ngettext`
method. Otherwise, when *n* is 1 *singular* is returned, and *plural* is
returned in all other cases.

Do a plural-forms lookup of a message id. *singular* is used as the message id
for purposes of lookup in the catalog, while *n* is used to determine which
plural form to use. The returned message string is a Unicode string.
Here is an example::

If the message id is not found in the catalog, and a fallback is specified, the
request is forwarded to the fallback's :meth:`ngettext` method. Otherwise, when
*n* is 1 *singular* is returned, and *plural* is returned in all other cases.
n = len(os.listdir('.'))
cat = GNUTranslations(somefile)
message = cat.ngettext(
'There is %(num)d file in this directory',
'There are %(num)d files in this directory',
n) % {'num': n}

Here is an example::

n = len(os.listdir('.'))
cat = GNUTranslations(somefile)
message = cat.ngettext(
'There is %(num)d file in this directory',
'There are %(num)d files in this directory',
n) % {'num': n}
.. method:: lgettext(message)
.. method:: lngettext(singular, plural, n)

Equivalent to :meth:`.gettext` and :meth:`.ngettext`, but the translation
is returned as a byte string encoded in the preferred system encoding
if no encoding was explicitly set with
:meth:`~NullTranslations.set_output_charset`.

.. method:: GNUTranslations.lngettext(singular, plural, n)
.. warning::

Equivalent to :meth:`gettext`, but the translation is returned as a
bytestring encoded in the selected output charset, or in the preferred system
encoding if no encoding was explicitly set with :meth:`set_output_charset`.
These methods should be avoided in Python 3. See the warning for the
:func:`lgettext` function.


Solaris message catalog support
Expand Down Expand Up @@ -509,7 +508,7 @@ module::

import gettext
t = gettext.translation('spam', '/usr/share/locale')
_ = t.lgettext
_ = t.gettext


Localizing your application
Expand Down
40 changes: 23 additions & 17 deletions Lib/gettext.py
View file
Expand Up @@ -270,7 +270,9 @@ def gettext(self, message):
def lgettext(self, message):
if self._fallback:
return self._fallback.lgettext(message)
return message
if self._output_charset:
return message.encode(self._output_charset)
return message.encode(locale.getpreferredencoding())

def ngettext(self, msgid1, msgid2, n):
if self._fallback:
Expand All @@ -284,9 +286,12 @@ def lngettext(self, msgid1, msgid2, n):
if self._fallback:
return self._fallback.lngettext(msgid1, msgid2, n)
if n == 1:
return msgid1
tmsg = msgid1
else:
return msgid2
tmsg = msgid2
if self._output_charset:
return tmsg.encode(self._output_charset)
return tmsg.encode(locale.getpreferredencoding())

def info(self):
return self._info
Expand Down Expand Up @@ -368,7 +373,7 @@ def _parse(self, fp):
if mlen == 0:
# Catalog description
lastk = None
for b_item in tmsg.split('\n'.encode("ascii")):
for b_item in tmsg.split(b'\n'):
item = b_item.decode().strip()
if not item:
continue
Expand Down Expand Up @@ -416,24 +421,24 @@ def lgettext(self, message):
if tmsg is missing:
if self._fallback:
return self._fallback.lgettext(message)
return message
tmsg = message
if self._output_charset:
return tmsg.encode(self._output_charset)
return tmsg.encode(locale.getpreferredencoding())

def lngettext(self, msgid1, msgid2, n):
try:
tmsg = self._catalog[(msgid1, self.plural(n))]
if self._output_charset:
return tmsg.encode(self._output_charset)
return tmsg.encode(locale.getpreferredencoding())
except KeyError:
if self._fallback:
return self._fallback.lngettext(msgid1, msgid2, n)
if n == 1:
return msgid1
tmsg = msgid1
else:
return msgid2
tmsg = msgid2
if self._output_charset:
return tmsg.encode(self._output_charset)
return tmsg.encode(locale.getpreferredencoding())

def gettext(self, message):
missing = object()
Expand Down Expand Up @@ -573,11 +578,11 @@ def dgettext(domain, message):
return t.gettext(message)

def ldgettext(domain, message):
codeset = _localecodesets.get(domain)
try:
t = translation(domain, _localedirs.get(domain, None),
codeset=_localecodesets.get(domain))
t = translation(domain, _localedirs.get(domain, None), codeset=codeset)
except OSError:
return message
return message.encode(codeset or locale.getpreferredencoding())
return t.lgettext(message)

def dngettext(domain, msgid1, msgid2, n):
Expand All @@ -592,14 +597,15 @@ def dngettext(domain, msgid1, msgid2, n):
return t.ngettext(msgid1, msgid2, n)

def ldngettext(domain, msgid1, msgid2, n):
codeset = _localecodesets.get(domain)
try:
t = translation(domain, _localedirs.get(domain, None),
codeset=_localecodesets.get(domain))
t = translation(domain, _localedirs.get(domain, None), codeset=codeset)
except OSError:
if n == 1:
return msgid1
tmsg = msgid1
else:
return msgid2
tmsg = msgid2
return tmsg.encode(codeset or locale.getpreferredencoding())
return t.lngettext(msgid1, msgid2, n)

def gettext(message):
Expand Down

0 comments on commit bd4baba

Please sign in to comment.