Gettext deprecation fixes (for Python 3.11 compatibility)

[akx]

Fixes #819 (even if Python 3.11 is only due around 2022-10-03).

[codecov]

Codecov Report

Merging #835 (248fa35) into master (9d6803a) will increase coverage by 0.06%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #835      +/-   ##
==========================================
+ Coverage   91.15%   91.21%   +0.06%     
==========================================
  Files          23       23              
  Lines        4149     4145       -4     
==========================================
- Hits         3782     3781       -1     
+ Misses        367      364       -3     

Impacted Files	Coverage Δ
babel/support.py	84.61% <100.00%> (+0.95%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9d6803a...248fa35. Read the comment docs.

[FelixSchwarz]

Looks good to me, deprecating l*gettext() makes sense. Should I run the test suite with Python 3.11 (though I'm pretty confident that your changes really fix the issue)?

[akx]

@FelixSchwarz Sure, doesn't hurt to have someone else give it a shot too! I did test locally with a freshly compiled py3.11a3 before pushing, of course:

The distutils warning was reported in #824 and will be dealt with ... somehow!

platform darwin -- Python 3.11.0a3, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
cachedir: .tox/py311/.pytest_cache
rootdir: /Users/akx/build/babel, configfile: setup.cfg
plugins: cov-3.0.0
collected 3158 items

[100%]

babel/messages/frontend.py:35
  /Users/akx/build/babel/babel/messages/frontend.py:35: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    from distutils import log as distutils_log

  py311: commands succeeded

[akx]

Ping, @FelixSchwarz :-)

[hrnciar]

Hello, I am the original reporter of this issue. Thank you for this PR. I've tried to use it to build babel with Python 3.11.0a5, but unfortunately, it still fails. I see you tested it with the 3rd alpha, so maybe there were some further changes between 3rd and 5th alpha. Also I noticed that failure occurs in npgettext() and pgettext() functions (see log output below). I don't see them changed in your PR so maybe they also need to be fixed.

=================================== FAILURES ===================================
_____________________ TranslationsTestCase.test_dnpgettext _____________________

self = <tests.test_support.TranslationsTestCase testMethod=test_dnpgettext>

    def test_dnpgettext(self):
        self.assertEqualTypeToo(
            'VohD1', self.translations.dngettext('messages1', 'foo1', 'foos1', 1))
        self.assertEqualTypeToo(
            'VohsD1', self.translations.dngettext('messages1', 'foo1', 'foos1', 2))
        self.assertEqualTypeToo(
>           'VohCTXD1', self.translations.dnpgettext('messages1', 'foo', 'foo1',
                                                     'foos1', 1))

tests/test_support.py:149: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
babel/support.py:512: in dnpgettext
    return self._domains.get(domain, self).npgettext(context, singular,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Translations: "PROJECT VERSION">, context = 'foo', singular = 'foo1'
plural = 'foos1', num = 1

    def npgettext(self, context, singular, plural, num):
        """Do a plural-forms lookup of a message id.  `singular` is used as the
        message id for purposes of lookup in the catalog, while `num` is used to
        determine which plural form to use.  The returned message string is an
        8-bit string encoded with the catalog's charset encoding, if known.
    
        If the message id for `context` is not found in the catalog, and a
        fallback is specified, the request is forwarded to the fallback's
        ``npgettext()`` method.  Otherwise, when ``num`` is 1 ``singular`` is
        returned, and ``plural`` is returned in all other cases.
        """
        ctxt_msg_id = self.CONTEXT_ENCODING % (context, singular)
        try:
            tmsg = self._catalog[(ctxt_msg_id, self.plural(num))]
>           if self._output_charset:
E           AttributeError: 'Translations' object has no attribute '_output_charset'

babel/support.py:414: AttributeError
_____________________ TranslationsTestCase.test_dpgettext ______________________

self = <tests.test_support.TranslationsTestCase testMethod=test_dpgettext>

    def test_dpgettext(self):
        self.assertEqualTypeToo(
            'VohD', self.translations.dgettext('messages1', 'foo'))
        self.assertEqualTypeToo(
>           'VohCTXD', self.translations.dpgettext('messages1', 'foo', 'foo'))

tests/test_support.py:128: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
babel/support.py:491: in dpgettext
    return self._domains.get(domain, self).pgettext(context, message)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Translations: "PROJECT VERSION">, context = 'foo', message = 'foo'

    def pgettext(self, context, message):
        """Look up the `context` and `message` id in the catalog and return the
        corresponding message string, as an 8-bit string encoded with the
        catalog's charset encoding, if known.  If there is no entry in the
        catalog for the `message` id and `context` , and a fallback has been
        set, the look up is forwarded to the fallback's ``pgettext()``
        method. Otherwise, the `message` id is returned.
        """
        ctxt_msg_id = self.CONTEXT_ENCODING % (context, message)
        missing = object()
        tmsg = self._catalog.get(ctxt_msg_id, missing)
        if tmsg is missing:
            if self._fallback:
                return self._fallback.pgettext(context, message)
            return message
        # Encode the Unicode tmsg back to an 8-bit string, if possible
>       if self._output_charset:
E       AttributeError: 'Translations' object has no attribute '_output_charset'

babel/support.py:382: AttributeError
_____________________ TranslationsTestCase.test_npgettext ______________________

self = <tests.test_support.TranslationsTestCase testMethod=test_npgettext>

    def test_npgettext(self):
        self.assertEqualTypeToo('Voh1',
                                self.translations.ngettext('foo1', 'foos1', 1))
        self.assertEqualTypeToo('Vohs1',
                                self.translations.ngettext('foo1', 'foos1', 2))
        self.assertEqualTypeToo('VohCTX1',
>                               self.translations.npgettext('foo', 'foo1',
                                                            'foos1', 1))

tests/test_support.py:93: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Translations: "PROJECT VERSION">, context = 'foo', singular = 'foo1'
plural = 'foos1', num = 1

    def npgettext(self, context, singular, plural, num):
        """Do a plural-forms lookup of a message id.  `singular` is used as the
        message id for purposes of lookup in the catalog, while `num` is used to
        determine which plural form to use.  The returned message string is an
        8-bit string encoded with the catalog's charset encoding, if known.
    
        If the message id for `context` is not found in the catalog, and a
        fallback is specified, the request is forwarded to the fallback's
        ``npgettext()`` method.  Otherwise, when ``num`` is 1 ``singular`` is
        returned, and ``plural`` is returned in all other cases.
        """
        ctxt_msg_id = self.CONTEXT_ENCODING % (context, singular)
        try:
            tmsg = self._catalog[(ctxt_msg_id, self.plural(num))]
>           if self._output_charset:
E           AttributeError: 'Translations' object has no attribute '_output_charset'

babel/support.py:414: AttributeError
______________________ TranslationsTestCase.test_pgettext ______________________

self = <tests.test_support.TranslationsTestCase testMethod=test_pgettext>

    def test_pgettext(self):
        self.assertEqualTypeToo('Voh', self.translations.gettext('foo'))
>       self.assertEqualTypeToo('VohCTX', self.translations.pgettext('foo',
                                                                     'foo'))

tests/test_support.py:73: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Translations: "PROJECT VERSION">, context = 'foo', message = 'foo'

    def pgettext(self, context, message):
        """Look up the `context` and `message` id in the catalog and return the
        corresponding message string, as an 8-bit string encoded with the
        catalog's charset encoding, if known.  If there is no entry in the
        catalog for the `message` id and `context` , and a fallback has been
        set, the look up is forwarded to the fallback's ``pgettext()``
        method. Otherwise, the `message` id is returned.
        """
        ctxt_msg_id = self.CONTEXT_ENCODING % (context, message)
        missing = object()
        tmsg = self._catalog.get(ctxt_msg_id, missing)
        if tmsg is missing:
            if self._fallback:
                return self._fallback.pgettext(context, message)
            return message
        # Encode the Unicode tmsg back to an 8-bit string, if possible
>       if self._output_charset:
E       AttributeError: 'Translations' object has no attribute '_output_charset'

babel/support.py:382: AttributeError
=============================== warnings summary ===============================
babel/messages/frontend.py:33
  /builddir/build/BUILD/Babel-2.9.1/babel/messages/frontend.py:33: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
    from distutils import log as distutils_log

babel/numbers.py::babel.numbers.format_number
babel/numbers.py::babel.numbers.format_number
babel/support.py::babel.support.Format.number
tests/test_numbers.py::test_format_number
tests/test_numbers.py::test_format_number
tests/test_support.py::test_format_number
  /builddir/build/BUILD/Babel-2.9.1/babel/numbers.py:352: DeprecationWarning: Use babel.numbers.format_decimal() instead.
    warnings.warn('Use babel.numbers.format_decimal() instead.', DeprecationWarning)

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
FAILED tests/test_support.py::TranslationsTestCase::test_dnpgettext - Attribu...
FAILED tests/test_support.py::TranslationsTestCase::test_dpgettext - Attribut...
FAILED tests/test_support.py::TranslationsTestCase::test_npgettext - Attribut...
FAILED tests/test_support.py::TranslationsTestCase::test_pgettext - Attribute...
============ 4 failed, 3127 passed, 5 skipped, 7 warnings in 5.55s =============

https://download.copr.fedorainfracloud.org/results/@python/python3.11/fedora-rawhide-x86_64/03593239-babel/builder-live.log.gz

[akx]

Hi @hrnciar, thanks for chiming in! I'll give it another shot with 3.11a5.

[akx]

Hi @hrnciar – I compiled a5 and ran tox against 248fa35 (the tip of this PR) and all is well.
The builder-live log you linked checks out revision a34512aaa09f8a9915e7d2cf9aab7253c90d6ec8, which is no commit in this repository.

To add to my confusement, there's a "Encode the Unicode tmsg back to an 8-bit string, if possible" comment as diff context in your comment. According to git log -S, that comment was removed in 3af52a9 (which has been in our master since 2021-09-14).

Can you ensure you're... building with the correct tree? 😄

[hrnciar]

You are correct :), sorry for the confusion. I applied your patch on top of the latest released version 2.9.1., therefore plenty of commits were missing. When I ran tox on top of the master branch with this patch it passed.