Locale.parse('zh_CN') raises a TypeError on Babel 2.0 with Python 3.3 and 3.4.

[jun66j5]

Python 3.4.3 (default, Apr 13 2015, 22:30:47)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import babel
>>> babel.__version__
'2.0'
>>> from babel.core import Locale
>>> Locale.parse('zh_CN')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv/py34/lib/python3.4/site-packages/babel/core.py", line 286, in parse
    language = get_global('language_aliases').get(language, language)
  File "/venv/py34/lib/python3.4/site-packages/babel/core.py", line 58, in get_global
    _global_data = pickle.load(fileobj)
TypeError: an integer is required (got type str)
>>>

Locale.parse('zh_CN') should work and load zh_CN localedata.

On Python 2.7, it works fine.

Python 2.7.3 (default, Jun 22 2015, 19:33:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import babel
>>> babel.__version__
'2.0'
>>> from babel.core import Locale
>>> Locale.parse('zh_CN')
Locale('zh', territory='CN', script='Hans')
>>>

[sils]

@jun66j5 thanks for your report!

Babel was barely maintained for some time and we're some new people trying to get it up to speed again. Because of this it might take some time until we're able to adress this bug.

[erickwilder]

@jun66j5 is this working in the previous (1.x) release?

[jun66j5]

Babel 1.3 works fine with Python 3.3 and 3.4.

Python 3.3.6 (default, Jan 28 2015, 16:27:12)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import babel
>>> babel.__version__
'1.3'
>>> from babel.core import Locale
>>> Locale.parse('zh_CN')
Locale('zh', territory='CN', script='Hans')

[blisc]

I got the a similar error trying to compile Python documentation with Sphinx.
Downgrading to babel 1.3 seems to have resolved the issue on 3.4.3

[sidloki]

Same error here. If I clone and checkout branch 2.0 and run:

/path/to/babel$ python3.4 setup.py import_cldr release sdist
/path/to/babel$ pip install dist/Babel-2.0.tar.gz

everything works fine (both python3.4 and python2.7). A package created with python2.7 is not working with python3.4.

[erickwilder]

I can't reproduce the error. Both running directly at the Python interpreter and writing a new test case and running against tox/pytest results on the Locale object being created (and tests passing)
I'm on a MacOSX and Python 3.x versions installed with pyenv. Anyone with a Linux environment can test the issue174/locale-parse-py3x branch?

[jun66j5]

babel/global.dat file in Babel-2.0.tar.gz on PyPI seems to be broken.
The global.dat cannot be loaded with Python 3.4 but it can be loaded with Python 2.6.

$ wget -q -P /tmp https://pypi.python.org/packages/source/B/Babel/Babel-2.0.tar.gz#md5=62917719897a81e22dcaa3b17eeb11d8
$ md5sum /tmp/Babel-2.0.tar.gz
62917719897a81e22dcaa3b17eeb11d8  /tmp/Babel-2.0.tar.gz
$ tar tzvf /tmp/Babel-2.0.tar.gz | grep /global.dat
-rw-r--r-- mitsuhiko/staff 116412 2015-07-27 19:35 Babel-2.0/babel/global.dat
$ tar xzf /tmp/Babel-2.0.tar.gz -C /tmp Babel-2.0/babel/global.dat
$ ls -l /tmp/Babel-2.0/babel/global.dat
-rw-r--r-- 1 jun66j5 jun66j5 116412 Jul 27 19:35 /tmp/Babel-2.0/babel/global.dat
$ python3.4
Python 3.4.3 (default, Apr 13 2015, 22:30:47)
[GCC 4.6.3] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> with open('/tmp/Babel-2.0/babel/global.dat', 'rb') as f: data = pickle.load(f)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: an integer is required (got type str)
>>>
$ python2.6
Python 2.6.9 (default, Oct 22 2014, 19:47:46)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> with open('/tmp/Babel-2.0/babel/global.dat', 'rb') as f: data = pickle.load(f)
...
>>> data.keys()
['territory_aliases', 'territory_zones', 'zone_aliases', 'territory_currencies', 'windows_zone_mapping', 'script_aliases', '_version', 'variant_aliases', 'language_aliases', 'meta_zones', 'likely_subtags', 'zone_territories']

[sidloki]

To reproduce the error:

# Python 2.7 environment
$ make clean
$ make import-cldr
$ py.test
# all tests passed

# python3.4 environment
$ make clean-pyc
$ py.test
# some tests failed with TypeError: an integer is required (got type str)

The problem is, that the global.dat created in a python2 environment cannot be used in a python3 environment.

If I create the global.dat in a python3 environment and use it under python2, I get the follwoing failures:

[doctest] babel.core.Locale.parse 
215 
216         >>> Locale.parse(l)
217         Locale('de', territory='DE')
218 
219         This also can perform resolving of likely subtags which it does
220         by default.  This is for instance useful to figure out the most
221         likely locale for a territory you can use ``'und'`` as the
222         language tag:
223 
224         >>> Locale.parse('und_AT')
Expected:
    Locale('de', territory='AT')
Got:
    Locale(u'de', territory=u'AT')

[doctest] babel.numbers.get_territory_currencies
076     of when the currency became active.  The longer the currency is being in
077     use the more to the left of the list it will be.
078 
079     The start date defaults to today.  If no end date is given it will be the
080     same as the start date.  Otherwise a range can be defined.  For instance
081     this can be used to find the currencies in use in Austria between 1995 and
082     2011:
083 
084     >>> from datetime import date
085     >>> get_territory_currencies('AT', date(1995, 1, 1), date(2011, 1, 1))
Expected:
    ['ATS', 'EUR']
Got:
    [u'ATS', u'EUR']

[shimizukawa]

To reproduce the error:

C:\> python
Python 3.4.1 (v3.4.1:c0e311e010fc, May 18 2014, 10:45:13) [MSC v.1600 64 bit (AMD64)] on win32
>>> from babel.messages.pofile import read_po
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\py34\lib\site-packages\babel\messages\__init__.py", line 12, in <module>
    from babel.messages.catalog import *
  File "D:\py34\lib\site-packages\babel\messages\catalog.py", line 23, in <module>
    from babel.dates import format_datetime
  File "D:\py34\lib\site-packages\babel\dates.py", line 28, in <module>
    from babel.util import UTC, LOCALTZ
  File "D:\py34\lib\site-packages\babel\util.py", line 278, in <module>
    from babel import localtime
  File "D:\py34\lib\site-packages\babel\localtime\__init__.py", line 21, in <module>
    from babel.localtime._win32 import _get_localzone
  File "D:\py34\lib\site-packages\babel\localtime\_win32.py", line 18, in <module>
    tz_names = get_global('windows_zone_mapping')
  File "D:\py34\lib\site-packages\babel\core.py", line 58, in get_global
    _global_data = pickle.load(fileobj)
TypeError: an integer is required (got type str)

[mitsuhiko]

This is a really annoying issue. Ideally we would stop using pickle and switch to something we can efficiently load from disk (msgpack as a start). Long term it would be great if we could mmap something, but unfortunately that might be pretty slow with the constant boxing :(

[icoxfog417]

As @jun66j5 says, global.dat on PyPI may be broken.
I installed from GitHub 2.0 branch as below instead of pip install babel, then the error didn't occur.

pip install git+https://github.com/mitsuhiko/babel.git@2.0

My environments is Windows 8.1 and Python 3.4.3.

[thuvh]

I think this error happens in window environments

[erickwilder]

@jun66j5
I saw that you're working on some changes to solve this issue. Any progress? Do you need some help?

[jun66j5]

Thanks, @erickwilder. Updated https://github.com/jun66j5/babel/commits/issue174/workaround.

Root cause is incompatible pickle data for datetime.date classes between Python 2 and 3. In that branch, the uses of the classes are removed and added unit tests for that.

Should I create a pull request with that branch?

[erickwilder]

@jun66j5
Yes please. Submit a pull request so we can review.
We'll need to backport changes of that to the 2.x branch, so your call to use master or the maintenance branch as a target when opening.

[ericholscher]

Ran into this one as well on Windows & 3.4: https://ci.appveyor.com/project/ericholscher/sphinx-autoapi/build/1.0.5

=================================== ERRORS ==================================== 
_________________ ERROR collecting tests/test_integration.py __________________ 
tests\test_integration.py:9: in <module> 
    from sphinx.application import Sphinx 
.tox\py34\lib\site-packages\sphinx\application.py:37: in <module> 
    from sphinx.builders import BUILTIN_BUILDERS 
.tox\py34\lib\site-packages\sphinx\builders\__init__.py:23: in <module> 
    from sphinx.util import i18n, path_stabilize 
.tox\py34\lib\site-packages\sphinx\util\i18n.py:15: in <module> 
    from babel.messages.pofile import read_po 
.tox\py34\lib\site-packages\babel\messages\__init__.py:12: in <module> 
    from babel.messages.catalog import * 
.tox\py34\lib\site-packages\babel\messages\catalog.py:23: in <module> 
    from babel.dates import format_datetime 
.tox\py34\lib\site-packages\babel\dates.py:28: in <module>
    from babel.util import UTC, LOCALTZ
.tox\py34\lib\site-packages\babel\util.py:278: in <module>
    from babel import localtime 
.tox\py34\lib\site-packages\babel\localtime\__init__.py:21: in <module> 
    from babel.localtime._win32 import _get_localzone 
.tox\py34\lib\site-packages\babel\localtime\_win32.py:18: in <module> 
    tz_names = get_global('windows_zone_mapping') 
.tox\py34\lib\site-packages\babel\core.py:58: in get_global 
    _global_data = pickle.load(fileobj) 
E   TypeError: an integer is required (got type str) 

[vstinner]

Cool! The fix (#188) was merged! Would it be possible to get a bugfix release? The latest Babel release on PyPI is incompatible with Python 3 :-(

[sils]

@Haypo we're still not having access to pypi to update that, it's a bit hard to reach the old maintainer. You can install Babel from git in the meantime: https://pip.pypa.io/en/latest/reference/pip_install.html#git

@erickwilder closing as it was fixed