QT backend changes locale

[phillipberndt]

I originally filed this as a bug against IPython, see ipython/ipython#3103

At some point, the QT backend calls setlocale() to localize the program. This leads to problems with other Python libraries relying on the default setting "C", especially in the parsing of floating point numbers: In Germany, we use "," as a decimal separator. In my case, I tried to use Cantera, a chemical kinetics library, which, due to the wrong locale setting, effectively floor()ed all numbers and produced lots of errors. See the bug report over at IPython for details.

Testcase:

>>> from ctypes import CDLL, c_char_p
>>> setlocale = CDLL("libc.so.6").setlocale
>>> setlocale.restype = c_char_p
>>> locale = lambda: setlocale(0, 0)
>>> locale()
'C'
>>> import matplotlib
>>> matplotlib.use('Qt4Agg')
>>> from pylab import *
>>> locale()
'C'
>>> plot(1, 2)
[<matplotlib.lines.Line2D object at 0x37d7090>]
>>> locale()
'de_DE.UTF-8'

It clearly is QT which calls setlocale:

>>> # Define locale() as above
>>> import qt
>>> a = qt.QApplication()
>>> a = qt.QApplication([])
>>> w = qt.QWidget()
>>> w.show()
>>> locale()
'de_DE.UTF-8'

I however still suggest you to store the old setlocale(0) value before initializing QT and restore it using setlocale(0, foo) afterwards: If I write a script using a toolkit like QT, I expect it to mess with my environment. When I use a plotting module which does all the magic in the background, I do not. Alternatively, please put a warning somewhere, in large, friendly letters ;-)

[WeatherGod]

is it possible that the QT install is misconfigured?

[phillipberndt]

I am sorry, but I am not familiar with QT. Where would I search for configuration errors?

All I found so far is http://qt-project.org/wiki/QtLocales

The Fallback locale (QSystemLocale::fallbackLocale [doc.qt.nokia.com]) is a QLocale created with the locale id
detected using environment variables (LC_ALL and LANG). If the environment variables are not set,
GetUserDefaultLCID is used on Windows and CFLocaleGetIdentifier is used on Mac OS X.

which makes it seem as if this behavior was intended.

[mdboom]

The standard convention is that libraries that are doing parsing of floating-point numbers are responsible for parsing those numbers in the correct locale (which for most formats is usually supposed to be "C") and not use the locale-aware APIs for doing so. Numpy, for example, implements their own floating-point parsing code to get around this, and matplotlib uses that when appropriate.

matplotlib has had similar bugs in the past, but I'm not aware of any at the moment. You can see how we fixed ours, for example, here:

b516ae0

The gist is to not use the C stdlib API printf, but to use the non-locale-aware functions in the Python/C API PyOS_double_to_string and friends.

Note that pygtk does the same thing as PyQt for example -- I think that's a pretty standard and correct thing for a GUI framework to do, so it's unlikely to change.

So -- sorry to give you the runaround, but I'm not sure this is a matplotlib bug -- it comes from the interaction of loading a GUI framework and the library you're using to read the data (in this case Cantera).

I'm going to close this bug, but if you discover that this is caused by the reading of data by any part of matplotlib, feel free to reopen.

[pitrou]

Related issue: numba/numba#1569
(LLVM produces buggy code after matplotlib changes the LC_NUMERIC locale)

[mdboom]

Yeah -- I suppose it's fairly rare to run libllvm in a GUI application so this never got tested.