Using matplotlib Qt severely affect floating points operation in numba

[F-A]

I matplotlib Qt is imported before compiling a numba function which takes a vector a numpy array as a parameter, basic floating operators do no longer works and float are automatically converted to integer.

Here is a minimum example of this bug:

import numpy as np
import numba

@numba.njit
def testFunction1(p):
    t = 1.1
    return t

vect = np.ones(100,dtype=np.float64)
print "Test before importing matplotlib: works"
print "Output with a vector as parameter: %f"%testFunction1(vect)
print "Output with a scalar as parameter: %f"%testFunction1(5)

print "Import matplotlib witout QT"
import matplotlib
matplotlib.use('Qt4Agg')
#matplotlib.use('Agg')
import matplotlib.pyplot as plt
plt.figure()

@numba.njit
def testFunction2(p):
    t = 1.1
    return t
print "Test with function compiled before importing matplotlib: works"
print "Output with a vector as parameter: %f"%testFunction1(vect)
print "Output with a scalar as parameter: %f"%testFunction1(5)

print "Test after importing matplotlib: fails"
print "Output with a vector as parameter: %f"%testFunction2(vect)
print "Output with a scalar as parameter: %f"%testFunction2(5)

On my machine this returns:

Test before importing matplotlib: works
Output with a vector as parameter: 1.100000
Output with a scalar as parameter: 1.100000
Import matplotlib witout QT
Test with function compiled before importing matplotlib: works
Output with a vector as parameter: 1.100000
Output with a scalar as parameter: 1.100000
Test after importing matplotlib: fails
Output with a vector as parameter: 1.000000
Output with a scalar as parameter: 1.100000

If instead of Qt4Agg I use Agg, the problem disappears.

The problem seems to be dependent on the configuration. Testing the code on 3 machines, the bug was present in 2 configurations.
I'm using Ubuntu 15.04 with gcc 4.9.2 . The conda packages I'm using are listed in the attached file.
Conda packages.txt

[F-A]

I just tried to install gcc 9.8.5 with conda and I still have the same problem.
Moreoever, I looked at the assembly code for the 2 functions testFunction1 and testFunction2 with a numpy array as input (using NUMBA_DUMP_ASSEMBLY ), surprisingly there is no noticeable differences between the two functions. The problem might come from compiling the assemly code in bitecode.

[pitrou]

This seems to be a locale-related problem in LLVM. A small reproducer (without matplotlib) is:

import locale
import numpy as np

from numba import jit

locale.setlocale(locale.LC_ALL, ('fr_FR', 'UTF-8'))

@jit("float64(float64[:])", nopython=True)
def f(x):
    return 12.3

print(f(np.empty(0)))

[pitrou]

I haven't found any way to reproduce in LLVM without invoking Numba's compile chain.
In any case, you can avoid this bug by running locale.setlocale(locale.LC_NUMERIC, 'C') after running plt.figure().

[pitrou]

I've determined this happens at Numba's NRT optimization step, where we print out and re-parse the LLVM module. At that point the float constants are expressed in exponent notation (e.g. "1.230000e+01"), which LLVM fails to parse again when the wrong locale is selected.

(@sklam, that's an interesting side-effect)

[pitrou]

I filed a bug upstream: https://llvm.org/bugs/show_bug.cgi?id=25788 and a patch at http://reviews.llvm.org/D15375

[pitrou]

We now have a runtime check to error out when LLVM has the wrong behaviour, and our LLVM builds also add a patch fixing the issue. Closing as fixed.

[t-bltg]

when building matplotlib from source (qt 4.8.7 - pyside), one way to revert LC_NUMERIC to 'C' is to patch this way

--- lib/matplotlib/backends/backend_qt4.py  2016-12-18 21:05:26.000000000 +0100
+++ lib/matplotlib/backends/backend_qt4.py  2017-01-11 23:44:42.161493028 +0100
@@ -6,6 +6,7 @@
 import os
 import re
 import signal
+import locale
 import sys
 
 import matplotlib
@@ -61,6 +62,7 @@
         if DEBUG:
             print('FigureCanvasQt qt4: ', figure)
         _create_qApp()
+        locale.setlocale(locale.LC_NUMERIC, 'C')  # NOTE: must call setlocale after qApp = QtWidgets.QApplication, in _create_qApp
 
         # Note different super-calling style to backend_qt5
         QtWidgets.QWidget.__init__(self)

more information here (read the Locale Settings part)