LSQBivariateSpline segmentation fault when quitting the Python terminal

[CongliWANG]

This is not really bugging me, but just would like to know if there is anything wrong with the code (I am on Ubuntu 18.04 if that helps).

Reproducing code example (test.py):

import numpy as np
from scipy.interpolate import LSQBivariateSpline

# generate data
x, y = np.meshgrid(
    np.linspace(0, 10, 30),
    np.linspace(0, 5, 40)
)
x = x.flatten()
y = y.flatten()
z = np.exp(-(x-5)**2/4) + np.exp(-(y-2)**2/3)

# define the knot positions
tx = [1, 2, 4, 5, 6, 8, 9]
ty = [0.2, 0.8, 1.1, 1.8, 2.5, 3, 3.7, 4.2, 4.9]

# get spline fit
s = LSQBivariateSpline(x, y, z, tx, ty, kx=3, ky=3)

# new evaluation
x2, y2 = np.meshgrid(
    np.linspace(0, 10, 50),
    np.linspace(0, 5, 60)
)
z_new = s(x2, y2, grid=False)

Error message:

Run code in terminal as python -i test.py, with the following error bumps out before quitting Python terminal:

python -i tmp.py 
>>> 
corrupted size vs. prev_size
Aborted (core dumped)

or

python tmp.py 
Segmentation fault (core dumped)

This error message shows up randomly, roughly 1 out of 5 trials will trigger the error.

Scipy/Numpy/Python version information:

1.6.0 1.19.4 sys.version_info(major=3, minor=8, micro=0, releaselevel='final', serial=0)

[congma]

I have been having memory errors with LSQBivariateSpline too.

A fairly reliable way to trigger this problem is to call the instantiation of LSQBivariateSpline multiple times in a script, possibly by just increase the coverage of the offending code. Whether python is called with interactive shell (-i) is immaterial.

I've set up a loop that repeatedly call the script, and beside simply Aborted or Segmentation Fault, occasionally I manage to get a Python traceback like this:

ValueError: 0-th dimension must be fixed to 1332 but got 4617315517961601024


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "t.py", line 27, in <module>
    z_new = s(x2, y2, grid=False)
  File "[redacted]/lib/python3.8/site-packages/scipy/interpolate/fitpack2.py", line 939, in __call__
    z, ier = dfitpack.bispeu(tx, ty, c, kx, ky, x, y)
ValueError: failed in converting 3rd argument `c' of dfitpack.bispeu to C/Fortran array

---

The test script was identical to the OP's except for larger input-array sizes and the duplicate calls to LSQBivariateSpline

import numpy as np
from scipy.interpolate import LSQBivariateSpline

# generate data
x, y = np.meshgrid(
    np.linspace(0, 10, 100),
    np.linspace(0, 5, 100)
)
x = x.flatten()
y = y.flatten()
z = np.exp(-(x-5)**2/4) + np.exp(-(y-2)**2/3)

# define the knot positions
tx = np.linspace(0.1, 9.9, 32)
ty = np.linspace(0.1, 4.9, 33)

# get spline fit
s = LSQBivariateSpline(x, y, z, tx, ty, kx=3, ky=3)
s = LSQBivariateSpline(x, y, z, tx, ty, kx=3, ky=3)
s = LSQBivariateSpline(x, y, z, tx, ty, kx=3, ky=3)

# new evaluation
x2, y2 = np.meshgrid(
    np.linspace(0, 10, 100),
    np.linspace(0, 5, 100)
)
z_new = s(x2, y2, grid=False)

[congma]

@CongliWANG If you can build scipy, could you please test your script against a build with the patch #13394? I think what happened was that the array allocated for one of the knot-location arrays was too small and causing out-of-bound access. This could happen, for example, when the spline degrees along x and y are the same but the numbers of knots differ.

[CongliWANG]

Hi @congma ,

Thank you for the fix. I checked out #13394 and built scipy with python -m pip install .. Then:

$ python -c "import scipy; print(scipy.__version__)"
1.7.0.dev0+c28f721

But there is a missing library when I run the test script:

Traceback (most recent call last):
  File "test.py", line 2, in <module>
    from scipy.interpolate import LSQBivariateSpline
  File "/home/wangc0a/.local/lib/python3.8/site-packages/scipy/interpolate/__init__.py", line 165, in <module>
    from .interpolate import *
  File "/home/wangc0a/.local/lib/python3.8/site-packages/scipy/interpolate/interpolate.py", line 11, in <module>
    import scipy.special as spec
  File "/home/wangc0a/.local/lib/python3.8/site-packages/scipy/special/__init__.py", line 636, in <module>
    from . import _basic
  File "/home/wangc0a/.local/lib/python3.8/site-packages/scipy/special/_basic.py", line 15, in <module>
    from . import specfun
ImportError: /home/wangc0a/.local/lib/python3.8/site-packages/scipy/special/specfun.cpython-38-x86_64-linux-gnu.so: undefined symbol: npy_PyErr_ChainExceptionsCause

I believe the installation was not complete. May you advise on how to fix this? Thanks.

[rgommers]

@CongliWANG that's hard to say without a complete build log. Note that pip won't give you one unless you add the -v (for "verbose") flag. I usually use python setup.py develop or some such thing for an in-place build in a separate dev environment. There's documentation for different OSes at http://scipy.github.io/devdocs/dev/contributor/contributor_toc.html#development-environment

If you're doing it just to verify the fix: I can confirm that the segfaults are gone with gh-13394.

[CongliWANG]

Hi @rgommers and @congma ,
Thank you for the message. I have built scipy from source and verified that the fix is working. Thank you again!