Sometimes import numpy as np yields NameError: name 'testing' is not defined

[standarddeviant]

I'm seeing a strange error about numpy 'testing' not being defined on an import of numpy.

Here's the traceback:

Exception in thread Thread-4: 
Traceback (most recent call last): 
  File "C:\Users\stddev\Documents\MATLAB\aplab_python\lib\threading.py", line 916, in _bootstrap_inner 
    self.run() 
  File "C:\Users\stddev\Documents\MATLAB\aplab_python\lib\threading.py", line 864, in run 
    self._target(*self._args, **self._kwargs) 
  File "C:\Users\stddev\src\aplab\ASR\AudioProcAsync.py", line 234, in AudioProcAsyncCreateReceiveJob_1 
    resp = AudioProcAsyncCreateJob_1(filePrefix, language, service) 
  File "C:\Users\stddev\src\aplab\ASR\AudioProcAsync.py", line 46, in AudioProcAsyncCreateJob_1 
    getAudioFile(filePrefix, fileTypes, maxChannels=2, format='linear16') 
  File "C:\Users\stddev\src\aplab\ASR\asrutils.py", line 615, in getAudioFile 
    data, rate = soundfile.read(tmpf, dtype='int16') 
  File "C:\Users\stddev\Documents\MATLAB\aplab_python\lib\site-packages\soundfile.py", line 375, in read 
    data = f.read(frames, dtype, always_2d, fill_value, out) 
  File "C:\Users\stddev\Documents\MATLAB\aplab_python\lib\site-packages\soundfile.py", line 961, in read 
    out = self._create_empty_array(frames, always_2d, dtype) 
  File "C:\Users\stddev\Documents\MATLAB\aplab_python\lib\site-packages\soundfile.py", line 1367, in _create_empty_array 
    import numpy as np 
  File "C:\Users\stddev\Documents\MATLAB\aplab_python\lib\site-packages\numpy\__init__.py", line 152, in <module> 
    test = testing.nosetester._numpy_tester().test 
NameError: name 'testing' is not defined 

I'm using a version of miniconda installed with numpy and other common packages.

The package doing this import is soundfile, but it looks like it's simply running import numpy as np. With the exact same code and input wave files, I see this error about 1% of the time.

By looking at numpy/__init__.py, I'm curious how testing is defined at all. It's not imported explicitly anywhere in the file. I noticed this is changed in master, but not yet in any release. Does anyone understand why this might work most of the time, but fail occasionally?

[njsmith]

I wonder if there's something funny going on with multiple threads trying to import numpy at the same time? I would have thought the interpreter's import lock would avoid that, but I don't know how it works in detail and I think it's changed over time.

What version of Python are you using?
If you import numpy at the top of your script before starting any threads, does that fix the problem?

[standarddeviant]

@njsmith That makes a lot of sense. I can't prove the negative, but I've tested enough trials that I think this fixed my issue. Should the issue stay open for the purposes of general threading safety re: numpy?

[njsmith]

It's probably not a big priority, but I guess if someone can figure out what's going on in more detail it'd be nice to know (and possibly fix, if it's possible in a reasonable way). Unfortunately, I'm not finding much details about the import lock, and there's at least one note that circular imports may get special handling, which is something we could be hitting here (most of numpy's subpackages are going to have import cycles involving the root numpy package).

@standarddeviant What version of Python are you using?

Maybe @ncoghlan will know off the top of his head?

[ncoghlan]

The gist of the current import lock semantics is that the global import lock now only protects a set of per-module-name locks, so different threads will only block each other if they're importing the same module (with deadlock detection to allow for circular dependencies). Original patch is at https://hg.python.org/cpython/rev/edb9ce3a6c2e, with the docs updates added to the now-legacy import lock manipulation API: https://docs.python.org/3/library/imp.html#imp.lock_held

https://bugs.python.org/issue31070 is a recently fixed race condition in that logic which I suspect has been around since the original implementation of per-module import locks in 3.3 (we just didn't notice until now, when some changes to the test suite for other issues made it easier to hit the problem while running the importlib tests).

So if you're able to try this out on Python 3.6.3, and can't readily reproduce the misbehaviour, then that's likely your culprit.

[standarddeviant]

@njsmith The python version I'm using is

Python 3.6.1 |Continuum Analytics, Inc.| (default, May 11 2017, 13:25:24) [MSC v.1900 64 bit (AMD64)] on win32

I'll test out my code (with out the above fix) on on 3.6.3 per @ncoghlan's post. If I can make a short, simple script that exposes this behavior, I'll post it here. Thank you both for the help and information!

[standarddeviant]

Here's a very simple script that will expose the behavior:

import soundfile
import threading
import sys

if len(sys.argv) > 1:
    import numpy

the_wavpath = "C:\\Users\\stddev\\woman1_wb.wav"
def read_wavpath(a_wavpath):
    return soundfile.read(a_wavpath)

for idx in range(1000):
    threading.Thread(target=read_wavpath, args=(the_wavpath,)).start()

When I run with no command line args, numpy is not imported at the top level, and I get many exceptions. On 3.6.0, this cmd line arg switch is the difference between many exceptions and none. I can post the output if desired.

I upgraded to python 3.6.3, and the command line arg has no effect regarding exceptions thrown:

C:\Users\stddev\src>python --version
Python 3.6.3 :: Anaconda custom (64-bit)

C:\Users\stddev\src>python numpy_soundfile_import_race.py

C:\Users\stddev\src>python numpy_soundfile_import_race.py 1

C:\Users\stddev\src>

Closing the issue given that it seems to be fixed in 3.6.3. Thanks again!