[BUG] Import sktime.registry.all_estimators : Weird Stdout of Print Statements

[jokikim]

Describe the bug

In Jupyter, whenever I run the following import - "from sktime.registry import all_estimators" - and then call the all_estimators function, I noticed that subsequent print statements start to output in weird places. It prints in the terminal and also within the the cell where I called the all_estimators function, but not in the actual cell with the print statement.

To Reproduce

# Cell 1) Import
from sktime.registry import all_estimators

# Cell 2) Call function
proba_models = all_estimators( "forecaster", filter_tags={"capability:pred_int": True}, as_dataframe=True)

# Cell 3) Print arbitrary statement
print('Hello World')

Notice the stdout of the print statement.

Expected behavior

In Jupyter, I expect the output of the PRINT statement to be in the same cell.

Additional context

Versions

I'm actually receiving an error when I run show_versions()

SystemError Traceback (most recent call last)
Cell In[2], line 1
----> 1 show_versions()

File ~/opt/anaconda3/lib/python3.9/site-packages/sktime/utils/_maint/_show_versions.py:111, in show_versions()
96 """Print python version, OS version, sktime version, selected dependency versions.
97
98 Pretty prints:
(...)
108 Selected dependencies are as in the DEFAULT_DEPS_TO_SHOW variable
109 """
110 sys_info = _get_sys_info()
--> 111 deps_info = _get_deps_info(deps=DEFAULT_DEPS_TO_SHOW)
113 print("\nSystem:") # noqa: T001, T201
114 for k, stat in sys_info.items():

File ~/opt/anaconda3/lib/python3.9/site-packages/sktime/utils/_maint/_show_versions.py:86, in _get_deps_info(deps)
84 mod = sys.modules[modname]
85 else:
---> 86 mod = importlib.import_module(modname)
87 ver = get_version(mod)
88 deps_info[modname] = ver

File ~/opt/anaconda3/lib/python3.9/importlib/init.py:127, in import_module(name, package)
125 break
126 level += 1
--> 127 return _bootstrap._gcd_import(name[level:], package, level)

File :1030, in _gcd_import(name, package, level)

File :1007, in find_and_load(name, import)

File :986, in find_and_load_unlocked(name, import)

File :680, in _load_unlocked(spec)

File :850, in exec_module(self, module)

File :228, in _call_with_frames_removed(f, *args, **kwds)

File ~/opt/anaconda3/lib/python3.9/site-packages/numba/init.py:42
38 from numba.core.decorators import (cfunc, generated_jit, jit, njit, stencil,
39 jit_module)
41 # Re-export vectorize decorators and the thread layer querying function
---> 42 from numba.np.ufunc import (vectorize, guvectorize, threading_layer,
43 get_num_threads, set_num_threads)
45 # Re-export Numpy helpers
46 from numba.np.numpy_support import carray, farray, from_dtype

File ~/opt/anaconda3/lib/python3.9/site-packages/numba/np/ufunc/init.py:3
1 # -- coding: utf-8 --
----> 3 from numba.np.ufunc.decorators import Vectorize, GUVectorize, vectorize, guvectorize
4 from numba.np.ufunc._internal import PyUFunc_None, PyUFunc_Zero, PyUFunc_One
5 from numba.np.ufunc import _internal, array_exprs

File ~/opt/anaconda3/lib/python3.9/site-packages/numba/np/ufunc/decorators.py:3
1 import inspect
----> 3 from numba.np.ufunc import _internal
4 from numba.np.ufunc.parallel import ParallelUFuncBuilder, ParallelGUFuncBuilder
6 from numba.core.registry import DelayedRegistry

SystemError: initialization of _internal failed without raising an exception

[fkiraly]

Well, that is indeed curious what happens with the print statement. I suppose something internally fiddles with the stdout and doesn't reset it properly.

I can reproduce it on current main.

If the argument suppress_import_stdout is False, the bug does not occur.

Thanks for reporting, @jokikim!

[fkiraly]

The bug is localized in all_objects of scikit-base.

[fkiraly]

Found & fixed: sktime/skbase#193

[fkiraly]

Regarding your show_versions problem, there seems to be some interaction with numba. I don't quite understand what is going on there.

Could you try this to see whether the bug is with numba or with sktime, or with sth else, e.g., your local env?

from importlib import import_module

import_module("numba")

[jokikim]

Yes, it was a numba issue. I updated it and the show_versions() is fixed. Not sure if you need it now, but here's the result:

System:
python: 3.9.12 (main, Apr 5 2022, 01:53:17) [Clang 12.0.0 ]
executable: /Users/jonathankim/opt/anaconda3/bin/python
machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
pip: 21.2.4
sktime: 0.19.1
sklearn: 1.0.2
numpy: 1.24.3
scipy: 1.10.1
pandas: 1.4.2
matplotlib: 3.5.1
joblib: 1.1.0
statsmodels: 0.13.2
numba: 0.57.0
pmdarima: None
tsfresh: None
tensorflow: None
tensorflow_probability: None

[fkiraly]

Thanks! Probably we should add skbase to the default version display: #4694

[fkiraly]

fixed via the fix sktime/skbase#192 in skbase