Oneflow fails in einops CI, likely due to conflict with new numpy

[arogozhnikov]

Summary

___________________ ERROR collecting tests/test_examples.py ____________________
tests/test_examples.py:5: in <module>
    from tests.test_ops import imp_op_backends
<frozen importlib._bootstrap>:1007: in _find_and_load
    ???
<frozen importlib._bootstrap>:986: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:680: in _load_unlocked
    ???
/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:168: in exec_module
    exec(co, module.__dict__)
tests/test_ops.py:10: in <module>
    imp_op_backends = collect_test_backends(symbolic=False, layers=False)
tests/__init__.py:64: in collect_test_backends
    result.append(backend_type())
einops/_backends.py:554: in __init__
    import oneflow as flow
../../../.local/lib/python3.9/site-packages/oneflow/__init__.py:199: in <module>
    import oneflow.framework.register_class_method_util as register_class_method_util
../../../.local/lib/python3.9/site-packages/oneflow/framework/register_class_method_util.py:17: in <module>
    import oneflow.framework.check_point_v2 as check_point_v2
../../../.local/lib/python3.9/site-packages/oneflow/framework/check_point_v2.py:30: in <module>
    import oneflow.framework.dtype as dtype_util
../../../.local/lib/python3.9/site-packages/oneflow/framework/dtype.py:49: in <module>
    oneflow.bool: np.bool,
/opt/hostedtoolcache/Python/3.9.16/x64/lib/python3.9/site-packages/numpy/__init__.py:284: in __getattr__
    raise AttributeError("module {!r} has no attribute "
E   AttributeError: module 'numpy' has no attribute 'bool'
------------------------------- Captured stderr --------------------------------
2022-12-27 07:50:33.696556: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.16/x64/lib
2022-12-27 07:50:33.696647: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.9.16/x64/lib
2022-12-27 07:50:33.696656: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.

Code to reproduce bug

See CI job for full detailed messages and configuration:

https://github.com/arogozhnikov/einops/actions/runs/3785978910/jobs/6436456017

System Information

• What is your OneFlow installation (pip, source, dockerhub): pip
• OS: linux
• OneFlow version (run python3 -m oneflow --doctor):
• Python version: 3.9
• CUDA driver version: None
• GPU models: None
• Other info:

[MirrorCY]

Just started using oneflow. seems to meet this issue.

Python 3.8.10 (default, Nov 14 2022, 12:59:47) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import oneflow
libibverbs not available, ibv_fork_init skipped
/home/ubuntu/.local/lib/python3.8/site-packages/oneflow/framework/dtype.py:48: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  oneflow.bool: np.bool,
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.8/site-packages/oneflow/__init__.py", line 219, in <module>
    import oneflow.framework.register_class_method_util as register_class_method_util
  File "/home/ubuntu/.local/lib/python3.8/site-packages/oneflow/framework/register_class_method_util.py", line 17, in <module>
    import oneflow.framework.check_point_v2 as check_point_v2
  File "/home/ubuntu/.local/lib/python3.8/site-packages/oneflow/framework/check_point_v2.py", line 30, in <module>
    import oneflow.framework.dtype as dtype_util
  File "/home/ubuntu/.local/lib/python3.8/site-packages/oneflow/framework/dtype.py", line 48, in <module>
    oneflow.bool: np.bool,
  File "/home/ubuntu/.local/lib/python3.8/site-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'bool'
>>> import numpy
>>> print(n
next(      nonlocal   not        numpy      
>>> print(numpy.__version__)
1.24.1
>>> 

[daquexian]

It is fixed by #9649. Could you please install the latest nightly version and try again? Thanks!

[MirrorCY]

It is fixed by #9649. Could you please install the latest nightly version and try again? Thanks!

didn't use the same computer.
OS: wsl2 Ubuntu 22.04.1 LTS
python3 -m pip install -U --pre oneflow -f https://staging.oneflow.info/branch/master/cu120

 python3
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import oneflow
libibverbs not available, ibv_fork_init skipped
/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/dtype.py:48: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  oneflow.bool: np.bool,
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/__init__.py", line 219, in <module>
    import oneflow.framework.register_class_method_util as register_class_method_util
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/register_class_method_util.py", line 17, in <module>
    import oneflow.framework.check_point_v2 as check_point_v2
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/check_point_v2.py", line 30, in <module>
    import oneflow.framework.dtype as dtype_util
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/dtype.py", line 48, in <module>
    oneflow.bool: np.bool,
  File "/home/ubuntu/.local/lib/python3.10/site-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'bool'. Did you mean: 'bool_'?
>>> import numpy
>>> print(numpy.__version__)
1.24.0
>>>

[MirrorCY]

This issue is still there after updating numpy.

ubuntu@DESKTOP-531RKJN:~$ pip install -U numpy
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: numpy in ./.local/lib/python3.10/site-packages (1.24.0)
Collecting numpy
  Downloading https://pypi.tuna.tsinghua.edu.cn/packages/3d/17/2cc40e1ed44f37b0bab7d62e0c6ba88362da23f48e52833ffdd1b9dfc220/numpy-1.24.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.3/17.3 MB 4.0 MB/s eta 0:00:00
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.24.0
    Uninstalling numpy-1.24.0:
      Successfully uninstalled numpy-1.24.0
Successfully installed numpy-1.24.1
ubuntu@DESKTOP-531RKJN:~$ python3
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import oneflow
libibverbs not available, ibv_fork_init skipped
/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/dtype.py:48: FutureWarning: In the future `np.bool` will be defined as the corresponding NumPy scalar.  (This may have returned Python scalars in past versions.
  oneflow.bool: np.bool,
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/__init__.py", line 219, in <module>
    import oneflow.framework.register_class_method_util as register_class_method_util
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/register_class_method_util.py", line 17, in <module>
    import oneflow.framework.check_point_v2 as check_point_v2
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/check_point_v2.py", line 30, in <module>
    import oneflow.framework.dtype as dtype_util
  File "/home/ubuntu/.local/lib/python3.10/site-packages/oneflow/framework/dtype.py", line 48, in <module>
    oneflow.bool: np.bool,
  File "/home/ubuntu/.local/lib/python3.10/site-packages/numpy/__init__.py", line 284, in __getattr__
    raise AttributeError("module {!r} has no attribute "
AttributeError: module 'numpy' has no attribute 'bool'. Did you mean: 'bool_'?
>>> import numpy
>>> print(numpy.__version__)
1.24.1
>>>

[daquexian]

@MirrorCY The command you used is incorrect. For nightly version we only provide cu102 and cu117 packages, as stated at https://github.com/Oneflow-Inc/oneflow#install-with-pip-package. python3 -m pip install -U --pre oneflow -f https://staging.oneflow.info/branch/master/cu120 installs oneflow 0.8.0 (instead of nightly) from PyPI, since the cu120 link (https://staging.oneflow.info/branch/master/cu120) is invalid.

Please uninstall the oneflow package you installed, and install cu117 nightly version by python3 -m pip install --pre oneflow -f https://staging.oneflow.info/branch/master/cu117. Note that you DO NOT need to install cuda 11.7 manually because oneflow already has cuda libraries built-in.

[daquexian]

@arogozhnikov Thanks for your report! We will publish oneflow 0.9.0 soon with the patch for this issue and keep you posted on the progress.

[MirrorCY]

Thank you very much. I'll give it a try!

[MirrorCY]

能用惹 🚀