BUG: Dependency check custom error loses information #61030

Noratrieb · 2025-03-02T10:07:09Z

Pandas version checks

I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

# in an environment where numpy fails to be imported because for example libstdc++.so.6 doesn't exist
import pandas

Issue Description

I was importing pandas in an environment where libstdc++.so.6 could not be found, which causes numpy to fail to load its C++ extensions.

The error I got was the following:

Traceback (most recent call last):
  File "/home/nora/projects/rplcs-events-tournament-1/report_generator.py", line 2, in <module>
    import pandas as pd
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/pandas/__init__.py", line 19, in <module>
    raise ImportError(
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

This does not contain any information about the missing shared library. If I patch out this custom throw

pandas/pandas/__init__.py

Line 13 in 57fd502

_missing_dependencies.append(f"{_dependency}: {_e}")

here and replace it with just raise _e I get the full error, which mentioned missing libstdc++.so.6 which was very useful to debug the issue (hidden in a details tag as it's big):

The full error

Traceback (most recent call last):
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/numpy/_core/__init__.py", line 23, in <module>
    from . import multiarray
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/numpy/_core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/numpy/_core/overrides.py", line 7, in <module>
    from numpy._core._multiarray_umath import (
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/numpy/__init__.py", line 114, in <module>
    from numpy.__config__ import show_config
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy._core._multiarray_umath import (
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/numpy/_core/__init__.py", line 49, in <module>
    raise ImportError(msg)
ImportError: 

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.12 from "/home/nora/projects/rplcs-events-tournament-1/.venv/bin/python3"
  * The NumPy version is: "2.2.3"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libstdc++.so.6: cannot open shared object file: No such file or directory


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/nora/projects/rplcs-events-tournament-1/report_generator.py", line 2, in <module>
    import pandas as pd
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/pandas/__init__.py", line 16, in <module>
    raise _e
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/pandas/__init__.py", line 14, in <module>
    __import__(_dependency)
  File "/home/nora/projects/rplcs-events-tournament-1/.venv/lib/python3.12/site-packages/numpy/__init__.py", line 119, in <module>
    raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

Expected Behavior

Pandas printing the full error, including nested exceptions.

Installed Versions

It can't run it because my environment is broken, but I can see the relevant code on master. Pandas version is 2.2.3

The text was updated successfully, but these errors were encountered:

rhshadrach · 2025-03-02T13:14:16Z

Thanks for the report! Can you paste the full error that was useful in debugging here.

The current method was implemented due to #12176. I don't think we should just remove it. Perhaps we could say "Run import ... to view the full error".

Noratrieb · 2025-03-02T13:53:39Z

I have pasted the full error behind a details tag, I'll edit the post to make it more visible.

chilin0525 · 2025-03-03T15:46:55Z

Hi @Noratrieb , I tried to reproduce your error message on the main branch using Docker (built from this Dockerfile). I deleted the shared object file libstdc++.so.6 and attempted to import pandas as you did. However, I received a specific error message indicating that libstdc++.so.6 is missing.

Could you provide more details on how to reproduce your error message? Thanks!

>>> import pandas as pd
+ /usr/local/bin/ninja
[1/1] Generating write_version_file with a custom command
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pandas/pandas/__init__.py", line 47, in <module>
    from pandas.core.api import (
  File "/home/pandas/pandas/core/api.py", line 46, in <module>
    from pandas.core.groupby import (
  File "/home/pandas/pandas/core/groupby/__init__.py", line 1, in <module>
    from pandas.core.groupby.generic import (
  File "/home/pandas/pandas/core/groupby/generic.py", line 66, in <module>
    from pandas.core.frame import DataFrame
  File "/home/pandas/pandas/core/frame.py", line 144, in <module>
    from pandas.core.generic import (
  File "/home/pandas/pandas/core/generic.py", line 173, in <module>
    from pandas.core.window import (
  File "/home/pandas/pandas/core/window/__init__.py", line 1, in <module>
    from pandas.core.window.ewm import (
  File "/home/pandas/pandas/core/window/ewm.py", line 11, in <module>
    import pandas._libs.window.aggregations as window_aggregations
ImportError: libstdc++.so.6: cannot open shared object file: No such file or directory

Noratrieb · 2025-03-03T18:16:04Z

I was on a NixOS system where libstdc++ was not in the search path. It looks like something tried to execute ninja in your reproduction, it might help your reproduction to remove ninja as well (I didn't have ninja available either).

Noratrieb · 2025-03-03T18:23:55Z

No, that's not actually relevant, I tried the dockerfile myself. It's interesting that with the dockerfile, the error seems to happen on line 47 of __init__.py, while for me it happened on line 11 for the import sanity check. i'm surprised this check is passing there.

it looks like import numpy actually works in there despite libstdc++.so.6 not being available. that seems surprising.

Noratrieb · 2025-03-03T18:30:35Z

Okay, I got a way to reproduce it. The libstdc++ case confuses me a bit, but if I remove libz (rm /usr/lib/x86_64-linux-gnu/libz.so && rm /lib/x86_64-linux-gnu/libz.so.1) I do actually get an import error when running import numpy and so I get a reproduction for the issue here.

chilin0525 · 2025-03-04T17:43:55Z

Thanks @Noratrieb ! I also got same error message.

>>> import pandas as pd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pandas/pandas/__init__.py", line 16, in <module>
    raise ImportError(
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

chilin0525 · 2025-03-04T18:01:46Z

If we try import numpy after import pandas, we still get the full error message, as shown below. Therefore, if we apply @rhshadrach's suggestion above, adding a hint for users to import numpy should ensure that the full error message is displayed.

>>> import pandas as pd
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pandas/pandas/__init__.py", line 16, in <module>
    raise ImportError(
ImportError: Unable to import required dependencies:
numpy: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.
>>> import numpy as np
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/numpy/core/__init__.py", line 24, in <module>
    from . import multiarray
  File "/usr/local/lib/python3.10/site-packages/numpy/core/multiarray.py", line 10, in <module>
    from . import overrides
  File "/usr/local/lib/python3.10/site-packages/numpy/core/overrides.py", line 8, in <module>
    from numpy.core._multiarray_umath import (
ImportError: libz.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/numpy/__init__.py", line 130, in <module>
    from numpy.__config__ import show as show_config
  File "/usr/local/lib/python3.10/site-packages/numpy/__config__.py", line 4, in <module>
    from numpy.core._multiarray_umath import (
  File "/usr/local/lib/python3.10/site-packages/numpy/core/__init__.py", line 50, in <module>
    raise ImportError(msg)
ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

    https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

  * The Python version is: Python3.10 from "/usr/local/bin/python3"
  * The NumPy version is: "1.26.4"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: libz.so.1: cannot open shared object file: No such file or directory


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/numpy/__init__.py", line 135, in <module>
    raise ImportError(msg) from e
ImportError: Error importing numpy: you should not try to import numpy from
        its source directory; please exit the numpy source tree, and relaunch
        your python interpreter from there.

chilin0525 · 2025-03-04T18:04:28Z

take

Noratrieb added Bug Needs Triage labels Mar 2, 2025

rhshadrach added Error Reporting Needs Discussion and removed Needs Triage labels Mar 2, 2025

rhshadrach removed the Needs Discussion label Mar 2, 2025

github-actions bot assigned chilin0525 Mar 4, 2025

chilin0525 linked a pull request Mar 8, 2025 that will close this issue

BUG: Improve ImportError message to suggest importing dependencies directly for full error details #61084

Open

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: Dependency check custom error loses information #61030

BUG: Dependency check custom error loses information #61030

Noratrieb commented Mar 2, 2025 •

edited

Loading

rhshadrach commented Mar 2, 2025

Noratrieb commented Mar 2, 2025

chilin0525 commented Mar 3, 2025 •

edited

Loading

Noratrieb commented Mar 3, 2025

Noratrieb commented Mar 3, 2025 •

edited

Loading

Noratrieb commented Mar 3, 2025

chilin0525 commented Mar 4, 2025

chilin0525 commented Mar 4, 2025 •

edited

Loading

chilin0525 commented Mar 4, 2025

BUG: Dependency check custom error loses information #61030

BUG: Dependency check custom error loses information #61030

Comments

Noratrieb commented Mar 2, 2025 • edited Loading

Pandas version checks

Reproducible Example

Issue Description

Expected Behavior

Installed Versions

rhshadrach commented Mar 2, 2025

Noratrieb commented Mar 2, 2025

chilin0525 commented Mar 3, 2025 • edited Loading

Noratrieb commented Mar 3, 2025

Noratrieb commented Mar 3, 2025 • edited Loading

Noratrieb commented Mar 3, 2025

chilin0525 commented Mar 4, 2025

chilin0525 commented Mar 4, 2025 • edited Loading

chilin0525 commented Mar 4, 2025

Noratrieb commented Mar 2, 2025 •

edited

Loading

chilin0525 commented Mar 3, 2025 •

edited

Loading

Noratrieb commented Mar 3, 2025 •

edited

Loading

chilin0525 commented Mar 4, 2025 •

edited

Loading