Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't import h5py with pandas #19666

Closed
BrendanMartin opened this Issue Feb 12, 2018 · 14 comments

Comments

Projects
None yet
4 participants
@BrendanMartin
Copy link

BrendanMartin commented Feb 12, 2018

Here's what I'm currently doing:

import h5py as h5
import pandas as pd

def init_stores():
    """Check if h5 exists, create one if it doesn't"""
    for store_path in store_paths:
        try:
            with h5.File(store_path, 'r'):
                pass
        except OSError:
            with h5.File(store_path, 'w'):
                logger.info("Created h5 file: {}".format(store_path))
                pass

def update_hdf():
    with pd.HDFStore(store_path) as store:
        print("In the update function")
        exit()

And I'm getting this error:

Traceback (most recent call last):
  File "C:\venv\lib\site-packages\pandas\io\pytables.py", line 445, in __init__
    import tables  # noqa
  File "C:\venv\lib\site-packages\tables\__init__.py", line 90, in <module>
    from .utilsextension import (
ImportError: DLL load failed: The specified procedure could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/.../hdf_stuff.py", line 93, in <module>
    main()
  File "C:/.../hdf_stuff.py", line 89, in main
    update_hdf()
  File "C:/.../hdf_stuff.py", line 39, in update_hdf
    with pd.HDFStore(store_path) as store:
  File "C:\venv\lib\site-packages\pandas\io\pytables.py", line 448, in __init__
    'importing'.format(ex=str(ex)))
ImportError: HDFStore requires PyTables, "DLL load failed: The specified procedure could not be found." problem importing

If I comment out the h5py import, it works.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Feb 12, 2018

Can you fill in your #stuff with what actually causes the failure? If I just be pass there, I don't get an exception.

Can you also include your version info + your h5py and HDF5 versions?

@BrendanMartin

This comment has been minimized.

Copy link
Author

BrendanMartin commented Feb 12, 2018

@TomAugspurger all I have is a print statement (edited) and it fails.

Versions:
python 3.6
pandas 0.22.0
h5py 2.7.1

I'm not sure how to see the HDF5 version. I've only used what comes with pandas.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Feb 12, 2018

OK, your edited version does not fail for me, so it's presumable an environment thing.

What version of pytables?

On mac / linux h5cc -showconfig will print out the HDF5 version. Pandas doesn't bundle HDF5. How'd you install pytables?

@BrendanMartin

This comment has been minimized.

Copy link
Author

BrendanMartin commented Feb 12, 2018

pip show tables
Version: 3.4.2

Installed with pip I believe

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Feb 12, 2018

@jreback

This comment has been minimized.

Copy link
Contributor

jreback commented Feb 12, 2018

please read the docs
HDFStore only works with pytables and has no support with h5py at all

@jreback jreback closed this Feb 12, 2018

@jreback

This comment has been minimized.

Copy link
Contributor

jreback commented Feb 12, 2018

you cannot use h5 files at all - maybe we need a bug warning in the docs - have seen a couple of issues like this in last few months

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Feb 13, 2018

@jreback

This comment has been minimized.

Copy link
Contributor

jreback commented Feb 13, 2018

this was an issue ages ago, but IIRC has been resolved in newer builds of hdf5. In any event this is not a pandas issue, rather an install / conda / pip one.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Feb 13, 2018

@BrendanMartin

This comment has been minimized.

Copy link
Author

BrendanMartin commented Feb 13, 2018

@TomAugspurger @jreback thanks for the help. Should I try to reinstall or just use pytables, or is there another solution? I'm not partial to using h5py since this is the first use in my project.

@TomAugspurger

This comment has been minimized.

Copy link
Contributor

TomAugspurger commented Feb 13, 2018

@BrendanMartin

This comment has been minimized.

Copy link
Author

BrendanMartin commented Feb 13, 2018

@TomAugspurger I'm just using pytables open_file instead. Thanks for the help.

@Gearlux

This comment has been minimized.

Copy link

Gearlux commented Mar 6, 2018

I managed to get pytables and h5py working together on windows10 by downgrading h5py to version 2.7.0.
pip uninstall h5py
pip install h5py=2.7.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.