Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation issues with support for Python API #279

Closed
ritiek opened this Issue Jun 20, 2018 · 8 comments

Comments

2 participants
@ritiek
Copy link

ritiek commented Jun 20, 2018

@ax3l I am really glad you're taking much efforts to have people adopt these standards which I hope people really do!

Install from source must be as easy as cmake .. && make && make install, otherwise we failed.

Yep, it did build smoothly with no problems when I passed no arguments but there was some trouble when passing -DopenPMD_USE_PYTHON=ON and -DopenPMD_USE_HDF5 to cmake.

It's been a while since I tried to build but AFAIK there was some error regarding MPI and HDF5 which I think building HDF5 and h5py with MPI support fixed it. I think it would be very nice if it were point to relevant docs for building with MPI support. (Also, it isn't immediately clear what MPI support does, it might be good to have some docs what additional features it brings to OpenPMD-API :)

That error regarding HDF5 and MPI went away but then now I face this one.

-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- Can NOT find 'adios_config' - set ADIOS_ROOT, ADIOS_DIR or INSTALL_PREFIX, or check your PATH
-- Could NOT find ADIOS (missing: ADIOS_LIBRARIES ADIOS_INCLUDE_DIRS) (Required is at least version "1.13.1")
CMake Error at CMakeLists.txt:176 (find_package):
  Could not find a package configuration file provided by "pybind11"
  (requested version 2.2.1) with any of the following names:

    pybind11Config.cmake
    pybind11-config.cmake

  Add the installation prefix of "pybind11" to CMAKE_PREFIX_PATH or set
  "pybind11_DIR" to a directory containing one of the above files.  If
  "pybind11" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "/home/ritiek/Downloads/openPMD-api-build/CMakeFiles/CMakeOutput.log".

I tried looking up and I think installing ADIOS might fix it but it was mentioned in under optional I/O dependencies in the README (and/or it could be that I have pybind installed incorrectly) but I didn't really try installing it yet. I guess I'll tinker a bit more and might raise an issue in OpenPMD-API repo with feedback on what could have been better in my perspective. :)


Also, I tried installing it with spack

spack install openpmd-api +python
spack load --dependencies openpmd-api

but for some reason spack always ended up installing Python 2.7.15 even when I created a symbolic link in /usr/bin/python which pointed to my Python 3.6 binary (I wanted it build OpenPMD-API for Python 3.6).

Initially it did say it was running Python 3.6 with spack python but that changed when I did spack install openpmd-api +python. I am not really sure whether this is something to do with OpenPMD-API or spack itself. I haven't used spack before this so I can't say how it works. Do you guys know of a way to specify the Python version?

And conda installation really did go smooth, no errors in da way. :D


That said, I am myself very new to HDF5 and OpenPMD in general, so it may be just something to do with my lack of clear understanding at the moment. :/


This report is just a copy-paste from PlasmaPy/PlasmaPy-PLEPs#13 (comment).

@ax3l ax3l self-assigned this Jun 20, 2018

@ax3l ax3l added this to To do in First Stable Release via automation Jun 20, 2018

@ax3l

This comment has been minimized.

Copy link
Member

ax3l commented Jun 20, 2018

Dear @ritiek, thank you a lot for trying out the API and your detailed report!

there was some trouble when passing -DopenPMD_USE_PYTHON=ON

  • Yes, the trouble here came from the missing dependency pybind11. Usually, one would install it in advance. Anyway, we will now remove this inconvenience now by shipping it with openPMD-api: #281

it isn't immediately clear what MPI support does

  • huh, good point. Indeed, Enable MPI support is a too short CMake string and our official docs could add a section with more intro to it: #449

MPI support for openPMD-api means: MPI parallel backend support for clusters. For example, parallel HDF5 or MPI aggregated ADIOS1 writing. MPI-enabled libraries should always be build from source on clusters, for various reasons (stability, hardware support such as infiniband, performance).

installing ADIOS might fix it

Nope, adios is just yet another backend that can write serial and MPI parallel files :-) You want ADIOS if you write on large clusters with more than ~1'000 MPI ranks (for example, see: https://arxiv.org/abs/1706.00522).

for some reason spack always ended up installing Python 2.7.15

  • Argh, yes sorry for that!
    I improved the documentation in #278

And conda installation really did go smooth, no errors in da way. :D

yay, excellent! :)

I am myself very new to HDF5 and OpenPMD in general

Don't worry, these are exactly the reports we like :-)

For further information and background, have you seen our README.md and manual yet? :-)

@ax3l ax3l added the documentation label Jun 20, 2018

@ax3l

This comment has been minimized.

Copy link
Member

ax3l commented Jun 22, 2018

@ritiek just for curiosity, can you try to compile the latest dev again from source and see if this works for you as well?

@ritiek

This comment has been minimized.

Copy link
Author

ritiek commented Jun 22, 2018

I just tried to build it from source and it seems to build fine now! I'll also try again with spack soon and let you know how it goes.

@ax3l

This comment has been minimized.

Copy link
Member

ax3l commented Jun 22, 2018

Jippey, thank you!

@ritiek

This comment has been minimized.

Copy link
Author

ritiek commented Jun 22, 2018

Spack installs python 3.6.5 too!

Jippey, thank you!

I should be the one to thank you. So, thank you! :D

@ritiek ritiek closed this Jun 22, 2018

First Stable Release automation moved this from To do to Done Jun 22, 2018

@ax3l

This comment has been minimized.

Copy link
Member

ax3l commented Sep 17, 2018

Hi @ritiek, the latest release 0.5.0a fixes a couple of python related issues, mainly attribute writing and proper float handling in datasets. Make sure to update in case you got any problems trying it! :)

@ritiek

This comment has been minimized.

Copy link
Author

ritiek commented Sep 17, 2018

That's great to know @ax3l! Though, there hasn't been much stuff going on around OpenPMD and HDF5 datasets in PlasmaPy at the moment but we'll keep the newer OpenPMD-api version in mind for the future. :)

@ax3l

This comment has been minimized.

Copy link
Member

ax3l commented Sep 17, 2018

Alright, sounds good! Thanks for the feedback :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.