Support HDF5

[sagetrac-magawake]

We would like to have hdf5 support into sage. This will include the C/C++ bindings and also should have the Java bindings.

To obtain HDF5:
http://www.hdfgroup.org/HDF5/release/obtain5.html

Current version of spkg (install in order):

• https://github.com/downloads/aginiewicz/spkgs/hdf5-1.8.9.spkg
• https://github.com/downloads/aginiewicz/spkgs/h5py-2.0.1.spkg

Component: packages: optional

Keywords: hdf5 h5py hdf5python

Author: Andrzej Giniewicz

Issue created by migration from https://trac.sagemath.org/ticket/8290

[williamstein]

comment:1

Here is a quick spkg that works if you have libhdf5 devel packages installed on your computer:

http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

Install it with

sage -i http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

[sagetrac-magawake]

comment:2

Thanks. Couple of requirements

• Have it completely independent therefore be shipped with SAGE
• Have all the C/C++ bindings includes into the SAGE tar ball

[sagetrac-magawake]

Changed keywords from hdf5 h5py to hdf5 h5py hdf5python

[williamstein]

comment:4

I also made an hdf5 library spkg, but the hdf5 version is evidently too new for h5py:

http://www.hdfgroup.org/ftp/HDF5/prev-releases/hdf5-1.8.0/src/hdf5-1.8.0.tar.gz

since compiling this, then h5py does not work.

[williamstein]

comment:5

Hi,

So if you build

http://sage.math.washington.edu/home/wstein/patches/hdf5-1.6.9.spkg

and

http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

then it should work. Example, do:

   sage -f  http://sage.math.washington.edu/home/wstein/patches/hdf5-1.6.9.spkg  http://sage.math.washington.edu/home/wstein/patches/h5py-1.2.1.spkg

I'm posting this for inclusion in the experimental repo.

[williamstein]

comment:6

Example:

Here's a trivial example showing how to create a new HDF5 file and store a 100 x 20 array of floats:

>>> f = h5py.File("myfile.hdf5", 'w')
>>> f["MyDataset"] = numpy.ones((100,20))

And to get your data back:

>>> dset = f["MyDataset"]
>>> subset = dset[20:80,:]

See http://code.google.com/p/h5py/

[sagetrac-magawake]

comment:7

Is it possible to have atleast hdf 1.8.2? 1.6.x is too old. Most of our files are 1.8.x format.

[jasongrout]

comment:8

pyTables might also be an option. For comparison, see http://www.pytables.org/moin/FAQ#HowdoesPyTablescomparewiththeh5pyproject.3F and http://code.google.com/p/h5py/wiki/FAQ#What%27s_the_difference_between_h5py_and_PyTables_?

[jasongrout]

comment:9

Also, on the front page of the h5py project, it says "Transparently supports both HDF5 1.6 and 1.8.".

[jasongrout]

comment:10

On http://code.google.com/p/h5py/wiki/FAQ, it says that Linux supports up to 1.8.3.

[jasongrout]

comment:11

On http://code.google.com/p/h5py/wiki/FAQ, it says that Linux supports up to 1.8.3.

[sagetrac-aginiewicz]

comment:12

actually h5py should support hdf5 up to 1.8.4 an PyTables up to 1.8.3. I'd say sage should use hdf5 1.8, it's first version to introduce external links and utf8 encoded strings so two things that seems to be quite important, right?

[sagetrac-bascorp2]

comment:13

pictures of puppies

[sagetrac-aginiewicz]

comment:14

I made quick update of those to hdf5 1.8, also this version of h5py works with hdf5 from spkg (at least it works for me - only had to specify "--hdf=$SAGE_LOCAL" to build script).

http://lab15.im.pwr.wroc.pl/~giniew/h5py-1.3.0.spkg

http://lab15.im.pwr.wroc.pl/~giniew/hdf5-1.8.4.spkg (actually, it's 1.8.4 patch 1, latest supported by h5py at time of writing - wasn't sure if I should add the patch1 to name here).

They are based on packages posted here earlier - just updated the src to pointed h5py to local hdf5 copy.

[mwhansen]

comment:15

Ping. These spkgs no longer exist.

[sagetrac-aginiewicz]

comment:16

Well, after a year I forgot about it - the files are present, but url changed. For now I don't have time to update them though (h5py is now 2.0.1 and hdf5 is 1.8.8). Anyway, working links:

http://im.pwr.wroc.pl/~giniew/h5py-1.3.0.spkg

and

http://im.pwr.wroc.pl/~giniew/hdf5-1.8.4.spkg

[sagetrac-aginiewicz]

Author: aginiewicz

[sagetrac-aginiewicz]

comment:17

I've updated spkgs with latest versions. Also, those spkgs now contains repositories and better descriptions in SPKG.txt files (description, license, dependencies, changelog - usual stuff). I've also added links to description to not hunt for them in comments

• https://github.com/downloads/aginiewicz/spkgs/hdf5-1.8.9.spkg
• https://github.com/downloads/aginiewicz/spkgs/h5py-2.0.1.spkg

[jdemeyer]

comment:18

Please fill in your real name as Author.

[sagetrac-aginiewicz]

Changed author from aginiewicz to Andrzej Giniewicz

[jdemeyer]

comment:20

Are these supposed to be standard packages, optional packages or experimental packages?

[sagetrac-aginiewicz]

comment:21

In its current state (i.e. only hdf5+h5py) it might be not that useful to include in standard spkg. I believe it could be optional package. But if some other packages would be compiled with hdf5 support (especially R, which is standard package) situation might change. This is of course only my opinion and it might be not mirror others opinion.

Anyway, I got e-mail that h5py package got broken during upload to github. I had to recreate and reupload it (now, I will not trust github any more and keep other copy around) - it should work now.

[sagetrac-mister-wardrop]

comment:22

This would be really useful for making data interchangeable with other computing platforms. I look forward to seeing this included in Sage by default :).

[mkoeppe]

comment:28

Setting spkg proposals that have not seen recent activity to "sage-wishlist".