handle compressed h5 files

[vagarwal87]

Not sure how difficult this is (or if it's already possible), but it would be a nice if Basset could handle compressed h5 files for training, prediction, etc. I tried de-compressing on the fly and passing the input to basset_train.lua, but it didn't work. Would be a handy feature if it were possible, as some of the h5 files take up a large amount of disk space.

[davek44]

Yea I agree the size of those files is concerning as the available data will continue to grow. It looks like h5py can handle compressed HDF5, but the Torch interface might require some work. Check it out here and see what you think https://github.com/deepmind/torch-hdf5. I'm moving towards TensorFlow for future work, so I'm unlikely to optimize Torch aspects of the code anymore.