Skip to content

HDFGroup/tar2h5

Repository files navigation

Welcome to tar2h5

Convert Tape ARchives to HDF5 files

Open in GitHub Codespaces

  • archive_checker - check how many files can be extracted from the input tar file
  • archive_checker_64k - check if any files within input tar files larger than 64 KB
  • h5compactor - convert input tar file into hdf5 file, all files within tar file should smaller than 64KB, using small files name as dataset names.
  • h5compactor-sha1 - convert input tar file into hdf5 file, all files within tar file should smaller than 64KB, using small files sha1 values as dataset names.
  • h5shredder - convert input tar file into hdf5 file, no file size limitation, concatenate data and offsets into 4 seperate arrays for better randomized access

Install dependent packages (on Ubuntu-22.04-LTS)

sudo apt install libhdf5-dev libhdf5-openmpi-dev
sudo apt install libarchive13 libarchive-dev
sudo apt install cmake
sudo apt install libopenmpi-dev
sudo apt install libssl1.1 libssl-dev 

Install tar2h5 software

git clone https://github.com/HDFGroup/tar2h5.git
cd tar2h5
cmake .
make

Uninstall tar2h5 software

make clean

Run CTest

ctest

Run tar2h5 software

./bin/archive_checker ./demo/tar2h5-demo.tar
./bin/archive_checker_64k ./demo/tar2h5-demo.tar
./bin/h5compactor ./demo/tar2h5-demo.tar
./bin/h5compactor-sha1 ./demo/tar2h5-demo.tar
./bin/h5shredder ./demo/tar2h5-demo.tar

Visualization with H5Web

https://myhdf5.hdfgroup.org/

Visualization with HDFCompass

https://support.hdfgroup.org/projects/compass/

Output File Format

  • compactor Output Sample compactor
  • compactor-sha1 Output Sample compactor-sha1
  • shredder Output Sample shredder

About

Convert Tape ARchives to HDF5 files

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published