Skip to content

Releases: legendPerceptor/SZ3

v0.1.2 Layer-by-layer compression

Choose a tag to compare

@legendPerceptor legendPerceptor released this 15 Apr 01:07

Fix a bug of write buffer size for MPI mode.

V-0.1.1 Layer-by-layer compression

Choose a tag to compare

@legendPerceptor legendPerceptor released this 14 Apr 16:06

Fix a bug of write offset in the MPI mode. The previous release's branch is not set to the mpi-multi-read.

v0.1.0 sz3 layer-by-layer compression

Choose a tag to compare

@legendPerceptor legendPerceptor released this 10 Apr 19:25

We add the layer-by-layer compression method and NUMA-adapted parallel compression support.

What problems did we solve?

  • The conventional tensor compression methods including SZ3 cannot compress exceptionally huge 3D tensors because of memory limit. We solve this problem by compressing the large file layer by layer.
  • Most previous compression methods cannot use multiple nodes to compress a single file on supercomputers. We provide a NUMA-adapted method that utilizes MPI programming to compress a single file with many processors on multiple nodes.
  • We also provide a multi-threading compression implementation for systems without a parallel file system or single-node compression with multiple processors.