wrapper of CBSL check v-test for a v test example.
You can use this library save a big vlang array to a file which is compressed by ZSTD, and then you can read it back to an array.
write array.data
read array.data
The library provides the binary serialization with the compression by Zstandard. A motivation of the library is to implement the data-compressed checkpoint/restart, which is well-known technique to recover computer failures in high-performance computing. This library aims to simple and lightweight access for users.
The library support C99 or later, and Fortran 2008 or later: perhaps we use the features are all supported by major compilers.
The library uses CMake version 3.3.x or later.
$ cmake --version
cmake version 3.14.3
CMake suite maintained and supported by Kitware (kitware.com/cmake).
$ mkdir build && cd build
$ cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/install/path ..
$ make
$ make test
$ make install
We can build and install zstd automatically in the build process.
Please pass -D INSTALL_ZSTD=on to cmake, we will install zstd package where CMAKE_INSTALL_PREFIX directory before building the library.
$ cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/install/path -D INSTALL_ZSTD=on ..
$ make
$ make test
$ make install
- CMake version 3.14.3
- GCC version 4.8.5
- Zstandard version 1.4.0
- CentOS Linux release 7.5.1804 (Core)
We provide the library performance benchmark with best and worst case. Please be reminded that the benchmark results not indicate the performance of Zstandard, these measure the overhead (use cost) of this library.
- Best case : all data is zero filled (A compression ratio achieves up to 99%)
- Worst case : data is generated by rand() (A compression ratio is lower than 1%)
benchmark target executes the benchmarks.
...
$ make benchmark
min data size = 262144.00 [B]
max data size = 134217728.00 [B]
data is zero filled (maximum compression)
<write data [Byte]> <time [seconds]> <speed [MiB/sec]>
262144 0.002397 109.360615
524288 0.002887 181.572393
1048576 0.003440 304.827177
...
A compressed file by the library can be decompressed by zstd command.
Copyright 2019 Yuta Hirokawa (University of Tsukuba, Japan)
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
- API error code
- Refectoring tests
The library does not support the binary compatibility and endianness conversion, which is required on the communication across machine. If you want it, please consider using other serialization libraries such as MessagePack.