Skip to content
No description, website, or topics provided.
C++ Makefile
Branch: master
Clone or download

SQUISH: Compression for Archival and Distribution of Structured Datasets

This repository contains code for near-optimal compression of relational datasets, using a combination of bayesian networks and arithmetic coding.

Details are provided in the paper "Squish: Near-Optimal Compression for Archival of Relational Datasets", available at this link:

The project is configured as a library, which can be used to create compression program for any relational file format.

An example program using SQUISH can be found in examples/sample.cpp, which compresses csv-style file:

./sample -c covtype.compressed covtype.config
./sample -d covtype.compressed covtype.recovered covtype.config

SQUISH allows user to define new data types and create associated SquID such that they can be compressed using SQUISH. The interface of SquID can be found in model.h. The SquIDModel class allows more flexible SquID creation. It is optional in the sense that most functions can simply return 0 or do nothing.

In SQUISH all primitive data types are implemented using SquID: categorical_model.h/cpp; numerical_model.h/cpp; string_model.h/cpp. A simpler SquID example built upon numerical SquID can be found in examples/corel.h.

You can’t perform that action at this time.