TensorCodec: Compact Lossy Compression of Tensors without Strong Data Assumptions

This repository is the official implementation of TensorCodec: Compact Lossy Compression of Tensors without Strong Data Assumptions, Taehyung Kwon, Jihoon Ko, Jinghong jung, and Kijung Shin, ICDM 2023

Requirements

• To run the provided codes, you need to install PyTorch. Since the installation commands for the package rely on the environments, please visit the page (https://pytorch.org/get-started/locally/) for guideline to install the package.

• The code should be run on the folder (./) which includes the TensorCodec folder. The dataset files should be located in ./input.

Initializing the orders

Initialization of the orders of the tensor is implemented in init_order.py.

Positional argument

• -sp, --save_path: path for saving the original tensor.
• -lp, --load_path: path for saving the reordered tensor.

Example commands and results

  python TensorCodec/init_order.py -lp input/action_orig.npy -sp input/action.npy

  order: 0, loss before: 2387.244534244066, loss after: 2387.244534244066
  order: 1, loss before: 46618.97891356207, loss after: 7292.559081679066
  order: 2, loss before: 20461.291436824664, loss after: 14315.363870162731
  Total elapsed time: 8.323100328445435

Running TensorCodec

Training (compressing) and evaluating (decompressing) process are implemented in main.py.

Positional argument

• action: train for compressing the matrix. test for checking the reconstruction loss of the trained model.
• -d, --dataset: data to be compressed

Optional arguments (common)

• -de, --device: GPU id(s) for execution.
• -rk, --rank: rank of TT cores.
• -hs, --hidden_size: size of the hidden dimension.
• -m, --model: type of the model (gru, lstm, mha). The default is lstm.
• -nb, --num_batch: the number of mini-batches for training.
• -b, --batch_size: the number of entries of the tensor which are processed simultaneosly in GPUs.

Optional arguments for training

• -lr, --lr: learning rate.
• -e, --epoch: maximum epoch numbers.
• -sp, --save_path: path for saving the parameters of the trained model and the new orders of the indices of the tensor (bijective function from indices of the reordered tensor to the indices of the original tensor).
• -tol, --tol: tolerance for training.

Optional argument for evaluating

• -lp, --load_path: path for loading the parameters of the trained model and the orders of the indices of the tensor.
• -sp, --save_path: path for saving the reconstructed tensor

Example command

  # Training
  python TensorCodec/main.py train -d action -de 0 1 2 3 -rk 6 -hs 8 -sp output/action_r6_h8 -e 5000 -lr 1 -m lstm -nb 100 -t 100 -b 2097152

  # Evaluating
  python TensorCodec/main.py test -lp output/action_r6_h8 -d action -de 0 1 2 3 -rk 6 -hs 8 -sp output/action_recon.npy

Evaluating the trained model

Command

• We uploaded the trained model for the 3 smallest tensors among 3-order tensors and the smallest tensor among 4-order tensors in the folder 'trained model'.
• The hyperparameters (rank and hidden dimension) correspond to the models with the fewest parameters shown in Figure 3 of the main paper for all datasets.
• You can run the code with the following commands. Note that the device option should be changed depending on the available GPUs.

  python TensorCodec/main.py test -d action -lp 'TensorCodec/trained model/action_r6_h8.pt' -de 0 1 2 3 -rk 6 -hs 8 -sp action_recon.npy
  python TensorCodec/main.py test -d airquality -lp 'TensorCodec/trained model/airquality_r7_h11.pt' -de 0 1 2 3 -rk 7 -hs 11 -sp airquality_recon.npy
  python TensorCodec/main.py test -d uber -lp 'TensorCodec/trained model/uber_r8_h7.pt' -de 0 1 2 3 -rk 8 -hs 7 -sp uber_recon.npy
  python TensorCodec/main.py test -d nyc -lp 'TensorCodec/trained model/nyc_r2_h5.pt' -de 0 1 2 3 -rk 2 -hs 5 -sp nyc_recon.npy

Expected results

	action	airquality	uber	nyc
Fitness	0.65	0.648	0.669	0.558
Compressed Size (bytes)	11686	26031	11870	5227

Real-world datasets we used

Name	shape	Density	Source	Link
Uber	183 x 24 x 1,140	0.138	FROSTT	Link
Air Quality	5,600 x 362 x 6	0.917	Air Korea	Link
Action	100 x 570 x 567	0.393	Multivariate LSTM-FCNs	Link
PEMS-SF	963 X 144 X 440	0.999	The UEA & UCR Time Series Classification Repository	Link
Activity	337 x 570 x 320	0.569	Multivariate LSTM-FCNs	Link
Stock	1,317 x 88 x 916	0.816	Zoom-Tucker	Link
NYC	265 X 265 X 28 X 35	0.118	New York City Government	Link
Absorb	192 x 228 x 30 x 120	1.000	Climate Data at the National Center for Atmospheric Research	Link