Collage

System for automated integration of deep learning backends.

Installation

Since our implementation uses TVM as the main code generator, install tvm under tvm/.

1. Install dependencies

sudo apt-get update
sudo apt-get install -y python3 python3-dev python3-setuptools gcc libtinfo-dev zlib1g-dev build-essential cmake libedit-dev libxml2-dev

pip3 install --user numpy decorator attrs tornado psutil xgboost cloudpickle pytest

2. Install backends (e.g., cuDNN, MKL,...).

• For cuDNN, install front-end library (commit 360d6e7164dfb7c802493fd1c0464f0d815b852a, tag: v0.1)

3. Create build directory and go to build directory

mkdir tvm/build && cd tvm/build

4. Prepare cmake configuration file. Make sure backend libaries of interest are built together. We provide cmake config that we used for our GPU/CPU experiments (config.cmake.gpu, config.cmake.cpu) in tvm/cmake/. Users may copy it to their build directory and rename it to config.cmake

cp ../cmake/config.cmake.gpu config.cmake

5. Run cmake and make

cmake .. && make -j$(nproc)

6. Declare following environment variables

export COLLAGE_HOME=/path/to/collage/repo
export COLLAGE_TVM_HOME=${COLLAGE_HOME}/tvm
export PYTHONPATH=${COLLAGE_TVM_HOME}/python:${COLLAGE_HOME}/python:${PYTHONPATH}

For more details, please reference TVM installation guide

7. Install Collage dependencies

pip3 install --user graphviz bitarray deap pandas

If you face issues, try

pip3 install --upgrade protobuf
sudo apt-get install graphviz

Demo

Install the following dependencies for deep learning models used for demo.

pip3 install --user torch==1.7.1 torchvision==0.8.2 tqdm protobuf onnx onnxruntime opencv-python transformers

We provide two demos (demo_performance.py, demo_customization.py) under demo/.

• demo_performance.py shows how collage optimizes given workloads with popular backends that Collage provides by default.
• demo_customization.py shows how users can register new backend with their custom codegen, pattern, pattern rule.

For the best result, it is highly recommend to create the tuning log by using autotune_tvm_ops.py before running those demos.

Note

• As Collage uses TVM as its code generator, it cannot support backends that TVM is unable to build. Tested backends are
  • TVM w/o tuning
  • TVM w/ AutoTVM
  • cuDNN v8.0.5
  • cuBLAS v11.3.0
  • TensorRT v7.2.2
  • MKL
  • DNNL
• Since Collage is essentially a profile-guided search, variance in the measurement may affect the final backend placement. For the best result, multiple runs are highly recommended.
• Due to some issues in the implementation, current evolutionary search only supports network implemented in get_network_from_torch(). If an user want to try new network, the network must be implemented within this function.

Cite

@article{jeon2021collage,
  title={Collage: Automated Integration of Deep Learning Backends},
  author={Jeon, Byungsoo and Park, Sunghyun and Liao, Peiyuan and Xu, Sheng and Chen, Tianqi and Jia, Zhihao},
  journal={arXiv preprint arXiv:2111.00655},
  year={2021}
}