ML Image Compression

Contents

• Some of my notes on traditional compression in ./JPEG Compression.ipynb

• Implementations

  • 1. An End-to-End Compression Framework Based on Convolutional Neural Networks
  • 2. Learning Convolutional Networks for Content-weighted Image Compression

• Papers & Resources

Implementations

1. An End-to-End Compression Framework Based on Convolutional Neural Networks

Architecture

Architecture used:

Results

Can be trained like:

python train.py --epochs 50 --lr 0.0002 --logs logs/

Method achieved a 75.3% SSIM on the test split of the CLIC dataset.

Notes

• Paper doesn't explicitly mention this but using normalization of the rgb values helped with training

• Making the output of the Cr network go through a sigmoid before going into Co gave better results and makes sense for the gradients to flow although not mentioned in the paper

• Some of the notation of Co(x_hat) in the paper is confusing when showing the residual loss, using eq. (5) works fine

• Because of the "same" padding used in the architecture, seeing somme ugly border artifacts in the reconstructed images

2. Learning Convolutional Networks for Content-weighted Image Compression

Architecture

General idea follows:

More details on the model were excluded from the paper and instead kept at a site that is no longer up. Wayback machine cache of it here.

Results

N/A

Notes

No training code by the authors and paper description is very brief. Not able to reproduce for now.

Papers & Resources

• An End-to-End Compression Framework Based on Convolutional Neural Networks: encoder/decoder architecture tries to learn a compact, image-like representation of an image and use an image codec to store it, then uses interpolation + decoder network in the reconstruction

• Better Compression with Deep Pre-Editing: Learns a transform on the original image such that it can be modified into a "similar" image that can be compressed by a traditional codec but with less artifacts.

• Learning Convolutional Networks for Content-weighted Image Compression: encoder/decoder architecture that extracts a spatial importance mask to predict optimal bit allocation

• High-Fidelity Generative Image Compression: google paper on GANs for compression (code)

• Deep Generative Models for Distribution-Preserving Lossy Compression: training GANs with flexible bitrate. The resulting models behave like generative models at zero bitrate, almost perfectly reconstruct the training data at high enough bitrate (code)

• Full Resolution Image Compression with Recurrent Neural Networks: google paper on using different kinds of RNNs (blog)

• Improved Lossy Image Compression with Priming and Spatially Adaptive Bit Rates for Recurrent Networks: google paper that uses RNNs to progressively reconstruct an image

• Nonlinear Transform Coding: (video talk)

• ML Image Compression Benchmark review

• tensorflow compression repo