Skip to content
This repository was archived by the owner on Aug 6, 2025. It is now read-only.

Conversation

@mmuckley
Copy link
Contributor

@mmuckley mmuckley commented Jun 23, 2023

This pull request implements, the generative image compression model from the following paper:

MJ Muckley, A El-Nouby, K Ullrich, H Jegou, J Verbeek.
Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models.
In ICML, 2023.

The PR includes:

  • A recursive bottleneck structure for implementing VQ-VAEs.
  • A new VQ-VAE model, based on the VQ-GAN paper, that has XCiT attention for improved memory efficiency.
  • A modification of the OASIS U-Net discriminator that clips the discriminator in the upsampling path corresponding to the VQ-VAE latent space.
  • A PyTorch Lightning module for HiFiC-style training of hyperprior models for targeted bitrates.
  • A PyTorch Lightning module for fine-tuning the HiFiC-autoencoders with MS-ILLM

Installation instructions and basic usage are covered in the README in the illm project folder

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 23, 2023
@mmuckley mmuckley changed the title Mmuckley/illm oss MS-ILLM models and training scripts Jun 23, 2023
@mmuckley mmuckley requested a review from marksibrahim July 7, 2023 14:28
@mmuckley mmuckley marked this pull request as ready for review July 7, 2023 14:28
@mmuckley mmuckley requested a review from KarenUllrich July 7, 2023 14:28
@mmuckley mmuckley force-pushed the mmuckley/loss_fns branch from 4c584a1 to ac9fa1a Compare July 12, 2023 22:37
@mmuckley mmuckley force-pushed the mmuckley/illm_oss branch from 1dae197 to 4148b0b Compare July 12, 2023 22:40
@mmuckley mmuckley requested a review from QuentinDuval July 14, 2023 14:52
@mmuckley mmuckley force-pushed the mmuckley/loss_fns branch from ac9fa1a to c860050 Compare July 28, 2023 19:45
Base automatically changed from mmuckley/loss_fns to main July 28, 2023 19:56
@mmuckley mmuckley force-pushed the mmuckley/illm_oss branch from d4f7992 to afedbee Compare July 28, 2023 19:58
@mmuckley mmuckley force-pushed the mmuckley/illm_oss branch from afedbee to b718ccd Compare August 9, 2023 01:56
@mmuckley mmuckley merged commit 73dcfe2 into main Aug 9, 2023
@mmuckley mmuckley deleted the mmuckley/illm_oss branch August 9, 2023 03:20
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants