Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Autoencoder implementation and training #79

Merged
merged 51 commits into from
Nov 17, 2023

Conversation

coryMosaicML
Copy link
Collaborator

@coryMosaicML coryMosaicML commented Oct 11, 2023

This PR adds an implementation of the latent diffusion autoencoder, loss function, and training setup for training the autoencoder in composer.

  • Basic network blocks are in diffusion/models/layers.py
  • Complete networks, loss, and the and composer models are in diffusion/models/autoencoder.py
  • For convenience in visualizing image reconstructions, there's a new image logging callback in diffusion/callbacks/log_diffusion_images.py
  • diffusion/algorithms/discriminator_schedule.py contains a composer algorithm for switching on the discriminator.
  • diffusion/train.py also contains some modifications to configure the param groups for training the autoencoder.

diffusion/train.py Outdated Show resolved Hide resolved
diffusion/train.py Outdated Show resolved Hide resolved
diffusion/train.py Outdated Show resolved Hide resolved
diffusion/train.py Outdated Show resolved Hide resolved
diffusion/train.py Outdated Show resolved Hide resolved
Copy link
Contributor

@Skylion007 Skylion007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some nits

Copy link
Contributor

@Landanjs Landanjs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall! The only bug I noticed was in the n layer discriminator and unlikely for us to hit it.

Most of my other comments are about where specific parameter settings came from. I'm sure for most of these you already thought about them and made a decision, so feel free to ignore my comments if they aren't important

diffusion/models/autoencoder.py Outdated Show resolved Hide resolved
diffusion/models/autoencoder.py Outdated Show resolved Hide resolved
diffusion/models/autoencoder.py Show resolved Hide resolved
diffusion/models/autoencoder.py Show resolved Hide resolved
diffusion/models/autoencoder.py Outdated Show resolved Hide resolved
diffusion/models/layers.py Show resolved Hide resolved
diffusion/models/layers.py Outdated Show resolved Hide resolved
diffusion/models/layers.py Show resolved Hide resolved
diffusion/models/layers.py Show resolved Hide resolved
diffusion/models/models.py Outdated Show resolved Hide resolved
@coryMosaicML coryMosaicML merged commit e6e0dc9 into mosaicml:main Nov 17, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants