-
Notifications
You must be signed in to change notification settings - Fork 4.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New modalities #33
Comments
Thanks for reporting @richardrl ! |
@anton-l - we need to make sure that |
Taking a look at the function as is:
Note that the input can be both Also there shouldn't be any I'd be in favor of implementing framework specific (one for PT one for TF) functions called
|
BTW, it's super nice to get all your feedback here @richardrl - thanks a lot! |
Now we use |
@anton-l @patrickvonplaten Thanks for your input thus far. I took the latest commit (as of this moment) and made a minimum reproduction of a 1D MLP model and training. I had to make an additional modification to
It seems to not capture the -33 mode after an epoch or two. Am running the training overnight to see what happens. Welcome you guys to try running this to see if there's anything I did wrong |
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
@anton-l as you've re-opened the issue -> are you planning on doing something with it? |
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. Please note that issues that do not follow the contributing guidelines are likely to be ignored. |
I think we have support for all shapes now, agreed with the stalebot :) |
One of the stated design decisions from the readme was to support arbitrary modalities, not just images.
I'm in the process of trying to adapt the code for 1D vectors (not H x W x C images).
this line:
noisy_images = noise_scheduler.training_step(clean_inputs, noise_samples, timesteps)
adapted from train_unconditional.py, takes in a (16, 198) clean_inputs tensor and returns a (16, 1, 16, 198) noisy_images tensor. So, 1D tensors are not working out of the box. I am just curious if I am taking the right approach to get 1D tensors to work or there's no avoiding custom coding a new DDPMScheduler etc.
EDIT: I got the DDPM training_step function to work with this change:
The text was updated successfully, but these errors were encountered: