Skip to content

Conversation

@thomaseding
Copy link
Contributor

@thomaseding thomaseding commented Feb 26, 2024

What does this PR do?

Adds examples/inference/vae_roundtrip.py to demonstrate VAE inference without any other models or pipelines.

Before submitting

This just introduces a new demo example file. No need to update documents or update tests.

@yiyixuxu
Copy link
Collaborator

why do we need this?

cc @sayakpaul

@thomaseding
Copy link
Contributor Author

thomaseding commented Feb 27, 2024

You dont need it. If you think this is not a worthwhile example script, we can just close the ticket. I just submitted this because it took me a while to figure out how to use the VAE due to its non-tensor return values. And I already wrote the script in the first place to evaluate how good/bad the standard SD VAEs are with heavily aliased images.

@sayakpaul
Copy link
Member

I just submitted this because it took me a while to figure out how to use the VAE due to its non-tensor return values.

You can mitigate that by specifying return_dict=False in your VAE call.

I am okay accommodating this script under the research_projects folder, if you're down.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@thomaseding
Copy link
Contributor Author

thomaseding commented Feb 27, 2024

I am okay accommodating this script under the research_projects folder, if you're down.

Sure why not. Alternatively there is also examples/community if that is better than placing under examples/research_projects/vae

@sayakpaul
Copy link
Member

examples/community are for pipelines that can be loaded with the "custom_pipeline" argument. So, I think research_projects is a better location.

@thomaseding thomaseding force-pushed the teding/feat/vae-roundtrip branch from cfca374 to 12522a6 Compare February 29, 2024 03:53
@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Mar 28, 2024
@sayakpaul
Copy link
Member

Will merge after the CI is green. Sorry for the delay.

@sayakpaul
Copy link
Member

We need to make sure the code quality test passes.

@github-actions github-actions bot removed the stale Issues that haven't received updates label Mar 29, 2024
@github-actions
Copy link
Contributor

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

@github-actions github-actions bot added the stale Issues that haven't received updates label Apr 22, 2024
@sayakpaul
Copy link
Member

@asomoza WDYT about this example?

@asomoza
Copy link
Member

asomoza commented Jun 30, 2024

I like the idea but maybe focus it to a more practical use case?

This is used a lot to store the embeddings of the images, so probably a encode and save and a decode functionality would help the community more.

But I also think it's fine as an example if people struggle on how to do this, since we now have the community scripts, maybe this can be simplified and put there?

@sayakpaul
Copy link
Member

But I also think it's fine as an example if people struggle on how to do this, since we now have the community scripts, maybe this can be simplified and put there?

Yes exactly. I wouldn't mind having it under research_projects really.

@thomaseding, apologies for the delay on our side. Could you maybe fix the styling problems so that we can merge this?

@thomaseding
Copy link
Contributor Author

thomaseding commented Jul 1, 2024

I can't figure out why the style linter is complaining about unsorted imports.

Anyway I fixed a scaling bug and also added TAESD support.

@sayakpaul
Copy link
Member

You can either search for the solutions and use them for the codes provided there "F403", for example. Or you can do "no qa" like done here:

from utils import ( # noqa: E402

@thomaseding thomaseding force-pushed the teding/feat/vae-roundtrip branch from 254d680 to 90cd240 Compare July 1, 2024 03:59
@sayakpaul
Copy link
Member

@asomoza could you give this a final review and merge?

@asomoza
Copy link
Member

asomoza commented Jul 4, 2024

Thanks a lot!

@asomoza asomoza merged commit 2e2684f into huggingface:main Jul 4, 2024
sayakpaul pushed a commit that referenced this pull request Dec 23, 2024
* Add vae_roundtrip.py example

* Add cuda support to vae_roundtrip

* Move vae_roundtrip.py into research_projects/vae

* Fix channel scaling in vae roundrip and also support taesd.

* Apply ruff --fix for CI gatekeep check

---------

Co-authored-by: Álvaro Somoza <asomoza@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale Issues that haven't received updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants