Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add HunYuanDiT #37

Closed
wants to merge 12 commits into from
Closed

add HunYuanDiT #37

wants to merge 12 commits into from

Conversation

chaojie
Copy link

@chaojie chaojie commented May 16, 2024

The functionality has been implemented first, but further optimization is needed. If it is possible to merge, need to modify the README first

@city96
Copy link
Owner

city96 commented May 16, 2024

Thank you for working on this! There's a few things that I think should be changed, let me know what you think:

First, I feel like loading from just one folder with fixed names is less than clear. The CLIP/mT5 model being separately selectable would make more sense, then we can split it up like this:

  • chinese-roberta-wwm-ext-large.bin goes in models/clip (technically not a CLIP model, not sure why they named it that?)
  • mt5-xl.bin goes in models/t5
  • HunyuanDiT.pt goes in models/unet

(the user would have to rename the files from just "pytorch_model.bin")

Second, there's a lot of small changes that need to be made, and some code redundancy that needs removed. For example, the MT5/CLIP are currently non-standard, and don't work with the regular text encode node, though this might be hard to implement.

I can't really merge this as-is, but I can adapt/fix parts of it and add it to the main repo if that works for you. I'll add your name to the readme for credit.

Oh, and I noticed you're saving the embeddings for testing. Could you send me those for easy debugging? clip_prompt_embeds.pt, etc. It would help a lot.

@chaojie
Copy link
Author

chaojie commented May 17, 2024

I agree with what you said. I have added support for custom resolution, batch image generation, and img2img generation. If there is anything worth referring to, you can go directly https://github.com/chaojie/ComfyUI_ExtraModels

@chaojie chaojie closed this May 20, 2024
@city96
Copy link
Owner

city96 commented May 20, 2024

I've got the initial proof of concept working (trying to integrate it more closely). Custom resolutions/etc will still need more work and it's a WIP at the moment, with a lot of it using the original reference code.

Again, thank you for your work, I will continue to reference it as I fix things.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants