-
Notifications
You must be signed in to change notification settings - Fork 106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ppdiffusers] Kandinsky2_2 trainning support #378
Conversation
Thanks for your contribution! |
Tsaiyue seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
ppdiffusers/examples/kandinsky2_2/text_to_image/train_text_to_image_prior_lora.py
Show resolved
Hide resolved
感觉readme里面给使用例子不需要使用 AutoPipelineForText2Image,直接使用对应的pipleine加载就行了 |
我看整体上没有问题了,你这里能测一下所有的在paddle2.6.0 + 这个版本的ppdiffusers能跑通吗? |
Hi, @JunnYu, 在paddle2.6.0下可以顺利跑通四个训练脚本。 同时解决了上述’无法使用pipeline.prior_prior.load_attn_procs()‘的问题,其原因为PriorTransformer需继承UNet2DConditionLoadersMixin,发现这个feature是在这个PR(Revert "【Hackathon 5th No.83】PaddleMIX ppdiffusers models模块功能升级同步HF)下被回滚了。 还有我这个CLA是咋回事来着~,已经签名了却仍然Pending。 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
这个CLA可以之后再看一下,确认一下你提交这个pr的跟你实际登录的是同一个github号,比如邮箱要一样啥的。 |
hi, @Tsaiyue
|
[Kandinsky2.2 训练支持 · Issue #268 · PaddlePaddle/PaddleMIX](#268)
1 前200steps loss对齐结果:
2 其他修改
[ppdiffusers/models/attention_processor.py/LoRAAttnAddedKVProcessor.call] : axis = 1 -> axis = 2
修改原因:运行python train_text_to_image_decoder_lora.py使用LoRAAttnAddedKVProcessor出现concat拼接维度错误。
3 对齐说明
关闭diffusers和ppdiffusers中dataloader中的shuffle,保证数据顺序一致;
设置同一随机种子,并将在trainning loop中造成随机性的noise和timesteps改为由numpy生成统一随机结果(提交代码已删除该逻辑)。
4 存在问题
只能识别部分组件,无法像diffusers自动识别所有组件。故在提交代码中采取下策:在AutoPipelineForText2Image前逐个定义好后传入,不够简洁。目前原因未定,看到一个[diffusers的issue]([https://github.com/huggingface/diffusers/issues/5044)与该问题类似。
----------期待回复与关于合入的建议, Thx :)------------------