IP-Adapter face id #2434
IP-Adapter face id #2434
Conversation
|
Hi, I have tested your latest branch, but got this error while ip-adapter-faceid_sd15[0a1757e9] loaded: Missing key(s) in state_dict: "perceiver_resampler.proj_in.weight", "perceiver_resampler.proj_in.bias", "perceiver_resampler.proj_out.weight", "perceiver_resampler.proj_out.bias", "perceiver_resampler.norm_out.weight", "perceiver_resampler.norm_out.bias", "perceiver_resampler.layers.0.0.norm1.weight", "perceiver_resampler.layers.0.0.norm1.bias", "perceiver_resampler.layers.0.0.norm2.weight", "perceiver_resampler.layers.0.0.norm2.bias", "perceiver_resampler.layers.0.0.to_q.weight", "perceiver_resampler.layers.0.0.to_kv.weight", "perceiver_resampler.layers.0.0.to_out.weight", "perceiver_resampler.layers.0.1.0.weight", "perceiver_resampler.layers.0.1.0.bias", "perceiver_resampler.layers.0.1.1.weight", "perceiver_resampler.layers.0.1.3.weight", "perceiver_resampler.layers.1.0.norm1.weight", "perceiver_resampler.layers.1.0.norm1.bias", "perceiver_resampler.layers.1.0.norm2.weight", "perceiver_resampler.layers.1.0.norm2.bias", "perceiver_resampler.layers.1.0.to_q.weight", "perceiver_resampler.layers.1.0.to_kv.weight", "perceiver_resampler.layers.1.0.to_out.weight", "perceiver_resampler.layers.1.1.0.weight", "perceiver_resampler.layers.1.1.0.bias", "perceiver_resampler.layers.1.1.1.weight", "perceiver_resampler.layers.1.1.3.weight", "perceiver_resampler.layers.2.0.norm1.weight", "perceiver_resampler.layers.2.0.norm1.bias", "perceiver_resampler.layers.2.0.norm2.weight", "perceiver_resampler.layers.2.0.norm2.bias", "perceiver_resampler.layers.2.0.to_q.weight", "perceiver_resampler.layers.2.0.to_kv.weight", "perceiver_resampler.layers.2.0.to_out.weight", "perceiver_resampler.layers.2.1.0.weight", "perceiver_resampler.layers.2.1.0.bias", "perceiver_resampler.layers.2.1.1.weight", "perceiver_resampler.layers.2.1.3.weight", "perceiver_resampler.layers.3.0.norm1.weight", "perceiver_resampler.layers.3.0.norm1.bias", "perceiver_resampler.layers.3.0.norm2.weight", "perceiver_resampler.layers.3.0.norm2.bias", "perceiver_resampler.layers.3.0.to_q.weight", "perceiver_resampler.layers.3.0.to_kv.weight", "perceiver_resampler.layers.3.0.to_out.weight", "perceiver_resampler.layers.3.1.0.weight", "perceiver_resampler.layers.3.1.0.bias", "perceiver_resampler.layers.3.1.1.weight", "perceiver_resampler.layers.3.1.3.weight". PTAL!!! Thank you! |
|
It's still WIP. I will flip it to ready-to-review once everything is ready. |
|
face_id working now. face_id_plus still WIP. |
|
@sdbds I think the implementation is mostly done. I decide to not including the insightface lib source code here as in previous PR there was some issues with running insightface from source. This PR includes an install script that fetches the pre-build wheel for windows arm64 users, but for all other users, they need to manually install insightface package. There are already sufficient discussions here about insightface installation: cubiq/ComfyUI_IPAdapter_plus#162 |
Thanks for the swift work! I will test it afterward. I noticed earlier when I found the onnx dependency that it seemed like the source code was more of a pain. |
|
Hello. Can you please tell me how to set preprocessor IP-Adapter face id? I have updated sd-webui-controlnet, but I don't have this preprocessor |
|
The PR has not been merged yet. You need to checkout the PR manually to use faceid or wait the PR to be merged. |
|
@sdbds Gentle ping on the review. |
|
where to get preprocessor for faceID? or how to make it preprocessor, not a model if it's just another model there? |
need insightFace |
|
please add FaceId portrait to webui A1111,THX!!!!!!!!!!!!!!!!!!!!!! |
it's already added several days ago: #2530 |
|
Great job,Thx!!!
…------------------ 原始邮件 ------------------
发件人: "Chenlei ***@***.***>;
发送时间: 2024年1月24日(星期三) 晚上9:51
收件人: ***@***.***>;
抄送: ***@***.***>; ***@***.***>;
主题: Re: [Mikubill/sd-webui-controlnet] IP-Adapter face id (PR #2434)
please add FaceId portrait to webui A1111,THX!!!!!!!!!!!!!!!!!!!!!!
it's already added several days ago: #2530
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Installation of insightface
FaceID requires
insightface, you need to install them in your A1111 venv. Check this ComfyUI issue for help.In summary, building insightface from source requies compiling some C++ code, which in principle should be avoided at all cost in this repository. See #1875. The workaround this PR has is downloading the pre-build wheel for the largest user group (Windows amd64 users). Other users will need build from source by running
pip install insightface.Models
The following table shows the combination of checkpoint and preprocessor to use for each FaceID IPAdapter Model. Any Tensor size mismatch you may get it is likely caused by a wrong combination.
It is recommended to load the corresponding LoRA when generating with a weight of 0.5 ~ 0.7.
How to use
Step1: Load LoRA
Step2: Configure ControlNet. Choose preprocessor and model.
Step3: Generate
Face input:
Output:
TODO
Next Steps