Replies: 5 comments 16 replies
-
Note: According to #1864 (comment), you need to install Microsoft Visual C++ (https://visualstudio.microsoft.com/visual-cpp-build-tools/) for some dependencies to build correctly. |
Beta Was this translation helpful? Give feedback.
-
Due to #1875, the feature cannot be merged to main yet. I uploaded the version before reset to |
Beta Was this translation helpful? Give feedback.
-
@huchenlei SO as you said before the dw pth model was not intended to be used for auto1111. so i guess just git checkout would needed for this purpose, right? ( or any other better ways recommanded?) Thanks |
Beta Was this translation helpful? Give feedback.
-
If I understand correctly, dw_openpose_full is the same as openpose_full? If not, how can I set up the dw_openpose_full? |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
See initial issue here: #1855
DW Openpose preprocessor greatly improves the accuracy of openpose detection especially on hands. Here is a comparison used in our unittest:
Input Image:
![woman](https://private-user-images.githubusercontent.com/20929282/258517170-918b8d6d-4e05-4609-9a1b-34a759edb7cd.jpeg?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM2NjgzNjksIm5iZiI6MTcyMzY2ODA2OSwicGF0aCI6Ii8yMDkyOTI4Mi8yNTg1MTcxNzAtOTE4YjhkNmQtNGUwNS00NjA5LTlhMWItMzRhNzU5ZWRiN2NkLmpwZWc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODE0JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgxNFQyMDQxMDlaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1jYTEzZWVkZjYzYzM0YTNlZTM3MWVmMjY1ZmY1Y2I4NjUzMGVkNjRlZDE5NmQ1N2Y0MGY2NWQyMGMyZjcxZmQxJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.kGXRTos-RZpYL_kU-tCcKBBdCTk-I27RfIEzWKAuYsU)
Openpose Full Output (Right hand missing):
![expected_woman_all_output](https://private-user-images.githubusercontent.com/20929282/258517224-eb781c40-c46b-499e-b571-d7b84008943e.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM2NjgzNjksIm5iZiI6MTcyMzY2ODA2OSwicGF0aCI6Ii8yMDkyOTI4Mi8yNTg1MTcyMjQtZWI3ODFjNDAtYzQ2Yi00OTllLWI1NzEtZDdiODQwMDg5NDNlLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE0VDIwNDEwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWZhNDRiNjJhNDdhMjM3YzQzOTE4MTRhN2FkMTJhMWEyMGVlMjlhMjc0MDNjZGRiYTA5ZmFjMjA3MzQyYTc0YmUmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.DTn9UbWAxNBfxIhiICsXRsSDHx7il0Hzl_5EXA4EgNI)
DW Openpose Full Output:
![expected_woman_dw_all_output](https://private-user-images.githubusercontent.com/20929282/258517279-1ac12389-f0a8-4515-a13f-eb58afb176fa.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM2NjgzNjksIm5iZiI6MTcyMzY2ODA2OSwicGF0aCI6Ii8yMDkyOTI4Mi8yNTg1MTcyNzktMWFjMTIzODktZjBhOC00NTE1LWExM2YtZWI1OGFmYjE3NmZhLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE0VDIwNDEwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPWJmMGMyYmYyMDJjZjM5MmNlOWQwYzBkZTQwNWVhOWM5ZjNiYTYxNzIzNGQyNDU1YzVkNjEyNGEzY2Q1ZTc1M2EmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.-YAJuj0fC0eK9vQj-POeT1Z1RJXMDQc0sSHexH7HT3o)
Usage:
![dw_pose](https://private-user-images.githubusercontent.com/20929282/258520905-58c5203d-46b6-42e1-abd9-16c334e07252.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM2NjgzNjksIm5iZiI6MTcyMzY2ODA2OSwicGF0aCI6Ii8yMDkyOTI4Mi8yNTg1MjA5MDUtNThjNTIwM2QtNDZiNi00MmUxLWFiZDktMTZjMzM0ZTA3MjUyLnBuZz9YLUFtei1BbGdvcml0aG09QVdTNC1ITUFDLVNIQTI1NiZYLUFtei1DcmVkZW50aWFsPUFLSUFWQ09EWUxTQTUzUFFLNFpBJTJGMjAyNDA4MTQlMkZ1cy1lYXN0LTElMkZzMyUyRmF3czRfcmVxdWVzdCZYLUFtei1EYXRlPTIwMjQwODE0VDIwNDEwOVomWC1BbXotRXhwaXJlcz0zMDAmWC1BbXotU2lnbmF0dXJlPTg3OTE0ODI2YjMzYjNkZjAyYzRmZWZhZDU1YmMwNDQyMmUwZDY2MTAyODI0MTdlYzg2ZDE1ZTBhZmE5MDQ5NGEmWC1BbXotU2lnbmVkSGVhZGVycz1ob3N0JmFjdG9yX2lkPTAma2V5X2lkPTAmcmVwb19pZD0wIn0.EfOaAidpXDsyXBVuP7dr7paoE9NTTou0FfaDlaMz-mI)
With this pose detection accuracy improvements, we are hyped to start re-train the ControlNet openpose model with more accurate annotations. There is a proposal in DW Pose repository: IDEA-Research/DWPose#2.
I would really want @lllyasviel to take the initiative for this retraining task, but he probably busy with other tasks. I am not familiar with ControlNet training, so I welcome suggestions and ideas on how should we proceed on this task with community support. You can find me on my discord server: https://discord.gg/GkaWcUat7R. Or just leave a comment here.
Beta Was this translation helpful? Give feedback.
All reactions