Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Great work!and some questions. #1

Closed
sdbds opened this issue Jul 7, 2024 · 2 comments
Closed

Great work!and some questions. #1

sdbds opened this issue Jul 7, 2024 · 2 comments

Comments

@sdbds
Copy link

sdbds commented Jul 7, 2024

Thank you for the open-source contribution! If necessary, I can help with the implementation of the web UI part of the code.
I have a few questions:

  1. It seems that the TEED test script is missing.
  2. Segment uses dhkim2810/MobileSAM, and it appears there is no corresponding semantic table (such as ADE20K) but rather random segmentation.
  3. Is it possible to input multiple different preprocessor images simultaneously like Anycontrol?
  4. Some preprocessors do not seem to use SOTA models, especially normalbae, which has a significant gap compared to Dsine.
@xinsir6
Copy link
Owner

xinsir6 commented Jul 8, 2024

Sure, I can append the teed test scripts later.
1 The teed example I displayed is detected in default resolution. The teed algorithm have a characteristic that the higher detect resolution the thiner line. The model propose 2 line model, thick and thin, it can cover the situation.
2 The reason segment use dhkim2810/MobileSAM is that the model trained on ADE20K performs not well in huge amount of data. I do experiment and find SAM series model can achieve better performance. Use mobile is about the speed consideration(when data is huge).
3 Yes, it is. In face, this model is trained in this way. The architecture is designed in lots of experiment and proves to be effective.
4 for convenience, we use the controlnet_aux package method. Other SOTA methods is absolutely OK and can improve the performance.

@sdbds
Copy link
Author

sdbds commented Jul 8, 2024

Sure, I can append the teed test scripts later. 1 The teed example I displayed is detected in default resolution. The teed algorithm have a characteristic that the higher detect resolution the thiner line. The model propose 2 line model, thick and thin, it can cover the situation. 2 The reason segment use dhkim2810/MobileSAM is that the model trained on ADE20K performs not well in huge amount of data. I do experiment and find SAM series model can achieve better performance. Use mobile is about the speed consideration(when data is huge). 3 Yes, it is. In face, this model is trained in this way. The architecture is designed in lots of experiment and proves to be effective. 4 for convenience, we use the controlnet_aux package method. Other SOTA methods is absolutely OK and can improve the performance.

Thank you for your answer! I just noticed that the controlnet_aux package is now hosted by Hugging Face official , and I also found the preprocessors for mobile_sam and teed there. I've understood how to choose models.

Currently, the most comprehensive SOTA preprocessors for controlnet are mainly located on comfyui's node, maintained by Fannovel16.
https://github.com/Fannovel16/comfyui_controlnet_aux

If you're looking to try other types, you might want to look this.

@sdbds sdbds closed this as completed Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants