Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for New Auto-Tagging/Captioning Model/s Available #16

Open
x-CK-x opened this issue Dec 30, 2023 · 15 comments
Open

Support for New Auto-Tagging/Captioning Model/s Available #16

x-CK-x opened this issue Dec 30, 2023 · 15 comments
Assignees
Labels
enhancement New feature or request help wanted Extra attention is needed High Priority TICKET_ACTIVE

Comments

@x-CK-x
Copy link
Owner

x-CK-x commented Dec 30, 2023

Several new model/s are available; The following must be done:

  • add them to the download tab
  • create a config for each model
  • create a generic handler for all the auto-tag/caption model/s
  • GPU support (optional)
  • protobuffer (.pb) to .onnx conversion to use the (onnx-runtime library)
@x-CK-x x-CK-x self-assigned this Dec 30, 2023
@x-CK-x x-CK-x added enhancement New feature or request help wanted Extra attention is needed High Priority labels Dec 30, 2023
@x-CK-x
Copy link
Owner Author

x-CK-x commented Dec 30, 2023

  • CogVLM might need a "newer" interface to work successfully

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

the new models: https://huggingface.co/Thouph

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

I need to add to the configs, information pertaining to the file type:
image

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

https://huggingface.co/Thouph/eva02-clip-vit-large-7704/tree/main with:

https://huggingface.co/Thouph/experimental-eva02-vit-large-448-adventurous/tree/main with:

https://huggingface.co/Thouph/eva02-vit-large-448-8046/tree/main with:

https://huggingface.co/Thouph/experimental_efficientnetv2_m_8038/tree/main with:

https://huggingface.co/Thouph/experimental_efficientnetv2_m_8035/tree/main with:

(Additional ViT Model/s potentially available soon as well)

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

Working on a possible way to automatically install CUDA in the same manner that it is done with auto1111.
The purpose of which would be to provide users the option of using their GPU.

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

6306c1b

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

5adc7aa

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

8cb8d12

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

Current TODO:

Create a generic handler for all the auto-tag/caption model/s using the configs just created.

GPU support: detect if GPU is available when user tries to enable GPU mode, if no gpu is present; then set mode back to cpu and display to user.

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

#32

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

33ab23a

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

50255a6

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 5, 2024

I need to make additional edits to the naming conventions of zack's model & tags file to reflect the consistencies of the new config file

@x-CK-x
Copy link
Owner Author

x-CK-x commented Jan 7, 2024

code refactored to the following:

                                   ------- Model_Manager
                                 /
                               /
                             /
GUI ------- Auto_Tag_Manager -------- Dataset_Manager
                    |        \
                    |          \
                    |            \
                    |              ------- Custom_File_Manager
                    |              
                    |              
                    |              
                     -------- Meta_Data_Extractor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed High Priority TICKET_ACTIVE
Projects
None yet
Development

No branches or pull requests

1 participant