-
-
Notifications
You must be signed in to change notification settings - Fork 115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement SoM client/server functionality #543
Comments
@FFFiend thoughts? 🙏 😄 |
I took a look at the Instead of using a gradio app url or HuggingFace space, we would like to create an entry point to the EC2 SoM instance we have available and return a marked screenshot as the output of Bit confused on how a marked screenshot is defined though 😄 |
@FFFiend thanks for your quick response!
Exactly right! We would need to integrate the
No worries! You can see the marked screenshot in the PR description, reproduced here: The original screenshot is on the left, the marked screenshot is on the right. |
awesome, so for client.py I'm envisioning client to work as follows:
The original repo doesn't have any architecture or heavy ML code laid out so I'm guessing the meat n potatoes is within the demo files, but I could be wrong. |
@FFFiend thanks for your patience! Just saw this 😅
Agreed.
This may be unnecessary. https://github.com/microsoft/SoM includes a
Bingo! This should all go in |
Feature request
To support https://github.com/openadaptai/SoM we need to implement a client.py with https://www.gradio.app/docs/client. See:
Motivation
https://github.com/openadaptai/SoM is state-of-the-art for visual understanding, and only runs on Linux / CUDA
Refer to system diagram:
Inference (SoM/SAM) must be done remotely.
We wish to implement:
openadapt/adapters/som/client.py
: modified version of client.py in Add Dockerfile and client.py; deploy to EC2 on AWS via Github Actions microsoft/SoM#19 to support getting marked screenshots during analysis (visualization) and replayopenadapt/adapters/som/server
, which can be a git submodule containing https://github.com/OpenAdaptAI/SoM/The text was updated successfully, but these errors were encountered: