-
Notifications
You must be signed in to change notification settings - Fork 121
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Multimodal Models - CLIP and relatives #29
Comments
Hello 👋 The introduction seems very fine.
It's very CLIP focused so it would be nice to be less specific IMO. I think thanks to CLIP we have many multimodal models this day but maybe keep it brief? Not sure, we can decide on the writing process as well.
This section is nice.
This section is nice, maybe make sure it doesn't overlap with the section where we talk about existing architectures or foundation models.
Maybe keep this brief and explain more at Computer Vision in the Wild section, WDYT? Also pinging @johko
|
Hey, thanks for the great outline @pedrogengo . Here are my thoughts: Introduction CLIP Losses/ self supervised learning Relatives
but those are just some suggestions, feel free to have a look at the transformers docs in the multimodal section: Applications Challenges Hope that helps you :) |
Thought:
Maybe we can divide it into better sections like and add models here.
|
In general a good idea, but the main problem I see with this is that many new models focus on the "foundation" part, so actually most are able to perform many tasks at once by now. I think the most important part here is to cover models that are good representatives for different common architectures or training strategies, so people taking the course get an overview of what is out there. |
Hello!
Inspired by #19 #28, me and my fellow collaborators have also outlined a course curriculum for our section but we would like to have some inputs and feedback from the HF team before we finalize it and start working on it. This is our chosen structure so far.
Introduction
CLIP
Losses/ self supervised learning
Relatives
Practical applications & challenges
References:
@mattmdjaga @froestiago
The text was updated successfully, but these errors were encountered: