Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support B/32, L/14, H/14, and g/14 trained on LAION-2B #825

Merged
merged 6 commits into from
Sep 23, 2022

Conversation

ZiniuYu
Copy link
Member

@ZiniuYu ZiniuYu commented Sep 20, 2022

This pr supports 4 new clip models introduced by https://laion.ai/blog/large-openclip/

  • ViT-B-32::laion2B-s34B-b79K (PyTorch+ONNX)
  • ViT-L-14::laion2B-s32B-b82K (PyTorch+ONNX)
  • ViT-H-14::laion2B-s32B-b79K (PyTorch)
  • ViT-g-14::laion2B-s12B-b42K (PyTorch)

It also sets timeout to 30 min in torch and onnx runtime since we have larger model files now

@codecov
Copy link

codecov bot commented Sep 20, 2022

Codecov Report

Merging #825 (3835f75) into main (c690c24) will decrease coverage by 2.09%.
The diff coverage is 62.50%.

@@            Coverage Diff             @@
##             main     #825      +/-   ##
==========================================
- Coverage   84.67%   82.57%   -2.10%     
==========================================
  Files          21       21              
  Lines        1566     1567       +1     
==========================================
- Hits         1326     1294      -32     
- Misses        240      273      +33     
Flag Coverage Δ
cas 82.57% <62.50%> (-2.10%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
server/clip_server/model/clip_onnx.py 88.88% <ø> (ø)
server/clip_server/model/pretrained_models.py 98.41% <ø> (ø)
server/clip_server/model/model.py 78.97% <62.50%> (+0.04%) ⬆️
server/clip_server/model/trt_utils.py 56.04% <0.00%> (-27.48%) ⬇️
server/clip_server/model/clip_trt.py 69.38% <0.00%> (-16.33%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@github-actions
Copy link

📝 Docs are deployed on https://ft-large-model--jina-docs.netlify.app 🎉

@ZiniuYu ZiniuYu marked this pull request as ready for review September 21, 2022 09:37
@ZiniuYu
Copy link
Member Author

ZiniuYu commented Sep 21, 2022

server/clip_server/model/model.py

This file is synced with https://github.com/mlfoundations/open_clip/blob/main/src/open_clip/model.py

@ZiniuYu ZiniuYu requested a review from a team September 22, 2022 02:35
@numb3r3
Copy link
Member

numb3r3 commented Sep 23, 2022

@ZiniuYu Please polish the PR's title. 4 new clip models is meaningless. Please highlights the contribution in the title

@ZiniuYu ZiniuYu changed the title feat: support 4 new clip models pretrained on laion2b feat: support B/32, L/14, H/14, and g/14 trained on LAION-2B. Sep 23, 2022
@ZiniuYu ZiniuYu changed the title feat: support B/32, L/14, H/14, and g/14 trained on LAION-2B. feat: support B/32, L/14, H/14, and g/14 trained on LAION-2B Sep 23, 2022
Copy link
Member

@numb3r3 numb3r3 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@numb3r3 numb3r3 merged commit 09d1548 into main Sep 23, 2022
@numb3r3 numb3r3 deleted the large-model branch September 23, 2022 07:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants