Skip to content

Feature Improvement: Automatically download and load MetaCLIP checkpoints #29

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Nov 27, 2023
Merged

Feature Improvement: Automatically download and load MetaCLIP checkpoints #29

merged 3 commits into from
Nov 27, 2023

Conversation

@Andy-LZH
Copy link
Contributor

@Andy-LZH Andy-LZH commented Nov 8, 2023 •
edited
Loading

Feature Improvement based on Issue #26

Code modified/added

  • src/open_clip/pretrained.py
  • tests/test.py

Now loading MetaCLIP(ViT-B-32, ViT-B32-quickgelu, ViT-B-16, ViT-L-14) can be done as:

# Load MetaCLIP B32 400M
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32-quickgelu', pretrained='metaclip400m')

# Load MetaCLIP B32 2.5B
model, _, preprocess = open_clip.create_model_and_transforms('ViT-B-32-quickgelu', pretrained='metaclip2_5b')

And thanks for reviewing this PR in advance.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 8, 2023
howardhsu
howardhsu reviewed Nov 27, 2023
View reviewed changes
src/open_clip/pretrained.py Outdated
openai="https://openaipublic.azureedge.net/clip/models/40d365715913c9da98579312b702a82c18be219cc2a73407c4526f58eba950af/ViT-B-32.pt",
laion400m_e31="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e31-d867053b.pt",
laion400m_e32="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e32-46683a32.pt",
metaclip400m="https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_400m.pt",
Copy link
Contributor

@howardhsu howardhsu Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for naming suggestions.

howardhsu
howardhsu reviewed Nov 27, 2023
View reviewed changes
src/open_clip/pretrained.py Outdated
laion2b_e16="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-laion2b_e16-af8dbd0c.pth",
laion400m_e31="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e31-d867053b.pt",
laion400m_e32="https://github.com/mlfoundations/open_clip/releases/download/v0.2-weights/vit_b_32-quickgelu-laion400m_e32-46683a32.pt",
metaclip400m="https://dl.fbaipublicfiles.com/MMPT/metaclip/b32_400m.pt",
Copy link
Contributor

@howardhsu howardhsu Nov 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all MetaCLIP are trained on quickgelu; let's NOT use nn.gelu explicitly.

@howardhsu howardhsu mentioned this pull request Nov 27, 2023
Closed
@howardhsu howardhsu merged commit 9c553d6 into facebookresearch:main Nov 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@howardhsu howardhsu howardhsu approved these changes

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@Andy-LZH @howardhsu @facebook-github-bot