Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: add wise-ft to clip finetuning #571

Merged
merged 7 commits into from
Oct 10, 2022
Merged

docs: add wise-ft to clip finetuning #571

merged 7 commits into from
Oct 10, 2022

Conversation

bwanglzu
Copy link
Member

@bwanglzu bwanglzu commented Oct 9, 2022

In this PR, we start to support WiSE-FT fine-tuning for models with a strong zero-shot capability, to merge weights between pre-trained model and zero-shot model. Currently all CLIP-family models are supported. Functionality has been implemented inside core. To make use of it:

import finetuner

run = finetuner.fit(
     model='ViT-B-32#openai',
     ...,
     loss = 'CLIPLoss',
-    callbacks=[],
+    callbacks=[WiSEFTCallback(alpha=0.4)],
)

Review documentation change here


  • This PR references an open issue
  • I have added a line about this change to CHANGELOG

@bwanglzu bwanglzu changed the title docs: add wise-ft in clip docs: add wise-ft to clip finetuning Oct 9, 2022
@bwanglzu bwanglzu marked this pull request as ready for review October 10, 2022 06:50
docs/tasks/text-to-image.md Outdated Show resolved Hide resolved
docs/tasks/text-to-image.md Outdated Show resolved Hide resolved
docs/tasks/text-to-image.md Outdated Show resolved Hide resolved
docs/tasks/text-to-image.md Outdated Show resolved Hide resolved
docs/tasks/text-to-image.md Outdated Show resolved Hide resolved
docs/tasks/text-to-image.md Outdated Show resolved Hide resolved
docs/walkthrough/using-callbacks.md Outdated Show resolved Hide resolved
@github-actions
Copy link

📝 Docs are deployed on https://ft-docs-wise-ft--jina-docs.netlify.app 🎉

@bwanglzu bwanglzu merged commit 27ad740 into main Oct 10, 2022
@bwanglzu bwanglzu deleted the docs-wise-ft branch October 10, 2022 15:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants