Skip to content
This repository has been archived by the owner on Oct 9, 2023. It is now read-only.

Add CLIP backbones for text / image classification #1458

Merged
merged 16 commits into from
Oct 1, 2022
Merged

Conversation

ethanwharris
Copy link
Collaborator

@ethanwharris ethanwharris commented Sep 22, 2022

What does this PR do?

  • Add CLIP backbone for image and text classification
  • Refactor text classifier to use adapters

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure your PR does only one thing, instead of bundling different changes together?
  • Did you make sure to update the documentation with your changes?
  • Did you write any new necessary tests? [not needed for typos/docs]
  • Did you verify new and existing tests pass locally with your changes?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

PR review

  • Is this pull request ready for review? (if not, please submit in draft mode)

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

@codecov
Copy link

codecov bot commented Sep 22, 2022

Codecov Report

Base: 92.29% // Head: 92.33% // Increases project coverage by +0.04% 🎉

Coverage data is based on head (5e36793) compared to base (dc30097).
Patch coverage: 95.93% of modified lines in pull request are covered.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1458      +/-   ##
==========================================
+ Coverage   92.29%   92.33%   +0.04%     
==========================================
  Files         287      291       +4     
  Lines       13061    13197     +136     
==========================================
+ Hits        12054    12185     +131     
- Misses       1007     1012       +5     
Flag Coverage Δ
pytest 13.16% <0.58%> (-0.13%) ⬇️
tpu 13.16% <0.58%> (-0.13%) ⬇️
unittests 92.86% <95.93%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
flash/image/classification/adapters.py 82.53% <ø> (ø)
flash/image/segmentation/model.py 94.31% <0.00%> (ø)
flash/core/utilities/embedder.py 88.88% <85.71%> (-0.70%) ⬇️
flash/text/classification/adapters.py 94.94% <94.94%> (ø)
flash/core/registry.py 96.20% <100.00%> (+0.07%) ⬆️
flash/core/utilities/providers.py 100.00% <100.00%> (ø)
flash/image/classification/backbones/__init__.py 100.00% <100.00%> (ø)
flash/image/classification/backbones/clip.py 100.00% <100.00%> (ø)
flash/text/classification/backbones/__init__.py 100.00% <100.00%> (ø)
flash/text/classification/backbones/clip.py 100.00% <100.00%> (ø)
... and 3 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@ethanwharris ethanwharris changed the title [WIP] Add CLIP backbones for text / image classification Add CLIP backbones for text / image classification Sep 30, 2022
@ethanwharris ethanwharris added this to the 0.9.0 milestone Sep 30, 2022
@ethanwharris ethanwharris added the enhancement New feature or request label Sep 30, 2022
@ethanwharris ethanwharris marked this pull request as ready for review September 30, 2022 18:22
Copy link
Contributor

@kaushikb11 kaushikb11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Ethan!

@ethanwharris ethanwharris enabled auto-merge (squash) October 1, 2022 09:51
@ethanwharris ethanwharris merged commit eb11c35 into master Oct 1, 2022
@ethanwharris ethanwharris deleted the feature/clip branch October 1, 2022 09:58
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants