Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Integration of VLM embedding model #446

Merged
merged 57 commits into from
Jun 5, 2024
Merged

feat: Integration of VLM embedding model #446

merged 57 commits into from
Jun 5, 2024

Conversation

FUYICC
Copy link
Contributor

@FUYICC FUYICC commented Feb 1, 2024 •
edited by coderabbitai bot

Description

Issue #445

Summary by CodeRabbit

  • New Features
    • Introduced CLIPEmbedding for image and text embedding functionalities.
  • Bug Fixes
    • Improved file encoding handling in license updates.
  • Tests
    • Added tests for the new CLIPEmbedding functionality, covering initialization, embedding processes, and output dimension retrieval.

Wendong-Fan and others added 18 commits November 24, 2023 01:01
@Wendong-Fan
add e5 embedding
d300035
@Wendong-Fan
fix typo in toml file
524bfd4
@Wendong-Fan
allow user to switch embeeding model from SentenceTransformer
0f13021
@Wendong-Fan
Move the import to __init__
9ddc871
@Wendong-Fan
polish docstring
e9c3135
@Wendong-Fan
remove # type: ignore
aeae92d
@Wendong-Fan
change embed_list return type and polish docstring
b431e67
@Wendong-Fan
use Union[List[List[float]], ndarray] instead of List[List[float]] | …
884f190 
…ndarray
@Wendong-Fan
change return of embed_list from ndarray to list
9cce263
@Wendong-Fan
change name from SentenceTransformerEmbedding into SentenceTransforme…
4c7b67c 
…rEncoder
@lightaime
update poetry
939808e
@lightaime
update poetry
e8ce692
@lightaime
update poetry
1bf7320
@lightaime
update poetry
653b381
@Wendong-Fan
remove ndarry and union in embedding base file
93e795e
@Wendong-Fan
Merge branch 'master' into feature/open_source_embedding_model
a50b478
@FUYICC
sentence-transformer
4d5ba2d
@FUYICC
integration of clip embedding and update of license
692a670
@FUYICC FUYICC self-assigned this Feb 1, 2024
@FUYICC FUYICC linked an issue Feb 1, 2024 that may be closed by this pull request
[Feature Request] Multi-modal RAG(Retrieval-Augmented Generation) #445
Closed
4 tasks
@FUYICC FUYICC closed this Feb 4, 2024
@FUYICC FUYICC reopened this Feb 5, 2024
@FUYICC FUYICC marked this pull request as ready for review February 9, 2024 03:11
@FUYICC FUYICC requested a review from Wendong-Fan May 5, 2024 17:14
@Wendong-Fan Wendong-Fan added this to the Sprint 4 milestone May 25, 2024
Wendong-Fan
Wendong-Fan reviewed May 26, 2024
View reviewed changes
Copy link
Member

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution and sorry for the late review, left some comments

camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
camel/embeddings/vlm_embedding.py Outdated Show resolved Hide resolved
pyproject.toml Show resolved Hide resolved
FUYICC and others added 6 commits May 27, 2024 20:57
@FUYICC @Wendong-Fan
Update camel/embeddings/vlm_embedding.py
de718ce 
Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>
@FUYICC @Wendong-Fan
Update camel/embeddings/vlm_embedding.py
6ebf5cd 
Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>
@FUYICC @Wendong-Fan
Update camel/embeddings/vlm_embedding.py
c969597 
Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>
@FUYICC @Wendong-Fan
Update camel/embeddings/vlm_embedding.py
6b2c48e 
Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>
@FUYICC @Wendong-Fan
Update camel/embeddings/vlm_embedding.py
b0cadb0 
Co-authored-by: Wendong-Fan <133094783+Wendong-Fan@users.noreply.github.com>
@FUYICC
one method for **kwargs
e2c7824
@Wendong-Fan Wendong-Fan changed the title Integration of VLM embedding model feat: Integration of VLM embedding model May 28, 2024
@FUYICC FUYICC requested a review from Wendong-Fan June 2, 2024 17:30
Wendong-Fan
Wendong-Fan approved these changes Jun 5, 2024
View reviewed changes
Copy link
Member

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @FUYICC 's contribution! LGTM

@Wendong-Fan Wendong-Fan merged commit 52938a7 into master Jun 5, 2024
6 checks passed
@Wendong-Fan Wendong-Fan deleted the CLIP_model branch June 5, 2024 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@coderabbitai coderabbitai[bot] coderabbitai left review comments

@Wendong-Fan Wendong-Fan Wendong-Fan approved these changes

@lightaime lightaime Awaiting requested review from lightaime

@Appointat Appointat Awaiting requested review from Appointat

@zechengz zechengz Awaiting requested review from zechengz

@dandansamax dandansamax Awaiting requested review from dandansamax

Assignees

@FUYICC FUYICC

Labels
Embeddings lgtm This PR has been approved by a maintainer size:L This PR changes 100-499 lines, ignoring generated files.
Projects
Project Camel
Status: Reviewing
Milestone
Sprint 4
Development

Successfully merging this pull request may close these issues.

[Feature Request] Multi-modal RAG(Retrieval-Augmented Generation)
6 participants
@FUYICC @Wendong-Fan @Appointat @dandansamax @zechengz @lightaime