Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add trust_remote_code init param to SentenceTransformer embedders #7356

Merged
merged 6 commits into from
Mar 14, 2024

Conversation

vblagoje
Copy link
Member

Why:

This change introduces a new parameter trust_remote_code in various classes of the Sentence Transformers embedder component. This parameter allows users to enable or disable the use of unverified model architectures, providing more flexibility in model selection.

What:

  • The trust_remote_code parameter is added to the __init__ method in _SentenceTransformersEmbeddingBackend, SentenceTransformersDocumentEmbedder, and SentenceTransformersTextEmbedder.
  • The get_embedding_backend method in _SentenceTransformersEmbeddingBackendFactory now accepts trust_remote_code as a parameter.
  • The warm_up method in both SentenceTransformersDocumentEmbedder and SentenceTransformersTextEmbedder now accepts trust_remote_code.
  • Test files are updated to include testing for the new trust_remote_code parameter.

How can it be used:

  • Users can now choose to allow unverified model architectures by setting trust_remote_code to True when initializing the Sentence Transformers embedder component.
  • This feature is useful when working with custom models or scripts not officially verified by HuggingFace.

How did you test it:

  • Unit tests for the __init__ methods in _SentenceTransformersEmbeddingBackend, SentenceTransformersDocumentEmbedder, and SentenceTransformersTextEmbedder ensure that the trust_remote_code parameter is correctly passed and saved during initialization.
  • Unit tests for the get_embedding_backend method in _SentenceTransformersEmbeddingBackendFactory check that the method accepts the trust_remote_code parameter and correctly passes it to the SentenceTransformer constructor.
  • Unit tests for the warm_up method in both SentenceTransformersDocumentEmbedder and SentenceTransformersTextEmbedder ensure that the method accepts the trust_remote_code parameter and correctly passes it to the _SentenceTransformersEmbeddingBackendFactory.get_embedding_backend method.

Notes for the reviewer:

  • This change affects various classes of the Sentence Transformers embedder component, so please ensure that all changes are consistent and correct.
  • The trust_remote_code parameter is specifically designed to handle custom models or scripts not officially verified by HuggingFace. If you find any issues with this feature or have suggestions for improvement, please let us know.

@vblagoje vblagoje requested review from a team as code owners March 13, 2024 19:12
@vblagoje vblagoje requested review from dfokina, julian-risch and anakin87 and removed request for a team and julian-risch March 13, 2024 19:12
@github-actions github-actions bot added topic:tests 2.x Related to Haystack v2.0 type:documentation Improvements on the docs labels Mar 13, 2024
@vblagoje
Copy link
Member Author

@anakin87 verified in an actual example we talked about 🚀 Let's see what CI says
@julian-risch I removed you as a reviewer and asked Stefano to review because he has all the context already and knows this code well

@coveralls
Copy link
Collaborator

Pull Request Test Coverage Report for Build 8270286697

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • 4 unchanged lines in 2 files lost coverage.
  • Overall coverage increased (+0.003%) to 89.164%

Files with Coverage Reduction New Missed Lines %
components/embedders/sentence_transformers_document_embedder.py 2 95.65%
components/embedders/sentence_transformers_text_embedder.py 2 94.74%
Totals Coverage Status
Change from base Build 8268196727: 0.003%
Covered Lines: 5324
Relevant Lines: 5971

💛 - Coveralls

vblagoje and others added 2 commits March 14, 2024 09:29
…mbedder.py

Co-authored-by: Stefano Fiorucci <stefanofiorucci@gmail.com>
@vblagoje
Copy link
Member Author

@anakin87 that should be it - I think, have another look please

Copy link
Member

@anakin87 anakin87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a comment.
Apart from that, this PR is ready to go!

Comment on lines 63 to 65
:param trust_remote_code:
If trust_remote_code is false only HuggingFace verified model architectures are allowed. If true
then custom models and scripts are allowed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also update this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done with 2bad887

@vblagoje vblagoje merged commit 2aae847 into main Mar 14, 2024
23 checks passed
@vblagoje vblagoje deleted the embedders_trusted branch March 14, 2024 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.x Related to Haystack v2.0 topic:tests type:documentation Improvements on the docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants