feat: add local_files_only to sentence-transformers embedders

[jantrienes]

Related Issues

None, however a similar issue came up in the past: #1256

Proposed Changes:

Pass local_files_only argument to SentenceTransformer to allow use of embedders in offline environments. When local_files_only=True and the model is available in cache, no outside networking requests are made. When the model is not available in cache, an exception is raised. See offline mode docs of Huggingface.

Note that for proper offline use, passing local_files_only=True is necessary even if the model is available in cache, as there are other places where the flag is used in SentenceTransformer (e.g., here).

How did you test it?

Unit tests. Additionally, following snippet demonstrates the desired behavior:

>>> from haystack.components.embedders import SentenceTransformersDocumentEmbedder
>>> SentenceTransformersDocumentEmbedder(model="all-MiniLM-L12-v2", progress_bar=False, local_files_only=True).warm_up()

[... truncated ...]

OSError: We couldn't connect to 'https://huggingface.co' to load the files, and couldn't find them in the cached files.
Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
>>> SentenceTransformersDocumentEmbedder(model="all-MiniLM-L12-v2", progress_bar=False, local_files_only=False).warm_up()
modules.json: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 349/349 [00:00<00:00, 246kB/s]

Notes for the reviewer

Checklist

• I have read the contributors guidelines and the code of conduct
• I have updated the related issue with new insights and changes
• I added unit tests and updated the docstrings
• I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
• I documented my code
• I ran pre-commit hooks and fixed any issue

[CLAassistant]

All committers have signed the CLA.

[coveralls]

Pull Request Test Coverage Report for Build 15117750305

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

• For more information on this, see Tracking coverage changes with pull request builds.
• To avoid this issue with future PRs, see these Recommended CI Configurations.
• For a quick fix, rebase this PR at GitHub. Your next report should be accurate.

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• 75 unchanged lines in 12 files lost coverage.
• Overall coverage decreased (-0.02%) to 90.425%

---

Files with Coverage Reduction	New Missed Lines	%
components/evaluators/sas_evaluator.py	1	62.3%
components/embedders/sentence_transformers_document_embedder.py	2	96.88%
components/embedders/sentence_transformers_text_embedder.py	2	96.43%
components/generators/utils.py	3	18.75%
core/pipeline/async_pipeline.py	3	67.32%
components/generators/hugging_face_local.py	4	91.76%
components/rankers/sentence_transformers_diversity.py	4	94.84%
components/rankers/transformers_similarity.py	5	91.43%
components/readers/extractive.py	7	95.71%
components/agents/agent.py	14	91.07%

Totals
Change from base Build 15065181550:	-0.02%
Covered Lines:	10955
Relevant Lines:	12115

---

💛 - Coveralls

[anakin87]

Hey @jantrienes, thanks for the PR.

I would say that there is no need to introduce a new parameter since we already provide config_kwargs: "Additional keyword arguments for AutoConfig.from_pretrained when loading the model configuration."

This works as expected and raises an error if the model is not available locally:

from haystack.components.embedders import SentenceTransformersTextEmbedder

text_embedder = SentenceTransformersTextEmbedder(model="sentence-transformers/all-mpnet-base-v2",
                config_kwargs={"local_files_only":True})
text_embedder.warm_up()

However, I agree that this feature is a bit hidden.
So I would propose to transform this PR and just improve the docstrings that describe the config_kwargs parameter: we can explicitly mention this use case.

What do you think?

[jantrienes]

Hi @anakin87, thanks a lot!

We initially used config_kwargs={"local_files_only": True}, but it didn’t fully prevent network access. In our deployment environment (behind a proxy), this led to infinite timeouts. Unfortunately, I can't provide a complete repro.

After closer inspection, we found that other parts of sentence-transformers also perform network I/O. For example, here:
https://github.com/UKPLab/sentence-transformers/blob/dd8a8d7d4d5660e7b2b69e8a1ece8202417c0069/sentence_transformers/SentenceTransformer.py#L1825-L1826

While these paths are guarded by not local_files_only, the check only works if local_files_only is passed directly to the SentenceTransformer init. As far as I can tell, config_kwargs isn't unpacked.

Given that local_files_only is part of the sentence-transformers API, I thought it would be safe to include it explicitly. Alternatively, this might be something to fix upstream, but I’m not familiar enough with HF internals to say for sure.

[anakin87]

Thanks. I'll take a better look then.

[anakin87]

Thanks for your clarification, that is correct.
I agree with this change and only left minor comments on the wording.

[anakin87]

Looks good. Thanks!