Offline mode

[JosephMarinier]

Hello! 👋

Would it be possible to have an offline mode similar to Hugginface's TRANSFORMERS_OFFLINE=1 environment variable documented here?

Example

For example, if I call SentenceTransformer(model_name), it downloads to cache. If I then turn off my Internet connection and re-run, I get something like requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /api/models/sentence-transformers/all-MiniLM-L12-v2 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x176176130>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known')).

I would like to tell sentence-transformers to go straight to the cache.

Workaround

I can do the following workaround:

SentenceTransformer(
    f"/Users/joseph/.cache/torch/sentence_transformers/sentence-transformers_{model_name}"
    if os.environ.get("TRANSFORMERS_OFFLINE") else model_name
)

But with SENTENCE_TRANSFORMERS_HOME, TORCH_HOME and XDG_CACHE_HOME in mind, it would be a lot more robust to support that inside SentenceTransformer().

[danielzgtg]

This is even more annoying when using sentence-transformers through a third party library such as KeyBERT. It's because now we need to find where the SentenceTransformer is created and override it. In this case it was in the README, but other libraries might not provide an override option.

[tomaarsen]

Hello!

As of #2345, running this script twice, first with internet and then without, will now result in the same output of (384,) both times. With other words, the cache is respected/used if the upstream model cannot be accessed.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
embeddings = model.encode("This is a test sentence")
print(embeddings.shape)

So I'll be closing this. I intend to include these changes in the upcoming 2.3.0 release, which should hopefully release within the next week or 2.

• Tom Aarsen