Add HF_HUB_OFFLINE env var #22
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
CI error looks transient |
julien-c
reviewed
Mar 18, 2021
|
|
||
| def test_offline_with_timeout(): | ||
| with offline(OfflineSimulationMode.CONNECTION_TIMES_OUT): | ||
| with pytest.raises(RequestWouldHangIndefinitelyError): |
There was a problem hiding this comment.
maybe use with self.assertRaises to be consistent with other tests here @lhoestq (not a big deal if we want to stay consistent with other implems in our librairies)
julien-c
reviewed
Mar 18, 2021
| timeout: float = 10.0, | ||
| **params, | ||
| ) -> requests.Response: | ||
| """Wrapper around requests to retry in case it fails with a ConnectTimeout, with exponential backoff. |
julien-c
reviewed
Mar 18, 2021
| def _request_with_retry( | ||
| method: str, | ||
| url: str, | ||
| max_retries: int = 0, |
There was a problem hiding this comment.
default should maybe be finite no? like 5?
julien-c
reviewed
Mar 18, 2021
src/huggingface_hub/file_download.py
Outdated
| if isinstance(exc, requests.exceptions.SSLError) or isinstance( | ||
| exc, requests.exceptions.ProxyError | ||
| ): | ||
| if isinstance(exc, requests.exceptions.ProxyError): |
There was a problem hiding this comment.
was this intended to remove this?
There was a problem hiding this comment.
Yes otherwise the test with the bogus url test fails (raises SSLError while ValueError is expected)
julien-c
approved these changes
Mar 18, 2021
|
also related to huggingface/transformers#10235 @lhoestq |
n1t0
reviewed
Mar 18, 2021
| logger.info( | ||
| f"{method} request to {url} timed out, retrying... [{tries/max_retries}]" | ||
| ) | ||
| sleep_time = max( |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds the
HF_HUB_OFFLINEenv var to tellhuggingface_hub(same as in hf/transformers and hf/datasets).This allows to tell the library to look directly at the cache instead of trying network calls.
Moreover I fixed/changed a few things:
force_download=Truedoesn't check the cache if there's no internet connection, but raises an error instead.offlinethat you can use to test the library's behavior under different offline modes (same as in hf/datasets)I'm not sure why the CI fails for python 3.9 though