Skip to content

Commit

Permalink
Add variable FSSPEC_CLIENT_KWARGS for specification of fsspec request…
Browse files Browse the repository at this point in the history
… kwargs
  • Loading branch information
gutzbenj committed Sep 30, 2021
1 parent bf02609 commit 78ec769
Show file tree
Hide file tree
Showing 6 changed files with 247 additions and 225 deletions.
30 changes: 15 additions & 15 deletions THIRD_PARTY_NOTICES
Expand Up @@ -137,7 +137,7 @@ http://www.gdal.org
UNKNOWN

GitPython
3.1.23
3.1.24
BSD License
Sebastian Thiel, Michael Trier
https://github.com/gitpython-developers/GitPython
Expand Down Expand Up @@ -1928,7 +1928,7 @@ POSSIBILITY OF SUCH DAMAGE.


anyio
3.3.1
3.3.2
MIT License
Alex Grönholm
UNKNOWN
Expand Down Expand Up @@ -2644,7 +2644,7 @@ limitations under the License.


cachetools
4.2.2
4.2.4
MIT License
Thomas Kemmer
https://github.com/tkem/cachetools/
Expand Down Expand Up @@ -3257,7 +3257,7 @@ That's all there is to it!


charset-normalizer
2.0.5
2.0.6
MIT License
Ahmed TAHRI @Ousret
https://github.com/ousret/charset_normalizer
Expand Down Expand Up @@ -6395,7 +6395,7 @@ THE SOFTWARE.


jsonschema
3.2.0
4.0.1
MIT License
Julian Berman
https://github.com/Julian/jsonschema
Expand All @@ -6421,7 +6421,7 @@ THE SOFTWARE.


jupyter-client
7.0.2
7.0.5
BSD License
Jupyter Development Team
https://jupyter.org
Expand Down Expand Up @@ -6488,7 +6488,7 @@ to indicate the copyright and license terms:


jupyter-core
4.7.1
4.8.1
BSD License
Jupyter Development Team
https://jupyter.org
Expand Down Expand Up @@ -10343,7 +10343,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


pytest-xdist
2.3.0
2.4.0
MIT License
holger krekel and contributors
https://github.com/pytest-dev/pytest-xdist
Expand Down Expand Up @@ -10557,7 +10557,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.


rapidfuzz
1.6.2
1.7.0
MIT License
Max Bachmann
https://github.com/maxbachmann/rapidfuzz
Expand All @@ -10584,7 +10584,7 @@ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

regex
2021.8.28
2021.9.24
Apache Software License
Matthew Barnett
https://bitbucket.org/mrabarnett/mrab-regex
Expand Down Expand Up @@ -12824,7 +12824,7 @@ https://github.com/jupyter/terminado


testfixtures
6.18.1
6.18.3
MIT License
Chris Withers
https://github.com/Simplistix/testfixtures
Expand Down Expand Up @@ -13334,7 +13334,7 @@ http://www.tornadoweb.org/


tqdm
4.62.2
4.62.3
MIT License; Mozilla Public License 2.0 (MPL 2.0)
UNKNOWN
https://tqdm.github.io
Expand Down Expand Up @@ -13362,7 +13362,7 @@ Mozilla Public Licence (MPL) v. 2.0 - Exhibit A

This Source Code Form is subject to the terms of the
Mozilla Public License, v. 2.0.
If a copy of the MPL was not distributed with this file,
If a copy of the MPL was not distributed with this project,
You can obtain one at https://mozilla.org/MPL/2.0/.


Expand Down Expand Up @@ -14043,7 +14043,7 @@ SOFTWARE.


urllib3
1.26.6
1.26.7
MIT License
Andrey Petrov
https://urllib3.readthedocs.io/
Expand Down Expand Up @@ -14818,7 +14818,7 @@ https://github.com/aio-libs/yarl/


zipp
3.5.0
3.6.0
MIT License
Jason R. Coombs
https://github.com/jaraco/zipp
Expand Down
19 changes: 16 additions & 3 deletions docs/usage/python-api.rst
Expand Up @@ -524,9 +524,22 @@ For more examples, please have a look at `example/radar/`_.
Caching
=======

The backbone of wetterdienst uses dogpile caching. It requires to create a directory under ``/home`` for the most cases.
If you are not allowed to write into ``/home`` you will run into ``OSError``. For this purpose you can set an environment variable
``WD_CACHE_DIR`` to define the place where the caching directory should be created.
The backbone of wetterdienst uses dogpile + fsspec caching. It requires to create a directory under ``/home`` for the
most cases. If you are not allowed to write into ``/home`` you will run into ``OSError``. For this purpose you can set
an environment variable ``WD_CACHE_DIR`` to define the place where the caching directory should be created.

FSSPEC is used for flexible file caching. It relies on the two libraries requests and aiohttp. Aiohttp is used for
asynchronous requests and may swallow some errors related to proxies, ssl or similar. Use the defined variable
FSSPEC_CLIENT_KWARGS to pass your very own client kwargs to fsspec e.g.

.. ipython:: python
from wetterdienst.util.cache import FSSPEC_CLIENT_KWARGS
FSSPEC_CLIENT_KWARGS = {
"proxy": "www.example.com"
}
.. _wradlib: https://wradlib.org/
.. _example/radar/: https://github.com/earthobservations/wetterdienst/tree/main/example/radar
Expand Down

0 comments on commit 78ec769

Please sign in to comment.