Skip to content

Commit

Permalink
Another round of updates to the documentation (earthobservations#153)
Browse files Browse the repository at this point in the history
  • Loading branch information
amotl committed Sep 15, 2020
1 parent fea8648 commit 7c41359
Show file tree
Hide file tree
Showing 23 changed files with 487 additions and 331 deletions.
64 changes: 29 additions & 35 deletions README.rst
@@ -1,6 +1,6 @@
#################################################################
Wetterdienst - Python library to ease access to open weather data
#################################################################
###########################################
Wetterdienst - Open weather data for humans
###########################################

.. image:: https://github.com/earthobservations/wetterdienst/workflows/Tests/badge.svg
:target: https://github.com/earthobservations/wetterdienst/actions?workflow=Tests
Expand All @@ -27,14 +27,23 @@ Wetterdienst - Python library to ease access to open weather data
:target: https://zenodo.org/badge/latestdoi/160953150


Welcome to Wetterdienst, your friendly weather service library for Python from the
neighbourhood! We are a group of people who try to make access to weather data in
Introduction
************
Welcome to Wetterdienst, your friendly weather service library for Python.

We are a group of like-minded people trying to make access to weather data in
Python feel like a warm summer breeze, similar to other projects like
`rdwd <https://github.com/brry/rdwd>`_
for the R language, which originally drew our interest in this project.
rdwd_ for the R language, which originally drew our interest in this project.

While our long-term goal is to provide access to multiple weather services,
we are still stuck with the German Weather Service (DWD). Contributions are
always welcome!

This program and its repository tries to use modern Python technologies
all over the place. The library is based on Pandas across the board,
uses Poetry for package administration and GitHub actions for
all things CI.

While our long-term goal is to provide you with data from multiple weather services,
we are still stuck with the German Weather Service (DWD).

Features
********
Expand All @@ -50,7 +59,6 @@ The library currently covers
To get better insight on which data we have currently made available, with this library
take a look at `data coverage`_.

.. _data coverage: https://wetterdienst.readthedocs.io/en/latest/pages/data_coverage.html

Details
=======
Expand Down Expand Up @@ -115,31 +123,17 @@ documentation, which will be constantly updated. To stay up to date with the
development, take a look at the changelog_. Also, don't miss out our examples_.


.. _Wetterdienst API: https://wetterdienst.readthedocs.io/en/latest/pages/api.html
.. _changelog: https://wetterdienst.readthedocs.io/en/latest/pages/api.html
.. _examples: https://github.com/earthobservations/wetterdienst/tree/master/example


Contribution
************
Check out our contribution section in the documentation! For a successful PR passing
all tests, you have to run

.. code-block:: bash
nox -s tests
nox -s black
nox -s lint
before committing. This will inform you in case of problems with tests and your code
format.


Data license
************
**CAUTION**
Although the data is specified as being open, the DWD asks you to reference them as
Copyright owner. To check out further, take a look at the
`Open Data Strategy at the DWD <https://www.dwd.de/EN/ourservices/opendata/opendata.html>`_
and the
`Official Copyright <https://www.dwd.de/EN/service/copyright/copyright_artikel.html?nn=495490&lsbId=627548>`_.
copyright owner. Please take a look at the `Open Data Strategy at the DWD`_ and the
`Official Copyright`_ statements before using the data.


.. _rdwd: https://github.com/brry/rdwd>
.. _Wetterdienst API: https://wetterdienst.readthedocs.io/en/latest/pages/api.html
.. _data coverage: https://wetterdienst.readthedocs.io/en/latest/pages/data_coverage.html
.. _changelog: https://wetterdienst.readthedocs.io/en/latest/pages/api.html
.. _examples: https://github.com/earthobservations/wetterdienst/tree/master/example
.. _Open Data Strategy at the DWD: https://www.dwd.de/EN/ourservices/opendata/opendata.html
.. _Official Copyright: https://www.dwd.de/EN/service/copyright/copyright_artikel.html?nn=495490&lsbId=627548
26 changes: 13 additions & 13 deletions docs/index.rst
Expand Up @@ -3,30 +3,30 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
#################################################################
Wetterdienst - Python library to ease access to open weather data
#################################################################
###########################################
Wetterdienst - Open weather data for humans
###########################################

*****
About
*****
.. toctree::
:maxdepth: 1

README <pages/welcome>
Introduction <pages/welcome>
pages/installation
pages/data_coverage
pages/api
API overview <pages/api>
pages/cli
pages/license_and_citation

********
Plumbing
********
.. toctree::
:maxdepth: 1

pages/library/index
pages/behind_the_scenes
pages/development
pages/changelog
pages/license_and_citation

Indices and tables
##################

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
150 changes: 61 additions & 89 deletions docs/pages/api.rst
@@ -1,26 +1,47 @@
###
API
###
The API is divided amongst the data products as written in the :ref:`data-coverage` chapter.
The API offers access to different data products. They are
outlined in more detail within the :ref:`data-coverage` chapter.

***********************
Historical Weather Data
***********************
The API for the historical weather data mainly consists of the following functions:
.. contents::
:local:
:depth: 1

``discover_climate_observations``
=================================
- Print out available time resolution, parameter, period type combinations and
subsets of it depending on the entered arguments.
************
Observations
************
Acquire historical weather data through requesting by
*parameter*, *time resolution* and *period type*.

``metadata_for_climate_observations``
=====================================
- Discover what data for a set of parameters (parameter, time_resolution,
period_type) is available, especially which stations can be found.
- With **create_new_file_index**, the function can be forced to retrieve a new list
of files from the server, which is usually avoided as it rarely changes.
Request arguments
=================
The options *parameter*, *time resolution* and *period type* can be used in three ways:

Let's get station information for a given parameter, time resolution and period type:
- by using the exact enumeration e.g.
.. code-block:: python
Parameter.CLIMATE_SUMMARY
- by using the enumeration string e.g.
.. code-block:: python
"climate_summary" or "CLIMATE_SUMMARY"
- by using the originally defined parameter string e.g.
.. code-block:: python
"kl"
Use ``wetterdienst.discover_climate_observations()`` to discover available
time resolution, parameter, period type combinations and their subsets
based on the obtained filter arguments.


Station list
============
Get station information for a given set of *parameter*, *time resolution*
and *period type* options.

.. code-block:: python
Expand All @@ -33,19 +54,22 @@ Let's get station information for a given parameter, time resolution and period
period_type=PeriodType.HISTORICAL
)
The function returns a pandas DataFrame with information about the available stations,
including the column **HAS_FILE**, that indicates if the station has a file with data on
the server (which may not always be the case!).
The function returns a Pandas DataFrame with information about the available stations.
The column ``HAS_FILE`` indicates whether the station actually has a file with data on
the server. That might not always be the case for stations which have been phased out.

When using ``create_new_file_index=True``, the function can be forced to retrieve
a new list of files from the server. Otherwise, data will be served from the
cache because this information rarely changes.

``DWDStationRequest``
=====================
Measurements
============
Use the ``DWDStationRequest`` class in order to get hold of measurement information.

Synopsis
--------
.. code-block:: python
from wetterdienst import DWDStationRequest, Parameter, PeriodType, TimeResolution
from wetterdienst import DWDStationRequest
from wetterdienst import Parameter, PeriodType, TimeResolution
request = DWDStationRequest(
station_ids=[3, 1048],
Expand All @@ -66,74 +90,20 @@ This gives us the most options to work with the data, getting multiple parameter
once, parsed nicely into column structure with improved parameter names and stored
automatically on the drive if wanted.

Details
-------
- A class that can combine multiple periods/date ranges for any number of stations
and parameters of one time resolution.
- Wraps ``collect_climate_observations_data``:

- Combines create_file_list_for_dwd_server, download_dwd_data and
parse_dwd_data for multiple stations
- Wraps the following three functions:

- ``create_file_list_for_climate_observations``
- is used with the help of the metadata to retrieve file paths to
files for a set of parameters + station id
- here also **create_new_file_index** can be used

- ``download_climate_observations_data_parallel``
- is used with the created file paths to download and store the data
(second os optionally, in a hdf)

- ``parse_climate_observations_data``
- is used to get the data into the Python environment in
shape of a pandas DataFrame.
- the data will be ready to be analyzed by you!


Additionally, the following functions allow you to reset the cache of the file/meta index:

- **reset_file_index_cache:**
- reset the cached file index to get latest list of files (only required for
constantly running system)

- **reset_meta_index_cache:**
- reset the cached meta index to get latest list of files (only required for
constantly running system)

Parameter, time resolution and period type can be entered in three ways:

- by using the exact enumeration e.g.
.. code-block:: python
Parameter.CLIMATE_SUMMARY
- by using the enumeration string e.g.
.. code-block:: python
"climate_summary" or "CLIMATE_SUMMARY"
- by using the originally defined parameter string e.g.
.. code-block:: python
"kl"
******************
Geospatial support
******************
==================

Inquire the list of stations by geographic coordinates.

``get_nearby_stations``
=======================
- Calculate the close weather stations based on the coordinates for the requested data.
- Either selected by rank (n stations) or by distance in km.
- It returns a DataFrame with meta data, distances [in km] and station ids
that can be used to download the data.
- Calculate weather stations close to the given coordinates and set of parameters.
- Either select by rank (n stations) or by distance in km.

.. code-block:: python
from datetime import datetime
from wetterdienst import get_nearby_stations, DWDStationRequest, Parameter, PeriodType, TimeResolution
from wetterdienst import get_nearby_stations, DWDStationRequest
from wetterdienst import Parameter, PeriodType, TimeResolution
stations = get_nearby_stations(
50.0, 8.9,
Expand All @@ -145,14 +115,16 @@ Geospatial support
num_stations_nearby=1
)
The function returns a meta data DataFrame, where we can find
weather station ids and distances to get the observation data:
The function returns a DataFrame with the list of stations with distances [in km]
to the given coordinates.

The station ids within the DataFrame:

.. code-block:: python
station_ids = stations.STATION_ID.unique()
Use these station ids to retrieve weather information:
can be used to download the observation data:

.. code-block:: python
Expand All @@ -174,7 +146,7 @@ Use these station ids to retrieve weather information:
Et voila: We just got the data we wanted for our location and are ready to analyse the
temperature on historical developments.

Check out the more advanced examples in the
Please also check out more advanced examples in the
`example <https://github.com/earthobservations/wetterdienst/tree/master/example>`_
folder on Github.

Expand Down
37 changes: 34 additions & 3 deletions docs/pages/behind_the_scenes.rst
@@ -1,5 +1,36 @@
Behind The Scenes
#################
Behind the scenes
#################

Details
-------
- The ``DWDStationRequest`` class can combine multiple periods/date ranges
for any number of stations and parameters of one time resolution.
- It wraps ``collect_climate_observations_data``, which in turn combines
``create_file_list_for_climate_observations``, ``download_climate_observations_data_parallel``
and ``parse_climate_observations_data`` for multiple stations.

- ``create_file_list_for_climate_observations``
- is used with the help of the metadata to retrieve file paths to
files for a set of parameters + station id
- here also **create_new_file_index** can be used

- ``download_climate_observations_data_parallel``
- is used with the created file paths to download and store the data
(second os optionally, in a hdf)

- ``parse_climate_observations_data``
- is used to get the data into the Python environment in
shape of a pandas DataFrame.
- the data will be ready to be analyzed by you!


Additionally, the following functions allow you to reset the cache of the file/meta index:

- **reset_file_index_cache:**
- reset the cached file index to get latest list of files (only required for
constantly running system)

Let's look at some anomalies that happen to be on the file server and how we manage to
get along with them.
- **reset_meta_index_cache:**
- reset the cached meta index to get latest list of files (only required for
constantly running system)
1 change: 1 addition & 0 deletions docs/pages/cli.rst
@@ -1,6 +1,7 @@
######################
Command line interface
######################

::

$ wetterdienst --help
Expand Down
2 changes: 1 addition & 1 deletion docs/pages/data_coverage.rst
@@ -1,7 +1,7 @@
.. _data-coverage:

#############
Data Coverage
Data coverage
#############

The DWD offers various datasets including but not only:
Expand Down

0 comments on commit 7c41359

Please sign in to comment.