Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add latest features to master #49

Merged
merged 32 commits into from
Apr 30, 2021
Merged

Add latest features to master #49

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
6f1281b
Update nc reader
sebhahn Jan 15, 2021
ef381e4
Update hdf5 reader
sebhahn Jan 25, 2021
d319517
Update bufr reader, rewriting tests
sebhahn Feb 3, 2021
beabd3b
Updated nc and bufr reader
sebhahn Feb 17, 2021
da6f11e
Remove tests temporary, update environment.yml
sebhahn Mar 23, 2021
3b2e19a
Update bufr reader
sebhahn Mar 24, 2021
a589336
Update hdf5 reader
sebhahn Mar 24, 2021
f83b0b5
Update bufr reader
sebhahn Mar 24, 2021
6a335ef
Refactor nc
sebhahn Mar 25, 2021
abd2b98
Update eps reader
sebhahn Mar 25, 2021
b4a348a
Update copyright
sebhahn Mar 25, 2021
5f653e3
Update level 1 tests
sebhahn Mar 28, 2021
8961a47
Update readers and tests
sebhahn Mar 30, 2021
593d575
Fix level1 tests
sebhahn Mar 31, 2021
ad0c077
Rename file
sebhahn Mar 31, 2021
9534686
Fix imports and documenation
sebhahn Mar 31, 2021
209d699
Update level2 reader and tests
sebhahn Mar 31, 2021
04a6e24
Remove obsolete test
sebhahn Mar 31, 2021
fe98b3a
Fix level2
sebhahn Apr 1, 2021
e938a05
Add and test file search
sebhahn Apr 8, 2021
b4d8a02
Testing
sebhahn Apr 9, 2021
f3428e4
Testing
sebhahn Apr 9, 2021
66bfc9f
Add new readers, update tests
sebhahn Apr 14, 2021
efb5242
Restructure package
sebhahn Apr 16, 2021
cbd1054
Fix bug in subset
sebhahn Apr 20, 2021
ae3c7cd
Update documentation
sebhahn Apr 22, 2021
ba8f361
Fix datatype
sebhahn Apr 22, 2021
b67aa45
Fix isinstance check of dict
Apr 22, 2021
17ff623
Fix docstring
sebhahn Apr 30, 2021
cce97c2
Refactoring
sebhahn Apr 30, 2021
3e7c38a
Update readme
sebhahn Apr 30, 2021
b13da98
Merge pull request #48 from TUW-GEO/new_reader
sebhahn Apr 30, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@
Changelog
=========

Version 2.0.0
=============

- New interface reading ASCAT Level 1b and Level 2 data
- Removing old interfaces to TU Wien data
- Restructure package and harmonize interface class names
- Update documentation

Version 1.2.0
=============

Expand Down
2 changes: 1 addition & 1 deletion LICENSE.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
The MIT License (MIT)

Copyright (c) 2020 TU Wien
Copyright (c) 2021 TU Wien

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
25 changes: 9 additions & 16 deletions README.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,20 +17,20 @@ ascat
.. image:: https://readthedocs.org/projects/ascat/badge/?version=latest
:target: http://ascat.readthedocs.org/

Read and convert data acquired by ASCAT on-board the series of Metop satellites. Written in Python.
Read and convert data acquired by ASCAT on-board the series of Metop satellites.

Works great in combination with `pytesmo <https://github.com/TUW-GEO/pytesmo>`_.

Citation
========

.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.596434.svg
:target: https://doi.org/10.5281/zenodo.596434
.. image:: https://zenodo.org/badge/DOI/10.5281/zenodo.4610836.svg
:target: https://doi.org/10.5281/zenodo.4610836

If you use the software in a publication then please cite it using the Zenodo DOI.
Be aware that this badge links to the latest package version.

Please select your specific version at https://doi.org/10.5281/zenodo.596434 to get the DOI of that version.
Please select your specific version at https://doi.org/10.5281/zenodo.4610836 to get the DOI of that version.
You should normally always use the DOI for the specific version of your record in citations.
This is to ensure that other researchers can access the exact research artefact you used for reproducibility.

Expand All @@ -39,9 +39,7 @@ You can find additional information regarding DOI versioning at http://help.zeno
Installation
============

The packages you have to install depend on the features you want to use. The ASCAT soil moisture products are disseminated in BUFR, NetCDF or GRIB format. In order to read them you will have to install the appropriate packages which will be explained shortly. Unfortunately neither BUFR nor GRIB readers work on Windows so if you need these formats then Linux or OS X are your only options.

For installation we recommend `Miniconda <http://conda.pydata.org/miniconda.html>`_. So please install it according to the official installation instructions. As soon as you have the ``conda`` command in your shell you can continue.
ASCAT data are distributed in BUFR, NetCDF, EPS Native and GRIB format. Unfortunately neither BUFR nor GRIB readers work on Windows so if you need these formats then Linux or OS X are your only options.

The following script will download and install all the needed packages.

Expand All @@ -58,7 +56,7 @@ Windows:
conda env create -f environment_win.yml


Supported Products
Supported datasets
==================

This gives a short overview over the supported products. Please see the documentation for detailed examples of how to work with a product.
Expand All @@ -78,12 +76,7 @@ Read ASCAT data from different sources into a common format supported by pytesmo
Contribute
==========

We are happy if you want to contribute. Please raise an issue explaining what is missing or if you find a bug. We will also gladly accept pull requests against our master branch for new features or bug fixes.

Development setup
-----------------

For Development we also recommend a ``conda`` environment. You can create one including test dependencies and debugger by running ``conda env create -f environment.yml``.
We are happy if you want to contribute. Please raise an issue explaining what is missing or if you find a bug. We will also gladly accept pull requests for new features or bug fixes.

Guidelines
----------
Expand All @@ -94,8 +87,8 @@ If you want to contribute please follow these steps:
- Clone the repository, make sure you use ``git clone --recursive`` to also get the test data repository.
- Make a new feature branch from the ascat master branch
- Add your feature
- Please include tests for your contributions in one of the test directories. We use py.test so a simple function called test_my_feature is enough
- Submit a pull request to our master branch
- Please include tests for your contributions in one of the test directories
- Submit a pull request

Note
====
Expand Down
2 changes: 0 additions & 2 deletions docs/changes.rst
View file
Open in desktop

This file was deleted.

14 changes: 0 additions & 14 deletions docs/environment.yml
View file
Open in desktop

This file was deleted.

204 changes: 43 additions & 161 deletions docs/read_cgls_swi_ts.ipynb
View file
Open in desktop

Large diffs are not rendered by default.

169 changes: 32 additions & 137 deletions docs/read_cgls_swi_ts.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@

Example CGLS SWI time series
----------------------------

.. code:: ipython2
.. code:: ipython3

import os
from ascat.cgls import SWI_TS
Expand All @@ -12,167 +11,63 @@ Example CGLS SWI time series
By default we should have the grid file from the SWI-STATIC collection
and the unzipped SWI-TS products in one folder like so:

.. code:: ipython2
.. code:: ipython3

ls ../tests/ascat_test_data/cglops/swi_ts


.. parsed-literal::

c_gls_SWI-STATIC-DGG_201501010000_GLOBE_ASCAT_V3.0.1.nc
c_gls_SWI-TS_201612310000_C0375_ASCAT_V3.0.1.nc

c_gls_SWI-STATIC-DGG_201501010000_GLOBE_ASCAT_V3.0.1.nc
c_gls_SWI-TS_201612310000_C0375_ASCAT_V3.0.1.nc

Now we can initialize the data reader

.. code:: ipython2
.. code:: ipython3

data_path = os.path.join('..', 'tests', 'ascat_test_data', 'cglops', 'swi_ts')
rd = SWI_TS(data_path)

With this object we can read the data

.. code:: ipython2
.. code:: ipython3

data = rd.read_ts(3002621)
print(data)


.. parsed-literal::

SSF SWI_001 SWI_005 SWI_010 SWI_015 SWI_020 \
2007-05-10 12:00:00 1.0 70.0 71.0 71.0 71.0 71.0
2007-05-14 12:00:00 1.0 71.5 71.5 71.5 71.5 71.5
2007-05-15 12:00:00 1.0 82.0 78.0 77.0 76.5 76.5
2007-05-16 12:00:00 1.0 82.5 79.0 78.0 77.5 77.0
2007-05-17 12:00:00 1.0 77.5 77.5 77.0 76.5 76.5
2007-05-18 12:00:00 1.0 75.0 76.5 76.0 76.0 76.0
2007-05-19 12:00:00 1.0 75.5 76.0 76.0 76.0 76.0
2007-05-20 12:00:00 1.0 86.0 81.0 79.5 79.5 79.0
2007-05-21 12:00:00 1.0 84.5 81.0 80.0 79.5 79.5
2007-05-22 12:00:00 1.0 81.5 80.5 80.0 79.5 79.0
2007-05-23 12:00:00 1.0 81.5 80.5 80.0 79.5 79.5
2007-05-24 12:00:00 1.0 81.0 81.0 80.0 80.0 79.5
2007-05-25 12:00:00 1.0 79.5 80.0 79.5 79.5 79.5
2007-05-26 12:00:00 1.0 81.5 80.5 80.0 79.5 79.5
2007-05-27 12:00:00 1.0 76.5 79.5 79.5 79.0 79.0
2007-05-28 12:00:00 1.0 80.5 79.5 79.5 79.5 79.5
2007-05-29 12:00:00 1.0 82.0 80.5 80.0 79.5 79.5
2007-05-30 12:00:00 1.0 83.0 81.0 80.5 80.0 80.0
2007-05-31 12:00:00 1.0 79.0 80.5 80.0 80.0 80.0
2007-06-01 12:00:00 1.0 75.5 79.5 79.5 79.5 79.5
2007-06-02 12:00:00 1.0 80.5 79.5 79.5 79.5 79.5
2007-06-03 12:00:00 1.0 85.0 81.5 80.5 80.5 80.0
2007-06-04 12:00:00 1.0 85.0 82.0 81.0 81.0 80.5
2007-06-05 12:00:00 1.0 84.5 82.0 81.5 81.0 81.0
2007-06-06 12:00:00 1.0 79.5 81.5 81.0 80.5 80.5
2007-06-07 12:00:00 1.0 77.5 80.0 80.0 80.0 80.0
2007-06-08 12:00:00 1.0 77.0 79.5 80.0 80.0 80.0
2007-06-09 12:00:00 1.0 77.0 79.5 79.5 80.0 80.0
2007-06-10 12:00:00 1.0 77.0 79.0 79.5 79.5 79.5
2007-06-11 12:00:00 1.0 76.5 78.0 79.0 79.0 79.5
... ... ... ... ... ... ...
2016-09-19 12:00:00 1.0 73.5 76.5 78.5 80.0 81.0
2016-09-20 12:00:00 1.0 74.5 76.0 78.5 80.0 81.0
2016-09-21 12:00:00 1.0 75.5 76.0 78.0 79.5 80.5
2016-09-22 12:00:00 1.0 73.5 75.5 77.5 79.0 80.5
2016-09-23 12:00:00 1.0 72.0 75.0 77.5 79.0 80.0
2016-09-24 12:00:00 1.0 69.5 73.5 76.5 78.0 79.5
2016-09-25 12:00:00 1.0 71.0 73.0 75.5 77.5 79.0
2016-09-26 12:00:00 1.0 71.5 73.0 75.5 77.0 78.5
2016-09-27 12:00:00 1.0 77.0 74.0 76.0 77.5 78.5
2016-09-28 12:00:00 1.0 83.5 76.0 76.5 78.0 79.0
2016-09-29 12:00:00 1.0 92.0 81.0 79.0 79.5 80.0
2016-09-30 12:00:00 1.0 93.5 82.5 80.0 80.0 80.5
2016-10-01 12:00:00 1.0 90.0 84.0 81.0 81.0 81.0
2016-10-02 12:00:00 1.0 84.0 83.5 81.0 80.5 81.0
2016-10-03 12:00:00 1.0 72.0 80.0 79.5 80.0 80.0
2016-10-04 12:00:00 1.0 64.5 76.0 77.5 78.5 79.0
2016-10-05 12:00:00 1.0 57.5 71.5 75.0 76.5 78.0
2016-10-06 12:00:00 1.0 65.0 70.5 74.0 75.5 77.0
2016-10-07 12:00:00 1.0 66.0 70.0 73.5 75.0 76.5
2016-10-08 12:00:00 1.0 68.0 69.5 73.0 75.0 76.0
2016-10-09 12:00:00 1.0 67.5 69.0 72.5 74.5 76.0
2016-10-10 12:00:00 1.0 65.5 68.5 71.5 74.0 75.5
2016-10-11 12:00:00 1.0 60.0 66.0 70.0 72.5 74.0
2016-10-12 12:00:00 1.0 54.0 63.5 68.5 71.0 73.0
2016-10-13 12:00:00 1.0 52.5 61.0 66.5 69.5 72.0
2016-10-14 12:00:00 1.0 55.0 60.0 65.5 69.0 71.0
2016-10-15 12:00:00 1.0 56.5 59.0 64.5 68.0 70.5
2016-10-16 12:00:00 1.0 61.5 60.0 64.5 67.5 70.0
2016-10-26 12:00:00 1.0 12.5 42.5 59.0 64.5 68.0
2016-10-28 12:00:00 1.0 69.5 54.0 60.0 64.5 67.5
SWI_001 SWI_005 SWI_010 SWI_015 SWI_020 SWI_040 \
2007-05-10 12:00:00 70.0 71.0 71.0 71.0 71.0 71.0
2007-05-14 12:00:00 71.5 71.5 71.5 71.5 71.5 71.5
2007-05-15 12:00:00 82.0 78.0 77.0 76.5 76.5 76.0
2007-05-16 12:00:00 82.5 79.0 78.0 77.5 77.0 77.0
2007-05-17 12:00:00 77.5 77.5 77.0 76.5 76.5 76.0
... ... ... ... ... ... ...
2016-10-14 12:00:00 55.0 60.0 65.5 69.0 71.0 76.5
2016-10-15 12:00:00 56.5 59.0 64.5 68.0 70.5 75.5
2016-10-16 12:00:00 61.5 60.0 64.5 67.5 70.0 75.5
2016-10-26 12:00:00 12.5 42.5 59.0 64.5 68.0 74.5
2016-10-28 12:00:00 69.5 54.0 60.0 64.5 67.5 74.0

SWI_040 SWI_060 SWI_100
2007-05-10 12:00:00 71.0 NaN NaN
2007-05-14 12:00:00 71.5 71.5 NaN
2007-05-15 12:00:00 76.0 76.0 76.0
2007-05-16 12:00:00 77.0 77.0 76.5
2007-05-17 12:00:00 76.0 76.0 76.0
2007-05-18 12:00:00 76.0 76.0 76.0
2007-05-19 12:00:00 76.0 75.5 75.5
2007-05-20 12:00:00 79.0 78.5 78.5
2007-05-21 12:00:00 79.0 79.0 79.0
2007-05-22 12:00:00 79.0 79.0 78.5
2007-05-23 12:00:00 79.0 79.0 79.0
2007-05-24 12:00:00 79.5 79.5 79.0
2007-05-25 12:00:00 79.0 79.0 79.0
2007-05-26 12:00:00 79.5 79.0 79.0
2007-05-27 12:00:00 79.0 79.0 79.0
2007-05-28 12:00:00 79.0 79.0 79.0
2007-05-29 12:00:00 79.5 79.5 79.0
2007-05-30 12:00:00 80.0 79.5 79.5
2007-05-31 12:00:00 79.5 79.5 79.5
2007-06-01 12:00:00 79.5 79.0 79.0
2007-06-02 12:00:00 79.5 79.0 79.0
2007-06-03 12:00:00 80.0 80.0 80.0
2007-06-04 12:00:00 80.5 80.0 80.0
2007-06-05 12:00:00 80.5 80.5 80.0
2007-06-06 12:00:00 80.5 80.0 80.0
2007-06-07 12:00:00 80.0 80.0 79.5
2007-06-08 12:00:00 80.0 79.5 79.5
2007-06-09 12:00:00 79.5 79.5 79.5
2007-06-10 12:00:00 79.5 79.5 79.5
2007-06-11 12:00:00 79.5 79.5 79.5
... ... ... ...
2016-09-19 12:00:00 83.0 84.0 84.0
2016-09-20 12:00:00 83.0 83.5 84.0
2016-09-21 12:00:00 83.0 83.5 84.0
2016-09-22 12:00:00 82.5 83.5 83.5
2016-09-23 12:00:00 82.5 83.0 83.5
2016-09-24 12:00:00 82.0 83.0 83.5
2016-09-25 12:00:00 81.5 82.5 83.0
2016-09-26 12:00:00 81.5 82.5 83.0
2016-09-27 12:00:00 81.5 82.5 83.0
2016-09-28 12:00:00 81.5 82.5 83.0
2016-09-29 12:00:00 82.0 83.0 83.5
2016-09-30 12:00:00 82.0 83.0 83.5
2016-10-01 12:00:00 82.5 83.0 83.5
2016-10-02 12:00:00 82.5 83.0 83.5
2016-10-03 12:00:00 82.0 82.5 83.0
2016-10-04 12:00:00 81.5 82.5 83.0
2016-10-05 12:00:00 80.5 81.5 82.5
2016-10-06 12:00:00 80.0 81.5 82.0
2016-10-07 12:00:00 79.5 81.0 82.0
2016-10-08 12:00:00 79.5 81.0 82.0
2016-10-09 12:00:00 79.0 80.5 81.5
2016-10-10 12:00:00 79.0 80.5 81.5
2016-10-11 12:00:00 78.0 80.0 81.0
2016-10-12 12:00:00 77.5 79.5 81.0
2016-10-13 12:00:00 77.0 79.0 80.5
2016-10-14 12:00:00 76.5 78.5 80.0
2016-10-15 12:00:00 75.5 78.0 80.0
2016-10-16 12:00:00 75.5 78.0 79.5
2016-10-26 12:00:00 74.5 77.0 79.0
2016-10-28 12:00:00 74.0 76.5 79.0
SWI_060 SWI_100 SSF
2007-05-10 12:00:00 NaN NaN 1.0
2007-05-14 12:00:00 71.5 NaN 1.0
2007-05-15 12:00:00 76.0 76.0 1.0
2007-05-16 12:00:00 77.0 76.5 1.0
2007-05-17 12:00:00 76.0 76.0 1.0
... ... ... ...
2016-10-14 12:00:00 78.5 80.0 1.0
2016-10-15 12:00:00 78.0 80.0 1.0
2016-10-16 12:00:00 78.0 79.5 1.0
2016-10-26 12:00:00 77.0 79.0 1.0
2016-10-28 12:00:00 76.5 79.0 1.0

[1603 rows x 9 columns]


Since the returned value is a pandas.DataFrame we can plot the data
easily.

.. code:: ipython2
.. code:: ipython3

fig, ax = plt.subplots(1, 1, figsize=(15, 5))
data[['SWI_001', 'SWI_010']].plot(ax=ax)
Expand All @@ -181,6 +76,6 @@ easily.



.. image:: read_cgls_swi_ts_files/read_cgls_swi_ts_9_0.png
.. image:: read_cgls_swi_ts_files/read_cgls_swi_ts_7_0.png


Binary file added docs/read_cgls_swi_ts_files/read_cgls_swi_ts_7_0.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file removed docs/read_cgls_swi_ts_files/read_cgls_swi_ts_9_0.png
View file
Open in desktop
Binary file not shown.
Loading