Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support context configurations (spatial/temporal subsets) and climatology-derived-configs. #39

Merged
merged 49 commits into from
Nov 30, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
6c92d28
Use a setup.cfg file for package description/config and add isort
kwilcox Mar 18, 2020
11a174a
Move config parsing to one utils function, add additional options
kwilcox Mar 18, 2020
fa735ce
Make add_flag_metadata accept generic kwargs, add the aggregate kwarg
kwilcox Mar 18, 2020
bee1786
Replace QcConfig with a wrapper class around StreamConfig
kwilcox Mar 19, 2020
7a2f340
Add in the first sources: Pandas and Numpy
kwilcox Mar 19, 2020
80d8c4a
Additional documentation, cleansing, and improvements
kwilcox May 29, 2020
9a3630a
Remove 3.6 from supported versions, add numba as a required dependency
kwilcox May 29, 2020
d18b57f
Source -> Stream. Create the Store concept and move netCDF writing there
kwilcox May 29, 2020
a6d2ad6
Fix XarrayStream getting passed an open Dataset, add a test
kwilcox May 29, 2020
c56e20b
Fix numba install/version. Must be a recent bug.
kwilcox May 29, 2020
6044825
Update docs
kwilcox Aug 6, 2020
ff274e4
Don't error if a defined stream config is not found on a Stream
kwilcox Aug 6, 2020
2ccf233
Don't error out if a region is defined as None/null
kwilcox Aug 6, 2020
21dc641
Adding additional notebook examples
kwilcox Aug 20, 2020
87089b7
Make sure quality results are uint8, save some memory in pandas results
kwilcox Aug 20, 2020
ffe0f2b
Don't error if a passed in column doesn't exist in the pandas frame
kwilcox Aug 20, 2020
fcb7868
Fix typo in docs
kwilcox Aug 20, 2020
ef46f05
Fix typo when testing for existing test results in XarrayStream
kwilcox Aug 20, 2020
73ab500
Add a test for a light config with no context defined
kwilcox Aug 20, 2020
dc31e85
For now, return the native results type for each test
kwilcox Aug 20, 2020
277663f
add intro/basic usage to docs
Oct 20, 2020
f2c7a8a
add docs todo
Oct 20, 2020
5750e67
Fix speed test(s) decorators
kwilcox Nov 17, 2020
3fdadc0
Add generic bokeh plotting functions for QC results
kwilcox Nov 20, 2020
aa2cee3
Fix date mapping error when using pandas Timestamp objects with a tim…
kwilcox Nov 20, 2020
3f9e5f5
Remove numba version limitation
kwilcox Nov 20, 2020
7979853
Add plotting example notebook
kwilcox Nov 20, 2020
f52d5a8
Starting the ERDDAP QC example
kwilcox Nov 20, 2020
be7d1da
[wip] add config_creator
yosoyjay Nov 16, 2020
9d132dd
Merge branch 'add-config-creator' into config-and-clim
kwilcox Nov 20, 2020
662c5b5
WIP: just pushing for Jesse!
kwilcox Nov 20, 2020
8a906aa
align config_creator implementation, docs, and test + reqs update
yosoyjay Nov 20, 2020
9000a5b
add processed NARR + Ocean Atlas for config_creator
yosoyjay Nov 20, 2020
525e32c
Better generic plotting, view ERDDAP QC results
kwilcox Nov 24, 2020
448aeb2
Add time and data accessors to each Stream, useful for plotting
kwilcox Nov 24, 2020
a1bef2a
Fix docs for create_config usage
kwilcox Nov 24, 2020
a65c0d1
Better error message for suspect span outside fail span
kwilcox Nov 24, 2020
c5cdf01
Instead of returning dictionary test results, yield the individual re…
kwilcox Nov 25, 2020
0272d99
Improve documentation on Streams and Stores
kwilcox Nov 30, 2020
29c609d
Add a CFNetCDFStore and PandasStore
kwilcox Nov 30, 2020
a0a3f20
Allow including and excluding QC methods by name
kwilcox Nov 30, 2020
963614e
More documentation updates for Stores and Results
kwilcox Nov 30, 2020
2ed2285
Use typing.NamedTuple to customize the __repr__ methods of results
kwilcox Nov 30, 2020
f16c44b
Short-cut when we ran a check for an entire stream with no subsetting
kwilcox Nov 30, 2020
2bb4ceb
Fix numpy deprecation warning
kwilcox Nov 30, 2020
0560e45
Add Bokeh plotting shortcuts for a list of CollectionResult objects
kwilcox Nov 30, 2020
4e437e0
Update PMEL ERDDAP notebook
kwilcox Nov 30, 2020
282203c
Remove unbuffer from modd commands
kwilcox Nov 30, 2020
1529048
Add an ERDDAP stream/store test case
kwilcox Nov 30, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,5 @@ docs/build
tests/data/*.csv
__pycache__
.vscode
build/
docs/source/_build
4 changes: 1 addition & 3 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ env:
jobs:
fast_finish: true
include:
- python: 3.6
env: TEST_TARGET=default
- python: 3.7
env: TEST_TARGET=default
- python: 3.7
Expand All @@ -24,7 +22,7 @@ jobs:
env: TEST_TARGET=coding_standards
- python: 3.8
env: TEST_TARGET=docs
- python: 3.6
- python: 3.8
env: TEST_TARGET=makerelease
allow_failures:
- python: 3.8
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ include *.rst
include *.txt
include *.ini
include VERSION
include setup.cfg
recursive-include ioos_qc *.py
2 changes: 2 additions & 0 deletions conda-recipe/meta.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,11 @@ requirements:
- python
- geojson
- netCDF4
- numba
- numpy >=1.14
- pygc
- ruamel.yaml
- shapely
- simplejson
- xarray

Expand Down
43 changes: 43 additions & 0 deletions docs/source/api/ioos_qc.config_creator.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
ioos\_qc.config\_creator package
================================

.. automodule:: ioos_qc.config_creator
:members:
:undoc-members:
:show-inheritance:

Submodules
----------

ioos\_qc.config\_creator.config\_creator module
-----------------------------------------------

.. automodule:: ioos_qc.config_creator.config_creator
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.config\_creator.fx\_parser module
------------------------------------------

.. automodule:: ioos_qc.config_creator.fx_parser
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.config\_creator.get\_assets module
-------------------------------------------

.. automodule:: ioos_qc.config_creator.get_assets
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.config\_creator.make\_config module
--------------------------------------------

.. automodule:: ioos_qc.config_creator.make_config
:members:
:undoc-members:
:show-inheritance:

83 changes: 65 additions & 18 deletions docs/source/api/ioos_qc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,43 +2,90 @@ ioos\_qc package
================

.. automodule:: ioos_qc
:members:
:undoc-members:
:show-inheritance:
:members:
:undoc-members:
:show-inheritance:

Subpackages
-----------

.. toctree::
:maxdepth: 4

ioos_qc.config_creator

Submodules
----------

ioos\_qc.argo module
--------------------

.. automodule:: ioos_qc.argo
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.config module
----------------------

.. automodule:: ioos_qc.config
:members:
:undoc-members:
:show-inheritance:
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.argo module
ioos\_qc.gliders module
-----------------------

.. automodule:: ioos_qc.argo
:members:
:undoc-members:
:show-inheritance:
.. automodule:: ioos_qc.gliders
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.plotting module
------------------------

.. automodule:: ioos_qc.plotting
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.qartod module
----------------------

.. automodule:: ioos_qc.qartod
:members:
:undoc-members:
:show-inheritance:
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.results module
-----------------------

.. automodule:: ioos_qc.results
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.stores module
----------------------

.. automodule:: ioos_qc.stores
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.streams module
-----------------------

.. automodule:: ioos_qc.streams
:members:
:undoc-members:
:show-inheritance:

ioos\_qc.utils module
---------------------

.. automodule:: ioos_qc.utils
:members:
:undoc-members:
:show-inheritance:

:members:
:undoc-members:
:show-inheritance:

3 changes: 2 additions & 1 deletion docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
#
import os
import sys

p = os.path.abspath(
os.path.dirname(os.path.dirname(os.path.dirname(__file__)))
)
Expand All @@ -41,7 +42,7 @@
'sphinx.ext.mathjax',
'sphinx.ext.viewcode',
'sphinx_autodoc_typehints',
"nbsphinx",
#"nbsphinx",
]

napoleon_google_docstring = True
Expand Down
2 changes: 2 additions & 0 deletions docs/source/examples.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@ You can download the source for these notebooks `here <https://github.com/ioos/i
* :doc:`Run QARTOD on a netCDF file, and store the results </examples/Qartod_netCDF_example>`

* :doc:`Run Attenuated Signal test on Salinity data to detect biofouling in an instrument </examples/QartodTestExample_SalinityAttenuation>`

* **TODO: Add links to new notebooks under usage/**
Loading