Data shift algorithm #124
Data shift algorithm #124
Conversation
…set for running unit tests
|
Also I was discussing with @kanderso-nrel this morning on having a published basis for this algorithm. Attached is the associated PVSC 2022 abstract for this work, as a reference |
There was a problem hiding this comment.
Here's a partial review -- I will take another look after some of the higher level questions below are discussed.
Other thoughts:
- Probably makes sense to add a bullet in
docs/index.rstunderLibrary Overviewfor this newquality.data_shiftssubmodule quality/__init__.pyneeds an entry fordata_shifts
pvanalytics/quality/data_shifts.py
Outdated
| _run_data_checks(time_series, use_default_models, method, cost, penalty) | ||
| # Run the filtering sequence, if marked as True | ||
| if filtering: | ||
| time_series = _erroneous_filter(time_series) |
There was a problem hiding this comment.
In the spirit of "one function, one job", I wonder if it should be left to the user to filter data prior to passing in?
Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com>
Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com>
Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com>
|
FYI the test failures should be fixed by the last commit in #129 |
|
@cwhanse I've got the issues that @kanderso-nrel brought up addressed, so this should be ready for review. Everything passing, but our coveralls coverage dropped by .2%. |
…dded unit test for this.
…ysis on, to filter out any data from the previous or next data shift.
|
This PR was accidentally merged into master. I reverted master back to the previous commit, and am reopening this. |
Description
This PR handles detecting data shifts in daily summed time series (power or irradiance), and removing data shift periods. The process of doing this is illustrated in the Jupyter notebook in the /docs/ folder. I am reporting the results of this algorithm (performance) in a submitted PVSC abstract/paper, which we can cite for performance metrics.
Checklist
The following items must be addressed before the code can be merged.
Please don't hesitate to ask for help if you are unsure of how to accomplish any of the items.
You are free to remove any checklist items that do not apply or add additional items that are
not on this list
docs/api.rstin
docs/whatsnewfor all changes. Includes link to the GitHub Issue with
:issue:`num`or this Pull Request with
:pull:`num`. Includes contributor nameand/or GitHub username (link with
:ghuser:`user`).