Multi yoy no label#498
Conversation
|
We could leverage the |
…ars, removing warning.
|
would this give us the behavior we're after? cdeline#1 @cdeline @martin-springer let me know what you think. |
There was a problem hiding this comment.
Pull request overview
This PR updates the year-on-year (YoY) degradation workflow to remove reliance on label=-based timestamping, adds explicit YoY timestamp metadata (YoY_times) to the degradation_year_on_year outputs, and refactors the degradation time-series plotting path to better support multi-YoY results.
Changes:
- Removes
label=-specific tests/fixtures and updatesTrendAnalysisdefaults to no longer passlabelthroughyoy_kwargs. - Extends YoY calc output metadata with
calc_info['YoY_times']and updatesdegradation_timeseries_plotto use timestamps/durations from that structure. - Updates v3.2.0 changelog entries to reflect the new
YoY_timesoutput and the revised multi-YoY time-series plotting approach.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| rdtools/test/plotting_test.py | Removes label-based coverage and updates multi-YoY handling assertions for degradation_timeseries_plot. |
| rdtools/test/degradation_test.py | Removes tests for now-removed/changed label='center' behavior. |
| rdtools/plotting.py | Refactors degradation_timeseries_plot to consume YoY_times and handle multi-YoY smoothing differently. |
| rdtools/degradation.py | Removes label documentation/logic and adjusts how YoY timestamps/indices are produced. |
| rdtools/analysis_chains.py | Removes default yoy_kwargs={"label": "right"} so chains don’t depend on the label parameter. |
| docs/sphinx/source/changelog/v3.2.0.rst | Updates changelog notes for YoY_times and the new multi-YoY plotting behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Mike's PR was good. I do want to update the rolling median though to have the |
…_periods=rolling_days//4)
|
OK, good to go @martin-springer and @mdeceglie. |
Since this would change the math relative to our latest release, I think the best path would be to make it an argument, we can change the default behavior in the next major version. |
Commit 15918a3 adds the new argument |
* update pvdaq_system_4 url * changelog * revert unnecessary changes in notebooks
* Multi yoy integration - combines PRs #456, #460 and #464 at once. (#470) * add keyword 'label' to degradation_timeseries_plot, enabling 'left' and 'center' labeling options. * Update changelog, add pytests, update sphinx documentation * fix flake8 grumbles * update pytests to include axes limits * fix flake8 grumbles * add 'label' input option to `degradation_year_on_year`. Fixes #459 * add pytests and update changelog. * flake8 grumbles * Minor updates to setup.py (constrain scipy<1.16) and refactor degradation_test * Custom fix for Pandas < 2.0.0 which can't average two columns of timestamps. * flake8 grumbles * keep TZ-aware timestamps. Update pytests to specifically test _avg_timestamp_old_Pandas * flake8 grumbles * try to UTC localize the pytest... * Add .asfreq() to get pytests to agree * switch to calendar.timegm to hopefully remove TZ issues.. * regardless of uncertainty_method, return calc_info{'YoY_values') * update _right dt labels to correct _left labels in degradation_year_on_year * update _avg_timestamp_old_Pandas to allow for numeric index instead of timestamp * add left label option to degradation_year_on_year * update degradation_year_on_year, index set to either left, center or right. Consistent with #394 - multi_yoy * update return for default = none uncertainty option * degradation_year_on_year - go back to single return when uncertainty_value = None to avoid breaking pytests. * add multi-year aggregation of slopes in degradation_year_on_year * add multi_yoy kwarg in degradation_year_on_year to toggle the multi-YoY function. * update plotting for detailed=True, allow usage_of_points > 2 * flake8 grumbles * update plotting detailed=True for (even) and (odd) number of points coloring * To allow multi_yoy=True in plotting.degradation_timeseries_plot, resample.mean() the YoY_values. * flake8 grumbles * Add warning to degradation_timeseries_plot when multi_YoY=True * update to warning message in plotting.degradation_timeseries_plot * fix flake8 grumbles * nbval fixes from qnguyen345-bare_except_error * Add pandas 3.0 futurewarning handling * Try again to solve pandas3.0 futurewarning * attempt 3 to fix nbval * Add infer_objects to remove futurewarning * minor inline comment update * update plotting tests to be relative value, update ordering of module import in plotting.py, per Copilot review. * update inline comments and whatsnew docs * Clean up inline comments per Copilot review * added multi-YoY pytest - still need to catch warnings * Add a warnings.catch_warnings to the plotting pytest * flake8 grumbles * add multi-YoY=True pytest * updated changelog * update changelog * implement copilot suggestions * linting * use s instead of ns for pandas 3 compatibility * update pandas version comparison * set matplotlib non-gui backend for tests * set dtype resolution based on pandas version * linting import order * boost degradation.py test coverage * update changelog * exclude coverage reports with suffixes * Update rdtools/degradation.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * remove label=None handling, rely on default 'right' behavior * refactor dt_center tz handling for old pandas * simplify _avg_timestamp_old_Pandas * degradation_timeseries_plot: change rolling median min_periods to rolling_days / 4. * remove degradation_timeseries_plot(label=) and just default to center=True * update sensor_analysis() and clearsky_analysis() docstrings to discuss passing `label=right` kwargs * flake8 updates * Initial commit - multi-YoY notebook * pretty-print notebook dataframes with tabulate. * update notebook requirements to silence pandas warnings * add multi-yoy nb to tutorials * fix nblink path * Change the yoy_values index to be named 'dt'. Add new illustrations at the end of the multi-YoY notebook. --------- Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: martin-springer <martinspringer.ms@gmail.com> Co-authored-by: Martin Springer <97482055+martin-springer@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * Increase test coverage (#486) * add keyword 'label' to degradation_timeseries_plot, enabling 'left' and 'center' labeling options. * Update changelog, add pytests, update sphinx documentation * fix flake8 grumbles * update pytests to include axes limits * fix flake8 grumbles * add 'label' input option to `degradation_year_on_year`. Fixes #459 * add pytests and update changelog. * flake8 grumbles * Minor updates to setup.py (constrain scipy<1.16) and refactor degradation_test * Custom fix for Pandas < 2.0.0 which can't average two columns of timestamps. * flake8 grumbles * keep TZ-aware timestamps. Update pytests to specifically test _avg_timestamp_old_Pandas * flake8 grumbles * try to UTC localize the pytest... * Add .asfreq() to get pytests to agree * switch to calendar.timegm to hopefully remove TZ issues.. * regardless of uncertainty_method, return calc_info{'YoY_values') * update _right dt labels to correct _left labels in degradation_year_on_year * update _avg_timestamp_old_Pandas to allow for numeric index instead of timestamp * add left label option to degradation_year_on_year * update degradation_year_on_year, index set to either left, center or right. Consistent with #394 - multi_yoy * update return for default = none uncertainty option * degradation_year_on_year - go back to single return when uncertainty_value = None to avoid breaking pytests. * add multi-year aggregation of slopes in degradation_year_on_year * add multi_yoy kwarg in degradation_year_on_year to toggle the multi-YoY function. * update plotting for detailed=True, allow usage_of_points > 2 * flake8 grumbles * update plotting detailed=True for (even) and (odd) number of points coloring * To allow multi_yoy=True in plotting.degradation_timeseries_plot, resample.mean() the YoY_values. * flake8 grumbles * Add warning to degradation_timeseries_plot when multi_YoY=True * update to warning message in plotting.degradation_timeseries_plot * fix flake8 grumbles * nbval fixes from qnguyen345-bare_except_error * Add pandas 3.0 futurewarning handling * Try again to solve pandas3.0 futurewarning * attempt 3 to fix nbval * Add infer_objects to remove futurewarning * minor inline comment update * update plotting tests to be relative value, update ordering of module import in plotting.py, per Copilot review. * update inline comments and whatsnew docs * Clean up inline comments per Copilot review * added multi-YoY pytest - still need to catch warnings * Add a warnings.catch_warnings to the plotting pytest * flake8 grumbles * add multi-YoY=True pytest * updated changelog * update changelog * implement copilot suggestions * linting * use s instead of ns for pandas 3 compatibility * update pandas version comparison * set matplotlib non-gui backend for tests * set dtype resolution based on pandas version * linting import order * boost degradation.py test coverage * add tests for error handling in analysis_chain * update changelog * update changelog * exclude coverage reports with suffixes * Update rdtools/degradation.py Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * remove label=None handling, rely on default 'right' behavior * refactor dt_center tz handling for old pandas * simplify _avg_timestamp_old_Pandas * degradation_timeseries_plot: change rolling median min_periods to rolling_days / 4. * remove degradation_timeseries_plot(label=) and just default to center=True * update sensor_analysis() and clearsky_analysis() docstrings to discuss passing `label=right` kwargs * flake8 updates * Initial commit - multi-YoY notebook * pretty-print notebook dataframes with tabulate. * update notebook requirements to silence pandas warnings * add multi-yoy nb to tutorials * fix nblink path * Change the yoy_values index to be named 'dt'. Add new illustrations at the end of the multi-YoY notebook. * clean up pending changelog --------- Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * rename pending to v3.2.0 * Update docstrings and address simple comments from copilot * flake8 grumbles * Fix broad `except ValueError` in `degradation_timeseries_plot` for multi-YoY detection (#490) * Initial plan * Replace broad except ValueError with explicit duplicate-index check in degradation_timeseries_plot Co-authored-by: cdeline <23244088+cdeline@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: cdeline <23244088+cdeline@users.noreply.github.com> * Multi yoy no label (#498) * updated degradation_timeseries_plot as version2 for now * fix flake8 grumbles * avoid pandas error "ValueError: Other Series must have a name" * Redo degradation_timeseries_plot - include in median if dt_center is within rolling window. * remove label= functionality. Update Multi-year_on_year_example.ipynb * Re-run Multi-year_on_year_example.ipynb, check for text changes. * update analysis_chains to remove `label=`. Update whatsnew * simplify multi-YoY timeseries plotting by filtering to slopes <= 2 years, removing warning. * update whatsnew * code cleanup * Revision of degradation_timeseries_plot * Bug fixes * Update notebooks to include `center=True` * Update nbval to include new notebook * add new kwarg to TrendAnalysis.plot_degradation_timeseries * Update plotting.py timeseries plot docstring. Also set pd.rolling(min_periods=rolling_days//4) * fix copilot documentation review recommendations; add plotting pytests * add min_periods_divisor argument * update pvdaq_system_4 url (#513) * update pvdaq_system_4 url * changelog * revert unnecessary changes in notebooks * update changelog * re-run notebooks --------- Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> Co-authored-by: martin-springer <martin.springer@nrel.gov> Co-authored-by: Martin Springer <97482055+martin-springer@users.noreply.github.com> * re-run TrendAnalysis nb * comments mdeceglie * update release date --------- Co-authored-by: Chris Deline <chris.deline@nrel.gov> Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: cdeline <23244088+cdeline@users.noreply.github.com> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov>




__init__.pylabel=