Drop py2.7 and add 3.7 and 3.8 to testing #135

kandersolar · 2020-01-30T16:01:46Z

Testing out changes to the travis config

kandersolar · 2020-01-30T17:37:23Z

@mdeceglie @cdeline -- test failures are because requirements.txt specifies numpy==1.13.3 and pypi doesn't have numpy==1.13.3 wheels for 3.7 and 3.8 because they didn't exist when 1.13.3 was released.

NEP-29 recommends numpy 1.15 as the minimum numpy version for this point, which does have 3.7 wheels on pypi. The oldest numpy release with 3.8 wheels is 1.17.3. What do you think about bumping up the numpy version in requirements.txt to 1.17.3 so that we can test on 3.7 and 3.8? Or bumping to 1.15 and not testing on 3.8?

Also pandas 1.0.0 just got released a few hours ago and broke get_clearsky_tamb, that's why I've excluded it here.

mdeceglie · 2020-01-30T21:48:59Z

I have no objection to increasing the numpy minimum version.

kandersolar · 2020-01-30T23:01:34Z

Tests pass on 3.8 now with updated requirements.txt entries for other packages as well:

numpy==1.13.3 -> 1.17.3
pandas==0.23.0 -> 0.25.2
scipy==1.2.2 -> 1.3.2
statsmodels==0.10.1 -> 0.10.2

mdeceglie

Looks good. Thanks @kanderso-nrel

* Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst

* configure travis to only deploy on one build * prevent NaNs from summing to 0 * Update pandas requirement min_count was added as an argument for dataframe.resample().sum() in 0.22.0 * add energy_from_power() and associated tests * Update example notebook Changes in step 0: -use interpolate to resample the dataframe -use new energy_from_power() function * fix integer division bug for phython 2 * pep8 compliance * Fix docstring spelling errors * change name of series returned by energy_from_power() * Update tests for expected series name * Allow energy_from_power() with single argument * change second argument of energy_from_power() to be a Timedelta (This is more appropriate and versatile than a float with implied units) * Add series interpolation * Add dataframe input capability to interpolate_to_index() * Change function name and add frequency specification capability to interpolate() * Speed up interpolate_series() (converting to integer index under the hood is faster than pandas timestamps) * Run example with rdtools.interpolate() in step 0 * New version of energy_from_power and incorporation into normalization * require pandas >= 0.23 * Remove warnings from interpolation when max_timedelta is inferred * work around for pandas issue * Add non-workaround case for tz naive series * Change the behavior of interpolate to return NaNs where max_delta is exceeded Old behavior was to have those times be missing from the output. The new behavior ensures that interpolate produces regular time series when a frequency is passed as target. * address bug related to python 2 sting casting * Add recomendation against using interpolation for downsampling * Update notebook_requirements to address security issue in notebook version * Update copyright line of LICENSE * Update to use pvlib>0.6.0 * Add code from pv_soiling to soiling.py Includes the addition of the wrapper function soiling_srr. The manual_cleanings keyword was also removed, it was not implimented in pv_soiling. * Add confidence metric keywords and calc_info to soiling_srr() * Avoid manipulating precip input name and change pandas syntax * pass args to calc_monte() in soiling_srr() * rename infer_clean to half_norm_clean * Retain timezone in pm_frame creation * Renormalize to median of first year (per yoy degradation). Renorm factor is also included with calc_info * Keep all the monte carlo soiling profiles * import soiling_srr in __init__.py * Add more to the soiling calc_info output * enable modification of validity criteria in calc_result_frame * Add validity parameters to soiling_srr() * Clarify units of exceedance_prob in degradation_year_on_year * add type to confidence_level in degradation_year_on_year * clean up recenter type specification * Update example notebook with soiling functionality * Add axis labels to soiling plots * Add random_seed to soiling_srr for repeatable results * use true division in soiling.py * require numpy greater than 1.12 Soiling code requires numpy distributions with scale 0, see numpy PR #5822 for more information * Update notebook with note about get_clearsky() behavior change Warnings were also removed from output, and the kernel was changed to `Python [default]` * Replace hashes in soiling_test with more interpretable tests * enforce column order in test_soiling_srr() test * refactored soiling method conditionals * Add soiling rate histogram to example * Refactor soiling to single class * change irradiance-weighted to insolation-weighted in soiling output * add soiling histogram for readme * Update readme with soiling information * Rename the example notebook * Add docstring to srr_analysis.__init__() * Docstring standardization and pep8 formatting (#125) * docstring formatting cleanup for sphinx * add sphinx configuration files and index.rst * document missing parameters for get_clearsky_tamb * Add docstrings for undocumented functions. * integrate ipython notebook examples with sphinx docs * Update requirements.txt for Sphinx: m2r, nbsphinx, nbsphinx-link * need to rev h5py, numpy and scipy to avoid setup install errors. * add ipython to requirements.txt to allow inline ipython notebook examples * rollback scipy to 1.2.2 (py27 compliant) * custom .yml and requirements file for rtd * point to index.html in conf.py * test out path fixes for README.md to RST conversion * Add Changelog to sphinx docs (#134) * add sphinx.ext.extlinks to conf.py * Drop py2.7 and add 3.7 and 3.8 to testing (#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * convert README.md to ReSt in index.rst, move images folder into sphinx source * Fix formatting, fix conf.py images folder * Fix crosslinks and image size * mention soiling * another crosslink * Improve module-level docstrings (#137) * Improve module docstrings for sphinx docs module summaries * Update v2.0.0.rst * pare down readme.md * incorporate new rdtools draft mission statement * Add developer notes section to sphinx docs (#136) * Add developer notes page and update setup.py with doc reqs * improve formatting in developer_notes.rst * missed a couple formatting issues * change RTD to use setuptools extras instead of sphinx_requirements.txt * add ipython to doc requirements * change backslashes to forward slashes because they work on windows too. also remove unnecessary note about ipython * cleanup * Update v2.0.0.rst * add [test] requirements to setup.py * update developer notes with more detailed installation instructions * add code requirements section * Add coverage instructions and configuration * link typo * simplify pytest instructions * clarify installing rdtools from local git repo * Bump bleach from 2.1.3 to 3.1.1 in /docs (#149) Bumps [bleach](https://github.com/mozilla/bleach) from 2.1.3 to 3.1.1. - [Release notes](https://github.com/mozilla/bleach/releases) - [Changelog](https://github.com/mozilla/bleach/blob/master/CHANGES) - [Commits](mozilla/bleach@v2.1.3...v3.1.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * rerun `versioneer install` to fix v-prefix issue (#153) * Cherry-pick the plotting module out of the system analysis branch (#138) * cherry-pick the plotting module commit * add module-level docstring * make plotting module docstrings numpydoc-compliant * update deg/soiling example to use plotting functions * add plotting module to api.rst * fix crosslinks * add matplotlib to setup.py * update requirements.txt matplotlib for 3.8 testing * update changelog * add bins parameter to plotting.degradation_summary_plot from #132 * add basic plotting tests * exercise all kwargs in plotting tests * simplify pytest fixture; test kwargs in separate functions * missed one * fancy error messages * Clean up errant spaces * remove normalize_with_pvwatts import Co-authored-by: Michael Deceglie <michael.deceglie@nrel.gov> Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * fix typo * make introductions more approachable to non-specialists * docs cleanup * improve contributing notes * drop m2r requirement since index.rst isn't generated from readme.md anymore * remove m2r from conf.py * requested changes * hyphen * Remove low power cutoff in filtering.clipping_filter (#144) * remove low power cutoff in filtering.clipping_filter * Update v2.0.0.rst * shame on me for not running tests locally * Update degradation_and_soiling_example.ipynb * Update degradation_and_soiling_example.ipynb * change normalized filter to > 0.01 per internal discussion * Workaround for reindexing bug in Pandas 1.0.0 (#142) * tentative workaround for pandas 1.0.0 reindexing bug * Bump requirements to Pandas <= 1.0.0 for Travis upgrade-strategy=eager pytests * update pandas req to exclude 1.0.0, 1.0.1 * Revert "tentative workaround for pandas 1.0.0 reindexing bug" This reverts commit 9b69a9d. * setup.py version lists are combined with AND, so <1.0.0, >=1.0.2 wasn't valid Co-authored-by: cdeline <chris.deline@nrel.gov> * Remove conflicting reqs (#164) * Remove conflicting reqs Some packages were specified in both notebook_requirements.txt and requirements.txt. In some cases versions conflicted. * Update requirements.txt with indirect requirements * Run notebook * Add a note to first install requirements.txt * Notebook updates -Add directions to install both requirement files -Incorporate bugfix from #166 * Changelog update * Apply suggestions to changelog Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update normalization.irradiance_rescale and create irradiance_rescale_test.py (#152) * Create irradiance_rescale_test.py * pep8 spaces * NameError -> UnboundLocalError * expose convergence_threshold and default method to 'iterative' * bugfix * allow max_iterations=0 * bugfix * changelog Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * Empty soiling intervals bugfix (#169) * Closes #129 bug for invalid soiling intervals * changelog entry * Add a test to catch nan soiling interval bug * clean up comment indentation * SRR fixes (#154) * fix precipitation frequency bug * doc typo * Normalized filter cherrypick (#139) * Add normalized_filter() function to replace the mannual filter in example * update docstring * add entry to api.rst * update changelog * update lower bound to 0.01 * changelog for notebook changes * add normalized_filter test to exercise default lower bound Co-authored-by: Michael Deceglie <michael.deceglie@nrel.gov> * update soiling docstrings to mention soiling stations and aggregation (#165) * capitalize the SRR class name (#168) * rename srr_analysis to SRRAnalysis * formatting fix * changelog * Miscellaneous cleanups (#162) * remove __future__ imports * normalize_with_sapm duplicate lines * setup.py status from beta to production/stable * Delete build.bat * copyright to 2020 in license and sphinx conf * changelog for energy_from_power, normalize_with_x (#172) * changelog for energy_from_power, normalize_with_x * revise changelog * migrate to pvlib>=0.7.0 (#170) * migrate to pvlib >=0.7.0 * update SAPM normalization docstrings to mention racking_model and module_type * update sapm normalization test to specify module_type * bump docs pvlib requirement to 0.7.1 * Update rdtools/normalization.py Co-Authored-By: Cliff Hansen <cwhanse@sandia.gov> * don't use keyword names for sapm_celltemp Co-authored-by: Cliff Hansen <cwhanse@sandia.gov> * better GitHub links in documentation (#156) * tentative fix * fix slash * include branch logic based on RTD version * small improvement * omit GH link element if not building development or master * change github link text to reflect the destination branch * link changelog page to changelog GH folder instead of the template rst file * bugfix * Generalized model normalizaiton (#173) * Add normalize_with_expected_power() and refactor normalization module * make minor adjustments to normalize_with_expected_power * Add tests for normalize_with_expected_power * update changelog * update docstring Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Change test file name * code formatting fixes * simplify conditional * return energy and insolation on same index * minor changelog update * add new function to __init__.py and api.rst Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Include emacs auto-save and backup files in gitignore (#146) * Add example notebook using the NREL PVDAQ #4 system (#171) * Create degradation_and_soiling_example_sanyo.ipynb * various typos * change weather columns based on inspection; rename nz_mask to normalized_mask; improve soiling analysis * bump pandas, pvlib, and statsmodels versions in requirements.txt * add discussion; rerun notebook * rerun original notebook with new requirements to remove warnings * update pvlib links in notebooks to fix rendering on RTD * use sanyo example for the RTD page * changelog * clean up labeling * Notebook updates -Update the notebook to use a more pronounced soiling signal -Comment updates -use normalized_filter() -Use default behavior of infer_freq() and energy_from_power() * Update docs/degradation_and_soiling_example_pvdaq_4.ipynb Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update docs/degradation_and_soiling_example_pvdaq_4.ipynb Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update notebook to use new normalize_with_expected_power() * limit notebook line length to 90 characters * Update notebook with data location on datahub * add pickle to gitignore * Cache data and fix typo Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * add the logo to documentation home page (#174) * add logo to index.rst * Update logo_horizontal_highres.png * Add logo to readme (#178) * Add logo to readme * Shrink logo * update DKASC link (#180) * SRR precipitation changes (#176) * remove "clean_wo_precip", add "clean_criterion" * loop to cumulative sum * missed some changes * add min_interval_length parameter to SRR * clean up code formatting * Move clean event outage and consecutive day logic to after cleaning events are established * Add precip and shift options for clean_criterion * Add precip_threshold * Update precipitation cleaning detection logic Use a three day window only for precip_and_shift case * Update docstrings about when precipitation is used * update soiling test for precip behavior * rerun notebooks pvdaq4 notebook unaffected * break a docstring line * update the changelog * docstring typo * change log typo fix Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Review updates * splat kwargs in test * remove random_seed Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * Change interpolate max_timedelta to 2x median, add warning (#182) * change threshold, add warning, update tests * fix how exclusion warning is calculated, rerun notebooks * changelog * add license file to manifest (#183) * Overview update (#186) * Overview update I suggest slightly rephrasing the overview. * Add change to readme * Add GitHub templates and Code of Conduct (#184) * Create pull_request_template.md * add default github templates * improvements from review * Comply with pv-terms (#185) * first pass at pvterms compat * change back to `gamma_pdc` * docstring update power->energy * Capitalize Celsius * dc_power to power_dc (local variable) * modeled_irrad to irrad_sim * 1time_series` to `power` * add simultated to docstring * update test with gamma_pdc * Fix failing tests Possibly related to a pandas update rather than anything in RdTools * Update soiling attributes * pvwatts kwargs docstring update * Updated example notebooks: 'tempco' = 'gamma_pdc'; 'wind' = 'wind_speed' * Pandas > 1.0 requires explicit registering of matplotlib converters. * update normalize_with_pvwatts and clearsky_pvwatts_kws keywords. Pref-> power_dc_rated. * Add sphinx requirements to requirements.txt * Update API example in index.rst. Add #185 updates to whatsnew * Add specific kwarg values that have changed Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: cdeline <chris.deline@nrel.gov> * Update changelog with 2.0b0 release date Co-authored-by: Michael Deceglie <michael.deceglie@nrel.gov> Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Cliff Hansen <cwhanse@sandia.gov> Co-authored-by: Will Vining <wfvining@gmail.com>

* Add normalized_filter() function to replace the mannual filter in example * Initial working system analysis class * system analysis version that reproduces notebook results Works for both sensor and clearsky workflows * Improve underlying analysis These improvements slightly change the clearsky results relative to the existing version of the notebook. * Allow poa, temperature, and temperature_coefficient to be None This leaves room for BYO modeled PV perofmrance in the future * Add system_analysis to rdtools namespace * Basic example of system_analysis() use * Update degradation_and_soiling_example.ipynb 1) Use apparent zenith in irradiance calculations instead of zenith 2) Apply clearsky filter based on poa irradiance rather than insolation 3) Include ground diffuse in irradiance transposition calculations * Update system_analysis_example notebook Update notes and change initial poa calculation to include ground diffuse * Add a plotting module * Update notebook to use plotting module * Add plotting methods * update system_analysis example notebook * add docstrings * Delete model chain dev notes * add matplotlib to setup.py and update requirements * change matplotlib version * Move docstring to class * renormalization bug fix * Update temperature input interface (Also fix merge bug in plotting) * update docstring for new temperature interface * Remove __init__ docstring * update class name to CamelCase * update kwarg explanatations in docstrings * remove model parameter from calc_clearsky_poa() It is still addressable through kwargs passed to pvlib * Update SystemAnalysis_example.ipynb * Reformat dosctrings to numpy style * command style docstrings * Remove duplicate line * Update plotting docstrings to numpy style * Format normalized_filter docstring according to numpy style Other docstrings in this module have been updated in #125 * Add an ad hoc filter example to the notebook * Fixes system_analysis bug #132 when pv_nameplate is passed into syste… (#133) * Fixes system_analysis bug #132 when pv_nameplate is passed into system_analysis object. * typo fix Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * add bins parameter to plotting.degradation_summary_plot from #132 (cherry picked from commit efcbe2e) # Conflicts: # rdtools/plotting.py * plotting bug fix * Drop py2.7 and add 3.7 and 3.8 to testing (#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * add matplotlib to requirements * matplotlib 3.1 * matplotlib * merge pytests and jupyter example * PVLib > 0.7 changes to cell_temp calculation * Modelchain pytests (#196) * pytests for system_analysis. coverage: 91%. * rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates * Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys * update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts * Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error. * Add tables=3.6.1 to requirements.txt to allow clearsky analysis * add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default. * Add RdAnalysis notebook for pvdaq4 system * add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example * Sphinx release notes for 2.1.0 updated and API update Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update tests, include tables in setup.py * suppress import warnings for soiling module. * Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings(). Re-run notebooks * Model chains exp energy (#232) * enable normalize_with_expected_energy. * Update RdAnalysis_example.ipynb with manually defined power_expected option. * Add post-filter error check for < 2 yrs data per kanderson * Remove error message for no thermal model if power_expected is passed in. * remove tcell_filter from filter list if power_expected is passed in but not cell_temperature * pep8 compliance * more robust frequency check * shorten long lines * Model chains set clearsky (#233) * Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo * Explicit error message if set_clearsky not run prior to clearsky_analysis. * remove Py2.7 check in block 3 of RdAnalysis_example.ipynb * boost analysis.py test coverage to 94% * add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run. Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * improve index equality conditional * pep8 cleanup * change dict key syntax * Clarify behavior when pv_nameplate is omitted * Useful errors for plotting methods * update requreiments for tables * reduce minimum version of tables * Delete RdAnalysis_example.ipynb * Remove max_timedelta parameter use in PVDAQ example * Move sun position calculation to clearsky section of example This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both * Update chage log version number * Allow warning for experimental modules * pep8 analysis_test.py * Change module and class names * Update example with new module/class name * Fix outdated ref to system_analysis * Update api.rst with analysis chains module name * Fix changelog underline length * update index.rst with TrendAnalysis info * poa -> poa_global * cell_temperature -> temperature_cell * ambient_temperature->temperature_ambient * temperature_coefficient->gamma_pdc * temperature_model docstring * pv_nameplate->power_dc_rated * clearsky_poa->poa_global_clearsky * clearsky_temperature_cell->temperature_cell_clearsky * clearsky_temperature_ambient->temperature_ambient_clearsky * Align yoy and srr parameters with functional API * Use `case` in plotting methods * Change notebook kernel * weakly privatize some methods * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Interpolate windspeed * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * implement review suggestions * more TrendAnalysis tests to improve coverage * cleanup * Fix copy/paste error in notebook * change _calc_cell_temeprature parameter order * More elegant error if filtering results in empty series * Changelog consolidation * change log update Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov>

* Add normalized_filter() function to replace the mannual filter in example * Initial working system analysis class * system analysis version that reproduces notebook results Works for both sensor and clearsky workflows * Improve underlying analysis These improvements slightly change the clearsky results relative to the existing version of the notebook. * Allow poa, temperature, and temperature_coefficient to be None This leaves room for BYO modeled PV perofmrance in the future * Add system_analysis to rdtools namespace * Basic example of system_analysis() use * Update degradation_and_soiling_example.ipynb 1) Use apparent zenith in irradiance calculations instead of zenith 2) Apply clearsky filter based on poa irradiance rather than insolation 3) Include ground diffuse in irradiance transposition calculations * Update system_analysis_example notebook Update notes and change initial poa calculation to include ground diffuse * Add a plotting module * Update notebook to use plotting module * Add plotting methods * update system_analysis example notebook * add docstrings * Delete model chain dev notes * add matplotlib to setup.py and update requirements * change matplotlib version * Move docstring to class * renormalization bug fix * Update temperature input interface (Also fix merge bug in plotting) * update docstring for new temperature interface * Remove __init__ docstring * update class name to CamelCase * update kwarg explanatations in docstrings * remove model parameter from calc_clearsky_poa() It is still addressable through kwargs passed to pvlib * Update SystemAnalysis_example.ipynb * Reformat dosctrings to numpy style * command style docstrings * Remove duplicate line * Update plotting docstrings to numpy style * Format normalized_filter docstring according to numpy style Other docstrings in this module have been updated in #125 * Add an ad hoc filter example to the notebook * Fixes system_analysis bug #132 when pv_nameplate is passed into syste… (#133) * Fixes system_analysis bug #132 when pv_nameplate is passed into system_analysis object. * typo fix Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * add bins parameter to plotting.degradation_summary_plot from #132 (cherry picked from commit efcbe2e) * plotting bug fix * Drop py2.7 and add 3.7 and 3.8 to testing (#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * add matplotlib to requirements * matplotlib 3.1 * matplotlib * merge pytests and jupyter example * PVLib > 0.7 changes to cell_temp calculation * Modelchain pytests (#196) * pytests for system_analysis. coverage: 91%. * rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates * Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys * update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts * Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error. * Add tables=3.6.1 to requirements.txt to allow clearsky analysis * add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default. * Add RdAnalysis notebook for pvdaq4 system * add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example * Sphinx release notes for 2.1.0 updated and API update Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update tests, include tables in setup.py * suppress import warnings for soiling module. * Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings(). Re-run notebooks * Model chains exp energy (#232) * enable normalize_with_expected_energy. * Update RdAnalysis_example.ipynb with manually defined power_expected option. * Add post-filter error check for < 2 yrs data per kanderson * Remove error message for no thermal model if power_expected is passed in. * remove tcell_filter from filter list if power_expected is passed in but not cell_temperature * pep8 compliance * more robust frequency check * shorten long lines * Model chains set clearsky (#233) * Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo * Explicit error message if set_clearsky not run prior to clearsky_analysis. * remove Py2.7 check in block 3 of RdAnalysis_example.ipynb * boost analysis.py test coverage to 94% * add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run. Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * improve index equality conditional * pep8 cleanup * change dict key syntax * Clarify behavior when pv_nameplate is omitted * Useful errors for plotting methods * update requreiments for tables * reduce minimum version of tables * Delete RdAnalysis_example.ipynb * Remove max_timedelta parameter use in PVDAQ example * Move sun position calculation to clearsky section of example This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both * Update chage log version number * Allow warning for experimental modules * pep8 analysis_test.py * Change module and class names * Update example with new module/class name * Fix outdated ref to system_analysis * Update api.rst with analysis chains module name * Fix changelog underline length * update index.rst with TrendAnalysis info * poa -> poa_global * cell_temperature -> temperature_cell * ambient_temperature->temperature_ambient * temperature_coefficient->gamma_pdc * temperature_model docstring * pv_nameplate->power_dc_rated * clearsky_poa->poa_global_clearsky * clearsky_temperature_cell->temperature_cell_clearsky * clearsky_temperature_ambient->temperature_ambient_clearsky * Align yoy and srr parameters with functional API * Use `case` in plotting methods * Change notebook kernel * weakly privatize some methods * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Interpolate windspeed * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * implement review suggestions * more TrendAnalysis tests to improve coverage * cleanup * Fix copy/paste error in notebook * change _calc_cell_temeprature parameter order * More elegant error if filtering results in empty series * Changelog consolidation * change log update Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov>

* Add normalized_filter() function to replace the mannual filter in example * Initial working system analysis class * system analysis version that reproduces notebook results Works for both sensor and clearsky workflows * Improve underlying analysis These improvements slightly change the clearsky results relative to the existing version of the notebook. * Allow poa, temperature, and temperature_coefficient to be None This leaves room for BYO modeled PV perofmrance in the future * Add system_analysis to rdtools namespace * Basic example of system_analysis() use * Update degradation_and_soiling_example.ipynb 1) Use apparent zenith in irradiance calculations instead of zenith 2) Apply clearsky filter based on poa irradiance rather than insolation 3) Include ground diffuse in irradiance transposition calculations * Update system_analysis_example notebook Update notes and change initial poa calculation to include ground diffuse * Add a plotting module * Update notebook to use plotting module * Add plotting methods * update system_analysis example notebook * add docstrings * Delete model chain dev notes * add matplotlib to setup.py and update requirements * change matplotlib version * Move docstring to class * renormalization bug fix * Update temperature input interface (Also fix merge bug in plotting) * update docstring for new temperature interface * Remove __init__ docstring * update class name to CamelCase * update kwarg explanatations in docstrings * remove model parameter from calc_clearsky_poa() It is still addressable through kwargs passed to pvlib * Update SystemAnalysis_example.ipynb * Reformat dosctrings to numpy style * command style docstrings * Remove duplicate line * Update plotting docstrings to numpy style * Format normalized_filter docstring according to numpy style Other docstrings in this module have been updated in NREL#125 * Add an ad hoc filter example to the notebook * Fixes system_analysis bug NREL#132 when pv_nameplate is passed into syste… (NREL#133) * Fixes system_analysis bug NREL#132 when pv_nameplate is passed into system_analysis object. * typo fix Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * add bins parameter to plotting.degradation_summary_plot from NREL#132 (cherry picked from commit efcbe2e) * plotting bug fix * Drop py2.7 and add 3.7 and 3.8 to testing (NREL#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * add matplotlib to requirements * matplotlib 3.1 * matplotlib * merge pytests and jupyter example * PVLib > 0.7 changes to cell_temp calculation * Modelchain pytests (NREL#196) * pytests for system_analysis. coverage: 91%. * rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates * Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys * update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts * Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error. * Add tables=3.6.1 to requirements.txt to allow clearsky analysis * add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default. * Add RdAnalysis notebook for pvdaq4 system * add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example * Sphinx release notes for 2.1.0 updated and API update Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update tests, include tables in setup.py * suppress import warnings for soiling module. * Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings(). Re-run notebooks * Model chains exp energy (NREL#232) * enable normalize_with_expected_energy. * Update RdAnalysis_example.ipynb with manually defined power_expected option. * Add post-filter error check for < 2 yrs data per kanderson * Remove error message for no thermal model if power_expected is passed in. * remove tcell_filter from filter list if power_expected is passed in but not cell_temperature * pep8 compliance * more robust frequency check * shorten long lines * Model chains set clearsky (NREL#233) * Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo * Explicit error message if set_clearsky not run prior to clearsky_analysis. * remove Py2.7 check in block 3 of RdAnalysis_example.ipynb * boost analysis.py test coverage to 94% * add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run. Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * improve index equality conditional * pep8 cleanup * change dict key syntax * Clarify behavior when pv_nameplate is omitted * Useful errors for plotting methods * update requreiments for tables * reduce minimum version of tables * Delete RdAnalysis_example.ipynb * Remove max_timedelta parameter use in PVDAQ example * Move sun position calculation to clearsky section of example This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both * Update chage log version number * Allow warning for experimental modules * pep8 analysis_test.py * Change module and class names * Update example with new module/class name * Fix outdated ref to system_analysis * Update api.rst with analysis chains module name * Fix changelog underline length * update index.rst with TrendAnalysis info * poa -> poa_global * cell_temperature -> temperature_cell * ambient_temperature->temperature_ambient * temperature_coefficient->gamma_pdc * temperature_model docstring * pv_nameplate->power_dc_rated * clearsky_poa->poa_global_clearsky * clearsky_temperature_cell->temperature_cell_clearsky * clearsky_temperature_ambient->temperature_ambient_clearsky * Align yoy and srr parameters with functional API * Use `case` in plotting methods * Change notebook kernel * weakly privatize some methods * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Interpolate windspeed * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * implement review suggestions * more TrendAnalysis tests to improve coverage * cleanup * Fix copy/paste error in notebook * change _calc_cell_temeprature parameter order * More elegant error if filtering results in empty series * Changelog consolidation * change log update Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov>

* add flake8 config * bulk flake8 fixes * add linting github workflow * update GH workflow * update lint workflow job name * add flake8 to extras_require * add code style section in docs * fix unrelated docs rendering issue * tabs to spaces * whatsnew * only run on PR diffs instead of entire package * fetch all branches in the flake8 workflow * gh actions bugfix, maybe * more maybe bugfix * introduce a pep8 violation to test the flake8 check * Revert "introduce a pep8 violation to test the flake8 check" This reverts commit c78dc68. * add flake8 config * bulk flake8 fixes * add linting github workflow * update GH workflow * update lint workflow job name * add flake8 to extras_require * add code style section in docs * fix unrelated docs rendering issue * tabs to spaces * whatsnew * only run on PR diffs instead of entire package * fetch all branches in the flake8 workflow * gh actions bugfix, maybe * more maybe bugfix * introduce a pep8 violation to test the flake8 check * Revert "introduce a pep8 violation to test the flake8 check" This reverts commit c78dc68. * Model chains (#117) * Add normalized_filter() function to replace the mannual filter in example * Initial working system analysis class * system analysis version that reproduces notebook results Works for both sensor and clearsky workflows * Improve underlying analysis These improvements slightly change the clearsky results relative to the existing version of the notebook. * Allow poa, temperature, and temperature_coefficient to be None This leaves room for BYO modeled PV perofmrance in the future * Add system_analysis to rdtools namespace * Basic example of system_analysis() use * Update degradation_and_soiling_example.ipynb 1) Use apparent zenith in irradiance calculations instead of zenith 2) Apply clearsky filter based on poa irradiance rather than insolation 3) Include ground diffuse in irradiance transposition calculations * Update system_analysis_example notebook Update notes and change initial poa calculation to include ground diffuse * Add a plotting module * Update notebook to use plotting module * Add plotting methods * update system_analysis example notebook * add docstrings * Delete model chain dev notes * add matplotlib to setup.py and update requirements * change matplotlib version * Move docstring to class * renormalization bug fix * Update temperature input interface (Also fix merge bug in plotting) * update docstring for new temperature interface * Remove __init__ docstring * update class name to CamelCase * update kwarg explanatations in docstrings * remove model parameter from calc_clearsky_poa() It is still addressable through kwargs passed to pvlib * Update SystemAnalysis_example.ipynb * Reformat dosctrings to numpy style * command style docstrings * Remove duplicate line * Update plotting docstrings to numpy style * Format normalized_filter docstring according to numpy style Other docstrings in this module have been updated in #125 * Add an ad hoc filter example to the notebook * Fixes system_analysis bug #132 when pv_nameplate is passed into syste… (#133) * Fixes system_analysis bug #132 when pv_nameplate is passed into system_analysis object. * typo fix Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * add bins parameter to plotting.degradation_summary_plot from #132 (cherry picked from commit efcbe2e) * plotting bug fix * Drop py2.7 and add 3.7 and 3.8 to testing (#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * add matplotlib to requirements * matplotlib 3.1 * matplotlib * merge pytests and jupyter example * PVLib > 0.7 changes to cell_temp calculation * Modelchain pytests (#196) * pytests for system_analysis. coverage: 91%. * rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates * Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys * update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts * Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error. * Add tables=3.6.1 to requirements.txt to allow clearsky analysis * add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default. * Add RdAnalysis notebook for pvdaq4 system * add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example * Sphinx release notes for 2.1.0 updated and API update Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update tests, include tables in setup.py * suppress import warnings for soiling module. * Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings(). Re-run notebooks * Model chains exp energy (#232) * enable normalize_with_expected_energy. * Update RdAnalysis_example.ipynb with manually defined power_expected option. * Add post-filter error check for < 2 yrs data per kanderson * Remove error message for no thermal model if power_expected is passed in. * remove tcell_filter from filter list if power_expected is passed in but not cell_temperature * pep8 compliance * more robust frequency check * shorten long lines * Model chains set clearsky (#233) * Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo * Explicit error message if set_clearsky not run prior to clearsky_analysis. * remove Py2.7 check in block 3 of RdAnalysis_example.ipynb * boost analysis.py test coverage to 94% * add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run. Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * improve index equality conditional * pep8 cleanup * change dict key syntax * Clarify behavior when pv_nameplate is omitted * Useful errors for plotting methods * update requreiments for tables * reduce minimum version of tables * Delete RdAnalysis_example.ipynb * Remove max_timedelta parameter use in PVDAQ example * Move sun position calculation to clearsky section of example This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both * Update chage log version number * Allow warning for experimental modules * pep8 analysis_test.py * Change module and class names * Update example with new module/class name * Fix outdated ref to system_analysis * Update api.rst with analysis chains module name * Fix changelog underline length * update index.rst with TrendAnalysis info * poa -> poa_global * cell_temperature -> temperature_cell * ambient_temperature->temperature_ambient * temperature_coefficient->gamma_pdc * temperature_model docstring * pv_nameplate->power_dc_rated * clearsky_poa->poa_global_clearsky * clearsky_temperature_cell->temperature_cell_clearsky * clearsky_temperature_ambient->temperature_ambient_clearsky * Align yoy and srr parameters with functional API * Use `case` in plotting methods * Change notebook kernel * weakly privatize some methods * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Interpolate windspeed * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * implement review suggestions * more TrendAnalysis tests to improve coverage * cleanup * Fix copy/paste error in notebook * change _calc_cell_temeprature parameter order * More elegant error if filtering results in empty series * Changelog consolidation * change log update Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> * flake8 and pytest fixes * add beta version to change log * add missing parenthesis * revert all the development stuff * move changelog entries to 2.0.5.rst; delete pending.rst * include 2.0.5 instead of pending * Remove pending changelog Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: cdeline <chris.deline@nrel.gov>

* first cut * add a comment * add missing comma * move comments to fix broken link * Model chains (#117) * Add normalized_filter() function to replace the mannual filter in example * Initial working system analysis class * system analysis version that reproduces notebook results Works for both sensor and clearsky workflows * Improve underlying analysis These improvements slightly change the clearsky results relative to the existing version of the notebook. * Allow poa, temperature, and temperature_coefficient to be None This leaves room for BYO modeled PV perofmrance in the future * Add system_analysis to rdtools namespace * Basic example of system_analysis() use * Update degradation_and_soiling_example.ipynb 1) Use apparent zenith in irradiance calculations instead of zenith 2) Apply clearsky filter based on poa irradiance rather than insolation 3) Include ground diffuse in irradiance transposition calculations * Update system_analysis_example notebook Update notes and change initial poa calculation to include ground diffuse * Add a plotting module * Update notebook to use plotting module * Add plotting methods * update system_analysis example notebook * add docstrings * Delete model chain dev notes * add matplotlib to setup.py and update requirements * change matplotlib version * Move docstring to class * renormalization bug fix * Update temperature input interface (Also fix merge bug in plotting) * update docstring for new temperature interface * Remove __init__ docstring * update class name to CamelCase * update kwarg explanatations in docstrings * remove model parameter from calc_clearsky_poa() It is still addressable through kwargs passed to pvlib * Update SystemAnalysis_example.ipynb * Reformat dosctrings to numpy style * command style docstrings * Remove duplicate line * Update plotting docstrings to numpy style * Format normalized_filter docstring according to numpy style Other docstrings in this module have been updated in #125 * Add an ad hoc filter example to the notebook * Fixes system_analysis bug #132 when pv_nameplate is passed into syste… (#133) * Fixes system_analysis bug #132 when pv_nameplate is passed into system_analysis object. * typo fix Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * add bins parameter to plotting.degradation_summary_plot from #132 (cherry picked from commit efcbe2e) * plotting bug fix * Drop py2.7 and add 3.7 and 3.8 to testing (#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * add matplotlib to requirements * matplotlib 3.1 * matplotlib * merge pytests and jupyter example * PVLib > 0.7 changes to cell_temp calculation * Modelchain pytests (#196) * pytests for system_analysis. coverage: 91%. * rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates * Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys * update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts * Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error. * Add tables=3.6.1 to requirements.txt to allow clearsky analysis * add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default. * Add RdAnalysis notebook for pvdaq4 system * add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example * Sphinx release notes for 2.1.0 updated and API update Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update tests, include tables in setup.py * suppress import warnings for soiling module. * Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings(). Re-run notebooks * Model chains exp energy (#232) * enable normalize_with_expected_energy. * Update RdAnalysis_example.ipynb with manually defined power_expected option. * Add post-filter error check for < 2 yrs data per kanderson * Remove error message for no thermal model if power_expected is passed in. * remove tcell_filter from filter list if power_expected is passed in but not cell_temperature * pep8 compliance * more robust frequency check * shorten long lines * Model chains set clearsky (#233) * Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo * Explicit error message if set_clearsky not run prior to clearsky_analysis. * remove Py2.7 check in block 3 of RdAnalysis_example.ipynb * boost analysis.py test coverage to 94% * add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run. Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * improve index equality conditional * pep8 cleanup * change dict key syntax * Clarify behavior when pv_nameplate is omitted * Useful errors for plotting methods * update requreiments for tables * reduce minimum version of tables * Delete RdAnalysis_example.ipynb * Remove max_timedelta parameter use in PVDAQ example * Move sun position calculation to clearsky section of example This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both * Update chage log version number * Allow warning for experimental modules * pep8 analysis_test.py * Change module and class names * Update example with new module/class name * Fix outdated ref to system_analysis * Update api.rst with analysis chains module name * Fix changelog underline length * update index.rst with TrendAnalysis info * poa -> poa_global * cell_temperature -> temperature_cell * ambient_temperature->temperature_ambient * temperature_coefficient->gamma_pdc * temperature_model docstring * pv_nameplate->power_dc_rated * clearsky_poa->poa_global_clearsky * clearsky_temperature_cell->temperature_cell_clearsky * clearsky_temperature_ambient->temperature_ambient_clearsky * Align yoy and srr parameters with functional API * Use `case` in plotting methods * Change notebook kernel * weakly privatize some methods * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Interpolate windspeed * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * implement review suggestions * more TrendAnalysis tests to improve coverage * cleanup * Fix copy/paste error in notebook * change _calc_cell_temeprature parameter order * More elegant error if filtering results in empty series * Changelog consolidation * change log update Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> * add new TrendAnalysis notebook to gallery * changelog * add beta version to change log * add missing parenthesis * revert all the development stuff * move changelog entry to 2.0.5.rst; delete pending.rst * lint Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: cdeline <chris.deline@nrel.gov>

@kanderso

* Model chains (#117) * Add normalized_filter() function to replace the mannual filter in example * Initial working system analysis class * system analysis version that reproduces notebook results Works for both sensor and clearsky workflows * Improve underlying analysis These improvements slightly change the clearsky results relative to the existing version of the notebook. * Allow poa, temperature, and temperature_coefficient to be None This leaves room for BYO modeled PV perofmrance in the future * Add system_analysis to rdtools namespace * Basic example of system_analysis() use * Update degradation_and_soiling_example.ipynb 1) Use apparent zenith in irradiance calculations instead of zenith 2) Apply clearsky filter based on poa irradiance rather than insolation 3) Include ground diffuse in irradiance transposition calculations * Update system_analysis_example notebook Update notes and change initial poa calculation to include ground diffuse * Add a plotting module * Update notebook to use plotting module * Add plotting methods * update system_analysis example notebook * add docstrings * Delete model chain dev notes * add matplotlib to setup.py and update requirements * change matplotlib version * Move docstring to class * renormalization bug fix * Update temperature input interface (Also fix merge bug in plotting) * update docstring for new temperature interface * Remove __init__ docstring * update class name to CamelCase * update kwarg explanatations in docstrings * remove model parameter from calc_clearsky_poa() It is still addressable through kwargs passed to pvlib * Update SystemAnalysis_example.ipynb * Reformat dosctrings to numpy style * command style docstrings * Remove duplicate line * Update plotting docstrings to numpy style * Format normalized_filter docstring according to numpy style Other docstrings in this module have been updated in #125 * Add an ad hoc filter example to the notebook * Fixes system_analysis bug #132 when pv_nameplate is passed into syste… (#133) * Fixes system_analysis bug #132 when pv_nameplate is passed into system_analysis object. * typo fix Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * add bins parameter to plotting.degradation_summary_plot from #132 (cherry picked from commit efcbe2e) # Conflicts: # rdtools/plotting.py * plotting bug fix * Drop py2.7 and add 3.7 and 3.8 to testing (#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * add matplotlib to requirements * matplotlib 3.1 * matplotlib * merge pytests and jupyter example * PVLib > 0.7 changes to cell_temp calculation * Modelchain pytests (#196) * pytests for system_analysis. coverage: 91%. * rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates * Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys * update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts * Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error. * Add tables=3.6.1 to requirements.txt to allow clearsky analysis * add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default. * Add RdAnalysis notebook for pvdaq4 system * add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example * Sphinx release notes for 2.1.0 updated and API update Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update tests, include tables in setup.py * suppress import warnings for soiling module. * Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings(). Re-run notebooks * Model chains exp energy (#232) * enable normalize_with_expected_energy. * Update RdAnalysis_example.ipynb with manually defined power_expected option. * Add post-filter error check for < 2 yrs data per kanderson * Remove error message for no thermal model if power_expected is passed in. * remove tcell_filter from filter list if power_expected is passed in but not cell_temperature * pep8 compliance * more robust frequency check * shorten long lines * Model chains set clearsky (#233) * Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo * Explicit error message if set_clearsky not run prior to clearsky_analysis. * remove Py2.7 check in block 3 of RdAnalysis_example.ipynb * boost analysis.py test coverage to 94% * add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run. Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * improve index equality conditional * pep8 cleanup * change dict key syntax * Clarify behavior when pv_nameplate is omitted * Useful errors for plotting methods * update requreiments for tables * reduce minimum version of tables * Delete RdAnalysis_example.ipynb * Remove max_timedelta parameter use in PVDAQ example * Move sun position calculation to clearsky section of example This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both * Update chage log version number * Allow warning for experimental modules * pep8 analysis_test.py * Change module and class names * Update example with new module/class name * Fix outdated ref to system_analysis * Update api.rst with analysis chains module name * Fix changelog underline length * update index.rst with TrendAnalysis info * poa -> poa_global * cell_temperature -> temperature_cell * ambient_temperature->temperature_ambient * temperature_coefficient->gamma_pdc * temperature_model docstring * pv_nameplate->power_dc_rated * clearsky_poa->poa_global_clearsky * clearsky_temperature_cell->temperature_cell_clearsky * clearsky_temperature_ambient->temperature_ambient_clearsky * Align yoy and srr parameters with functional API * Use `case` in plotting methods * Change notebook kernel * weakly privatize some methods * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Interpolate windspeed * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * implement review suggestions * more TrendAnalysis tests to improve coverage * cleanup * Fix copy/paste error in notebook * change _calc_cell_temeprature parameter order * More elegant error if filtering results in empty series * Changelog consolidation * change log update Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> * add beta version to change log * add missing parenthesis * add TrendAnalysis notebook to gallery * good merging gone bad * fix analysis_chains_test.py * pin numexpr to 2.7.1 in requirements-min.txt * Update changelog with beta release * Fix typo in TrendAnalysis docstring (#264) * parameters -> params * changelog * Consecutive cleaning logic (#199) * Remove consecutive cleaning filter * Change day_scale default to 13 and raise warning for even values * Update soiling tests * Run notebooks * Update changelog * changelog adjustment * Update rdtools/soiling.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * keep invalid intervals at beginning and end * Revert "keep invalid intervals at beginning and end" This reverts commit b3f5efc. * keep invalid intervals at beginning abd end This revision properly hanldes the trim option * Handle consecutive invalid intervals correctly for random_clean * Remove outdated comment * Adjust trend_analysis soiling test * Add a warning for >20% invalid data and random cleaning assumptions * Run notebooks degradation_and_soiling_example.ipynb exhibits a small change in the clearsky degradation results attributed to changes in random number generator state from soiling workflow changes * patch test * Add outlier factor to adjust cleaning detection sensitivity * fix flake8 errors * Add a test for outlier_factor * add outlier_factor to change log * add #234 to change log * Add invalid interval retention at start and end to change log * Clean up review suggestions * add references to #272 * fix line length * parametrize some tests * update changelog * Update the notebooks Only some warnings changed slightly * change min_interval_length default from 2 to 7 * fix line length * Add Matt to the change log contributors Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Issue #120 Revise NaN pixel handling in clearsky_temperature.py (#274) * Revise NaN pixel handling in clearsky_temperature.py. Raise specific warning. Fixes #120 * Changelog update * rewrite cs tamb test in pytest style * add test for not-on-land Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> * Test suite warning clean-up (#273) * suppress experimental module warnings * close figs to avoid pyplot warning * use odd day_scale value * don't use deprecated check_less_precise * prevent pandas warning about average of non-numeric types * soiling warnings * interpolation warning * add notes * cleaner suppression for soiling warning * revert unnecessary changes * units cleanup (#276) * Update requirements files (#275) * bump numpy version to handle windows compat issue * bump pygments and jinja2 using dependabot reccs * add missing indirect dependencies to requirements.txt * add missing indirect dependencies to notebook_requirements.txt * changelog * partially un-bump numpy * update nbsphinx version in setup.py spatialaudio/nbsphinx#563 Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * add intersphinx so that type links work (#258) * add intersphinx so that type links work [Sphinx intersphinx](https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html) is not added by default, it must be enabled and configured. Added links for: * python * numpy * pandas * scipy this will create links for types like `float`, `dict`, _etc_. and `pandas.DataFrame`, `numpy.ndarray`, _etc_. * remove trailing single quote in doc confs Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * add pvlib intersphinx * addresses #241 * add mpl intersphinx url * fix up docstring types * more type fixes * python 3.8 -> python 3 * changelog * make soiling docstrings more consistent * fix indents in soiling docstrings * fix unexpected indent * Get the bulleted lists working in sphinx * attempt to get tuple type link working * Attempt to get type links working on TrendAnalysis * Fix trailing whitespace * type link adjustments * change from datetime to pandas timedelta in TrenAnalysis docs * Couple more link type adjustments * add type aliases * remove numpy.array Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Store filter components on TrendAnalysis (#263) * add attributes * test * changelog * unrelated linting * bug fixes * remove unnecessary default filter, test * fix up index union funny business * preliminary fix for #266 * add test to cover new warnings * improve warning messages * expand tests * delete extra blank line Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Loosen soiling input check to allow implicit daily index (#279) * remove empty line in pending.rst * use pd.infer_freq instead of checking index.freq directly * add test * add test for other warnings/errors too, for coverage * Adjust trend analysis interpolation behavior (#278) * Interpolate things onto pv index * add test for interpolation behavior * Update rdtools/test/analysis_chains_test.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Test set_clearsky interpolation * consolidate dummy series in test Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Plotting function allows data gap visualization (#282) * add 'usage_of_points' calculation to degradation.py. Seems to work ok now.. * usage of points pytest * add 'usage_of_points' to rd_result output * plotting function update add 'detailed' option, = False by default * PEP8 updates * streamline color selector with map instead of apply * add detailed=True kwarg to test_degradation_summary_plots_kwargs * Update plotting docstring. Add changelog entry. Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Update TrendAnalysis notebook to avoid filter index mismatch warning * Adjust TrendAnalysis interpolations for index mismatch (#285) * dedent non-pv inputs * dedent clearsky inputs as well * add basic test * Use interval averages for clear-sky irradiance modeling (#281) * prelim 1-min+aggregate fix * update test asserts * add manual high-res sim+aggregation to DKASC notebook * set first value to nan * update from review: backfill change * add solar_position_method kwarg to TrendAnalysis.set_clearsky * maybe one day I'll learn to test changes prior to pushing * explicitly set default method=nrel_numpy * add test for solar_position_method kwarg using pytest-mock * use method='ephemeris' in tests for speed Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Miscellaneous documentation fixes (#287) * bump sphinx_rtd_theme version * fix broken link * sphinx deprecation warning * use intersphinx for pvlib functions * more intersphinx * notebook typos * update various copyrights * Bump notebook from 6.1.5 to 6.4.1 in /docs Bumps [notebook](https://github.com/jupyter/jupyterhub) from 6.1.5 to 6.4.1. - [Release notes](https://github.com/jupyter/jupyterhub/releases) - [Changelog](https://github.com/jupyterhub/jupyterhub/blob/main/CHECKLIST-Release.md) - [Commits](https://github.com/jupyter/jupyterhub/commits) --- updated-dependencies: - dependency-name: notebook dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Added new geometric clipping filter + unit test. (#200) * Added new geometric clipping filter + unit test. * Build unit tests + update __init__.py file * Updated changelog, API.rst and index.rst to reflect newly added clipping function. * Updated filter logic. * Added experimentally derived equation for handling relation between data sampling frequency and derivative threshold cutoff. Set as a default parameter, can be overridden by the user. * Updates to logic for filter for anomaly handling. * Added function for generating interactive graphs for tuning the clipping filter. Uses plotly to generate. * Instead of omitting outliers outright, set all outliers (high and low) to 0 automatically. * Pulled the function for running the clipping filter out of the plotly function. * Added plotly to requirements.txt. * Added plotly to setp.py file. * Built the unit tests for the plotly functionality (clipping filter plotting) * Added line for running the geometric clipping filter in the ipynb notebook. * Added the tune_clip_filter_plot to the initialization statement (debugging the Travic CI build failure) * Debugging for travis CI * Debugged the unit tests for the plots per @kanderso 's recommendations * Retrying plotting builds... * Added functions for logic-based and XGBoost clipping filters. * Added the Xgboost model in the new /models folder. * Added models folder + xgboost model. * Edits to the xgboost methodology * Refactoring of logic-based method (vectorized looping). * Added scikitlearn and joblib packages as installs. * Added xgboost package as a requirement * Bug fixing for travis ci build. * move changelog entry to pending * put back lost comma * added models folder to MANIFEST file. * Updated code to fit pep8 format. * Removed comma * Changed unit test script for plotting to logic_clip_filter from geometric_clip_filter. * Fixed pep8 formatting for the unit test scripts. * Fixed pep8 formatting per @kanderso's recommendations. * Fixed pep8 issues for the SRR method. * Updated initialization script. * Whitespace removal initialization script. * Updated api.rst with the two new clipping filters. * Fixed the xgboost model load. * Added min download requirements. * Switched to pytest on the filtering unit tests. * Formatting pep8 * Fixed pep8 errors * Updated the model path to eliminate filenotfound error. * created model path. * Fixing the file path. * Adjusted file path. * Fixing whitespace issue. * Adjusted the model path...again. * Model path fix again. * Fixing directory. * Refactored to combine duplicate code. * More refactoring on methods. * Fixed test script. * Updated the script to handle older package versions. * Updates based on @kanderso-nrel's code review. * Fixed unit tests to properly reference pytest fixtures. * Fixed the clipping filter outputs to only include the boolean series. * Debugging the series naming conventions for the clipping filters using actual fleets test cases. * Added new master wrapper function for running the clipping filter + associated unit test. * Added quantile_clip_filter to initialization script. * Updated plotting unit test. * Update to plotting unit tests. * Debugging xgboost model path. * Testing model path output for unit tests. * Updated model path again? * Added Value Error unit test for incorrect kwargs passed into the master clip_filter function. * Updating the deprecation warning per @mdeceglie's suggestion on PR. Made an official DeprecationWarning. Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * Updates to include numbers package. * Updated comments for only handling AC power, per @mdeceglie's suggestion. We've used the filters on energy in the fleets pipeline, but it hasn't been rigorously tested, so let's leave it with AC power only. * Updates per @mdeceglie's review. * Troubleshooting model load for unit tests. * Fixing model import * Changed joblib version for importing model... * Updated min xgboost version. * Switched out model to update joblib version. * Updated to include scikit learn in package installs (debug per @kandersonrel's recommendation * Update to unit tests. * updated unit test again. * updates to xgboost method to handle missing time series rows (automatically not set as clipping. * Updated to 'rolling_range' from rolling derivative. * Debug unit tests * Updated to single axis tracking, per comment #200 (comment) * Unit tested exceptions in logic to up coverage * Added unit tests to increase coverage. * Updated the filter names, per Mike's request. * Updating to fit pep8 standards. * Updated the warning thrown for unit test. * Renamed plot function in init script. * Unit test for one minute intervals * Added 1 min interval coverage for XGBoost filter unit test. * Updated pytest fixture name. * Added unit tests for handling variable sampling frequency. * Updated function to handle unit tests (switched functions). * Unit test fixes. * Updated model. * Flake8 compliance * Flake8 compliance in plotting.py * Update plotting.py * Updated seconds to minutes for xgboost. * Updated the plotting coloring * Updated the unit tests based on filter updates. * Updated filters. * Update clipping filters. * Updated clipping filters + tested on the test set. * Added a warning that the clipping filters are still under development, per @mdeceglie's recommendation. * Fixing flake8 issues. * Update flake8> * More flake8 * update unit tests for new code updates * Update to unit tests for clip_filter wrapper * Update to unit tests. * Update to unit tests. * Unit tests. * Unit test. * Unit test * Unit tests * unit test debugging. * more debug * Changed dataframe to series for return in logic based clipping filter function. * Debugging datframe to series conversion. * More debugging * Additional debugging to match Matt's original filter. * Update filtering * Updated both filters based on rigorous testing. * Update xgb filter * Update to analysis chains based on push to dev last month. * updated numpy versioning to stop erroring * numpy fix * updated pandas to not get is_element error * Updated clipping threshold method for both clipping methods. * Updated jupyter notebook with new filters. * Update the python notebook for Trend Analysis. * doc string default adjustment * logic filter docstring update * update warnings Makes them more similar to those raised by other modules * Updated if-else statement to make it more concise. * Update examples of new plotting and clipping functions in the notebooks * Fixed interpolation issue for randomly sampled data for the xgboost method. * updated matt's code to handle irregular time intervals. * updated logic filter. * fix formatting issue. * Fix docstring overrides * Changed nearest to ffill for reindex. * Fixed my unit test file * Fixed the 0 on the docstring, per @kanderso's rec * Update the power_ac variable to signal, per @mdeceglie's request. * Removed trailing whitespace. * Update the scripts to handle all of the docstring issues. * Reverted naming conventions to power_ac from signal on filtering, updated to 'signal' in plotting. * tune_filter_plot docstring edit * docstring clean-up * address remaining items from review * changelog clean-up * unrestrict pandas version * Final updates from Matt on the logic-based filter. * changed from value error to index error for scrambled index on unit test. * update number of warnings thrown. * remove brackets for fixture function call in unit tests. * Updated unit test time series. * updated required min pandas installation (to calculate resample.quantile--from pandas 0.24.0 * Dropped back down min pandas version and updated +tested pd.Grouper method for pandas daily resampling with quantile. * removed brackets from unit test. * Made updates to the unit tests + doc strings per @mdeceglie's recs. * Updated the jupyter notebooks to handle the new clipping filters. Co-authored-by: Kirsten Perry <kperry@nrel.gov> Co-authored-by: kperry2215 <kirsten.perry2215@gmail.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * update tornado version in notebook_requirements.txt * add helpful hint about xgboost package * fix code style * further notebook requirements update * final notebook run * Changelog updates * gotta have the :func: * Apply suggestions from code review Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * review changes to 2.1.0 changelog * Recommit plotly plots in example * Add TrendAnalysis integration to PR template * add and apply requires_pvlib_below_090 pytest marker * bump pvlib requirement from <0.9.0 to <0.10.0 * bump pvlib min to 0.9.0; remove tables * resolve requirements-min scipy conflict * remove unnecessary [doc] requirements * set minimum pvlib back to 0.7.0 but still leave out tables except for requirements-min.txt * add warnings to sapm docstrings * changelog * more changelog * change log tables update * Update to include the DOI * Updated some of the doc strings for the clipping filters. * sort requirements * update pyzmq version * release date Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: Mark Mikofski <bwana.marko@yahoo.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Kirsten Perry <70228568+kperrynrel@users.noreply.github.com> Co-authored-by: Kirsten Perry <kperry@nrel.gov> Co-authored-by: kperry2215 <kirsten.perry2215@gmail.com>

@kanderso

* Model chains (#117) * Add normalized_filter() function to replace the mannual filter in example * Initial working system analysis class * system analysis version that reproduces notebook results Works for both sensor and clearsky workflows * Improve underlying analysis These improvements slightly change the clearsky results relative to the existing version of the notebook. * Allow poa, temperature, and temperature_coefficient to be None This leaves room for BYO modeled PV perofmrance in the future * Add system_analysis to rdtools namespace * Basic example of system_analysis() use * Update degradation_and_soiling_example.ipynb 1) Use apparent zenith in irradiance calculations instead of zenith 2) Apply clearsky filter based on poa irradiance rather than insolation 3) Include ground diffuse in irradiance transposition calculations * Update system_analysis_example notebook Update notes and change initial poa calculation to include ground diffuse * Add a plotting module * Update notebook to use plotting module * Add plotting methods * update system_analysis example notebook * add docstrings * Delete model chain dev notes * add matplotlib to setup.py and update requirements * change matplotlib version * Move docstring to class * renormalization bug fix * Update temperature input interface (Also fix merge bug in plotting) * update docstring for new temperature interface * Remove __init__ docstring * update class name to CamelCase * update kwarg explanatations in docstrings * remove model parameter from calc_clearsky_poa() It is still addressable through kwargs passed to pvlib * Update SystemAnalysis_example.ipynb * Reformat dosctrings to numpy style * command style docstrings * Remove duplicate line * Update plotting docstrings to numpy style * Format normalized_filter docstring according to numpy style Other docstrings in this module have been updated in #125 * Add an ad hoc filter example to the notebook * Fixes system_analysis bug #132 when pv_nameplate is passed into syste… (#133) * Fixes system_analysis bug #132 when pv_nameplate is passed into system_analysis object. * typo fix Co-Authored-By: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * add bins parameter to plotting.degradation_summary_plot from #132 (cherry picked from commit efcbe2e) # Conflicts: # rdtools/plotting.py * plotting bug fix * Drop py2.7 and add 3.7 and 3.8 to testing (#135) * Drop 2.7 and add 3.7 and 3.8 to testing, update docs. * creating DatetimeIndex directly is deprecated, switch to pd.date_range * require pandas < 1.0.0 * bump requirements.txt numpy to 1.17.3 for testing on py3.8 * more requirements.txt updates for py3.8 wheel availability * Update v2.0.0.rst * add matplotlib to requirements * matplotlib 3.1 * matplotlib * merge pytests and jupyter example * PVLib > 0.7 changes to cell_temp calculation * Modelchain pytests (#196) * pytests for system_analysis. coverage: 91%. * rename system_analysis.py to analysis.py. Rename class to RdAnalysis. Update pvwatts_kws to match Master updates * Allow temp_model_params to either be string or dict with 'a','b', 'deltaT' keys * update calc_cell_temperature for pvlib > 0.6.3, use energy_normalized in normalize_with_pvwatts * Warn if temp coefficient not passed into normalize_with_Pvwatts instead of exit with error. * Add tables=3.6.1 to requirements.txt to allow clearsky analysis * add tables to setup.py since it appears to be a required pvlib for the clearsky workflow and not installed by default. * Add RdAnalysis notebook for pvdaq4 system * add max_timedelta=15T and clearsky poa calculation update to pvdaq4 standard notebook example * Sphinx release notes for 2.1.0 updated and API update Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Update tests, include tables in setup.py * suppress import warnings for soiling module. * Change warning suppression from warnings.resetwarnings to with warnings.catch_warnings(). Re-run notebooks * Model chains exp energy (#232) * enable normalize_with_expected_energy. * Update RdAnalysis_example.ipynb with manually defined power_expected option. * Add post-filter error check for < 2 yrs data per kanderson * Remove error message for no thermal model if power_expected is passed in. * remove tcell_filter from filter list if power_expected is passed in but not cell_temperature * pep8 compliance * more robust frequency check * shorten long lines * Model chains set clearsky (#233) * Move CS inputs to new set_clearsky function: pvlib_location ,pv_tilt, pv_azimuth, clearsky_poa, clearsky temp, albedo * Explicit error message if set_clearsky not run prior to clearsky_analysis. * remove Py2.7 check in block 3 of RdAnalysis_example.ipynb * boost analysis.py test coverage to 94% * add pv_energy pytest case. Change to ValueError rather than basic Exception when set_clearsky hasn't been run. Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * improve index equality conditional * pep8 cleanup * change dict key syntax * Clarify behavior when pv_nameplate is omitted * Useful errors for plotting methods * update requreiments for tables * reduce minimum version of tables * Delete RdAnalysis_example.ipynb * Remove max_timedelta parameter use in PVDAQ example * Move sun position calculation to clearsky section of example This brings the clearsky results into better allignement with the objected oriented example, now sun position is calculated after interpolation on both * Update chage log version number * Allow warning for experimental modules * pep8 analysis_test.py * Change module and class names * Update example with new module/class name * Fix outdated ref to system_analysis * Update api.rst with analysis chains module name * Fix changelog underline length * update index.rst with TrendAnalysis info * poa -> poa_global * cell_temperature -> temperature_cell * ambient_temperature->temperature_ambient * temperature_coefficient->gamma_pdc * temperature_model docstring * pv_nameplate->power_dc_rated * clearsky_poa->poa_global_clearsky * clearsky_temperature_cell->temperature_cell_clearsky * clearsky_temperature_ambient->temperature_ambient_clearsky * Align yoy and srr parameters with functional API * Use `case` in plotting methods * Change notebook kernel * weakly privatize some methods * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Interpolate windspeed * Update rdtools/analysis_chains.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * implement review suggestions * more TrendAnalysis tests to improve coverage * cleanup * Fix copy/paste error in notebook * change _calc_cell_temeprature parameter order * More elegant error if filtering results in empty series * Changelog consolidation * change log update Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> * add beta version to change log * add missing parenthesis * add TrendAnalysis notebook to gallery * good merging gone bad * fix analysis_chains_test.py * pin numexpr to 2.7.1 in requirements-min.txt * Update changelog with beta release * Fix typo in TrendAnalysis docstring (#264) * parameters -> params * changelog * Consecutive cleaning logic (#199) * Remove consecutive cleaning filter * Change day_scale default to 13 and raise warning for even values * Update soiling tests * Run notebooks * Update changelog * changelog adjustment * Update rdtools/soiling.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * keep invalid intervals at beginning and end * Revert "keep invalid intervals at beginning and end" This reverts commit b3f5efc. * keep invalid intervals at beginning abd end This revision properly hanldes the trim option * Handle consecutive invalid intervals correctly for random_clean * Remove outdated comment * Adjust trend_analysis soiling test * Add a warning for >20% invalid data and random cleaning assumptions * Run notebooks degradation_and_soiling_example.ipynb exhibits a small change in the clearsky degradation results attributed to changes in random number generator state from soiling workflow changes * patch test * Add outlier factor to adjust cleaning detection sensitivity * fix flake8 errors * Add a test for outlier_factor * add outlier_factor to change log * add #234 to change log * Add invalid interval retention at start and end to change log * Clean up review suggestions * add references to #272 * fix line length * parametrize some tests * update changelog * Update the notebooks Only some warnings changed slightly * change min_interval_length default from 2 to 7 * fix line length * Add Matt to the change log contributors Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Issue #120 Revise NaN pixel handling in clearsky_temperature.py (#274) * Revise NaN pixel handling in clearsky_temperature.py. Raise specific warning. Fixes #120 * Changelog update * rewrite cs tamb test in pytest style * add test for not-on-land Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> * Test suite warning clean-up (#273) * suppress experimental module warnings * close figs to avoid pyplot warning * use odd day_scale value * don't use deprecated check_less_precise * prevent pandas warning about average of non-numeric types * soiling warnings * interpolation warning * add notes * cleaner suppression for soiling warning * revert unnecessary changes * units cleanup (#276) * Update requirements files (#275) * bump numpy version to handle windows compat issue * bump pygments and jinja2 using dependabot reccs * add missing indirect dependencies to requirements.txt * add missing indirect dependencies to notebook_requirements.txt * changelog * partially un-bump numpy * update nbsphinx version in setup.py spatialaudio/nbsphinx#563 Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * add intersphinx so that type links work (#258) * add intersphinx so that type links work [Sphinx intersphinx](https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html) is not added by default, it must be enabled and configured. Added links for: * python * numpy * pandas * scipy this will create links for types like `float`, `dict`, _etc_. and `pandas.DataFrame`, `numpy.ndarray`, _etc_. * remove trailing single quote in doc confs Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * add pvlib intersphinx * addresses #241 * add mpl intersphinx url * fix up docstring types * more type fixes * python 3.8 -> python 3 * changelog * make soiling docstrings more consistent * fix indents in soiling docstrings * fix unexpected indent * Get the bulleted lists working in sphinx * attempt to get tuple type link working * Attempt to get type links working on TrendAnalysis * Fix trailing whitespace * type link adjustments * change from datetime to pandas timedelta in TrenAnalysis docs * Couple more link type adjustments * add type aliases * remove numpy.array Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Store filter components on TrendAnalysis (#263) * add attributes * test * changelog * unrelated linting * bug fixes * remove unnecessary default filter, test * fix up index union funny business * preliminary fix for #266 * add test to cover new warnings * improve warning messages * expand tests * delete extra blank line Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Loosen soiling input check to allow implicit daily index (#279) * remove empty line in pending.rst * use pd.infer_freq instead of checking index.freq directly * add test * add test for other warnings/errors too, for coverage * Adjust trend analysis interpolation behavior (#278) * Interpolate things onto pv index * add test for interpolation behavior * Update rdtools/test/analysis_chains_test.py Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Test set_clearsky interpolation * consolidate dummy series in test Co-authored-by: Kevin Anderson <57452607+kanderso-nrel@users.noreply.github.com> * Plotting function allows data gap visualization (#282) * add 'usage_of_points' calculation to degradation.py. Seems to work ok now.. * usage of points pytest * add 'usage_of_points' to rd_result output * plotting function update add 'detailed' option, = False by default * PEP8 updates * streamline color selector with map instead of apply * add detailed=True kwarg to test_degradation_summary_plots_kwargs * Update plotting docstring. Add changelog entry. Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Update TrendAnalysis notebook to avoid filter index mismatch warning * Adjust TrendAnalysis interpolations for index mismatch (#285) * dedent non-pv inputs * dedent clearsky inputs as well * add basic test * Use interval averages for clear-sky irradiance modeling (#281) * prelim 1-min+aggregate fix * update test asserts * add manual high-res sim+aggregation to DKASC notebook * set first value to nan * update from review: backfill change * add solar_position_method kwarg to TrendAnalysis.set_clearsky * maybe one day I'll learn to test changes prior to pushing * explicitly set default method=nrel_numpy * add test for solar_position_method kwarg using pytest-mock * use method='ephemeris' in tests for speed Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * Miscellaneous documentation fixes (#287) * bump sphinx_rtd_theme version * fix broken link * sphinx deprecation warning * use intersphinx for pvlib functions * more intersphinx * notebook typos * update various copyrights * Added new geometric clipping filter + unit test. (#200) * Added new geometric clipping filter + unit test. * Build unit tests + update __init__.py file * Updated changelog, API.rst and index.rst to reflect newly added clipping function. * Updated filter logic. * Added experimentally derived equation for handling relation between data sampling frequency and derivative threshold cutoff. Set as a default parameter, can be overridden by the user. * Updates to logic for filter for anomaly handling. * Added function for generating interactive graphs for tuning the clipping filter. Uses plotly to generate. * Instead of omitting outliers outright, set all outliers (high and low) to 0 automatically. * Pulled the function for running the clipping filter out of the plotly function. * Added plotly to requirements.txt. * Added plotly to setp.py file. * Built the unit tests for the plotly functionality (clipping filter plotting) * Added line for running the geometric clipping filter in the ipynb notebook. * Added the tune_clip_filter_plot to the initialization statement (debugging the Travic CI build failure) * Debugging for travis CI * Debugged the unit tests for the plots per @kanderso 's recommendations * Retrying plotting builds... * Added functions for logic-based and XGBoost clipping filters. * Added the Xgboost model in the new /models folder. * Added models folder + xgboost model. * Edits to the xgboost methodology * Refactoring of logic-based method (vectorized looping). * Added scikitlearn and joblib packages as installs. * Added xgboost package as a requirement * Bug fixing for travis ci build. * move changelog entry to pending * put back lost comma * added models folder to MANIFEST file. * Updated code to fit pep8 format. * Removed comma * Changed unit test script for plotting to logic_clip_filter from geometric_clip_filter. * Fixed pep8 formatting for the unit test scripts. * Fixed pep8 formatting per @kanderso's recommendations. * Fixed pep8 issues for the SRR method. * Updated initialization script. * Whitespace removal initialization script. * Updated api.rst with the two new clipping filters. * Fixed the xgboost model load. * Added min download requirements. * Switched to pytest on the filtering unit tests. * Formatting pep8 * Fixed pep8 errors * Updated the model path to eliminate filenotfound error. * created model path. * Fixing the file path. * Adjusted file path. * Fixing whitespace issue. * Adjusted the model path...again. * Model path fix again. * Fixing directory. * Refactored to combine duplicate code. * More refactoring on methods. * Fixed test script. * Updated the script to handle older package versions. * Updates based on @kanderso-nrel's code review. * Fixed unit tests to properly reference pytest fixtures. * Fixed the clipping filter outputs to only include the boolean series. * Debugging the series naming conventions for the clipping filters using actual fleets test cases. * Added new master wrapper function for running the clipping filter + associated unit test. * Added quantile_clip_filter to initialization script. * Updated plotting unit test. * Update to plotting unit tests. * Debugging xgboost model path. * Testing model path output for unit tests. * Updated model path again? * Added Value Error unit test for incorrect kwargs passed into the master clip_filter function. * Updating the deprecation warning per @mdeceglie's suggestion on PR. Made an official DeprecationWarning. Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> * Updates to include numbers package. * Updated comments for only handling AC power, per @mdeceglie's suggestion. We've used the filters on energy in the fleets pipeline, but it hasn't been rigorously tested, so let's leave it with AC power only. * Updates per @mdeceglie's review. * Troubleshooting model load for unit tests. * Fixing model import * Changed joblib version for importing model... * Updated min xgboost version. * Switched out model to update joblib version. * Updated to include scikit learn in package installs (debug per @kandersonrel's recommendation * Update to unit tests. * updated unit test again. * updates to xgboost method to handle missing time series rows (automatically not set as clipping. * Updated to 'rolling_range' from rolling derivative. * Debug unit tests * Updated to single axis tracking, per comment #200 (comment) * Unit tested exceptions in logic to up coverage * Added unit tests to increase coverage. * Updated the filter names, per Mike's request. * Updating to fit pep8 standards. * Updated the warning thrown for unit test. * Renamed plot function in init script. * Unit test for one minute intervals * Added 1 min interval coverage for XGBoost filter unit test. * Updated pytest fixture name. * Added unit tests for handling variable sampling frequency. * Updated function to handle unit tests (switched functions). * Unit test fixes. * Updated model. * Flake8 compliance * Flake8 compliance in plotting.py * Update plotting.py * Updated seconds to minutes for xgboost. * Updated the plotting coloring * Updated the unit tests based on filter updates. * Updated filters. * Update clipping filters. * Updated clipping filters + tested on the test set. * Added a warning that the clipping filters are still under development, per @mdeceglie's recommendation. * Fixing flake8 issues. * Update flake8> * More flake8 * update unit tests for new code updates * Update to unit tests for clip_filter wrapper * Update to unit tests. * Update to unit tests. * Unit tests. * Unit test. * Unit test * Unit tests * unit test debugging. * more debug * Changed dataframe to series for return in logic based clipping filter function. * Debugging datframe to series conversion. * More debugging * Additional debugging to match Matt's original filter. * Update filtering * Updated both filters based on rigorous testing. * Update xgb filter * Update to analysis chains based on push to dev last month. * updated numpy versioning to stop erroring * numpy fix * updated pandas to not get is_element error * Updated clipping threshold method for both clipping methods. * Updated jupyter notebook with new filters. * Update the python notebook for Trend Analysis. * doc string default adjustment * logic filter docstring update * update warnings Makes them more similar to those raised by other modules * Updated if-else statement to make it more concise. * Update examples of new plotting and clipping functions in the notebooks * Fixed interpolation issue for randomly sampled data for the xgboost method. * updated matt's code to handle irregular time intervals. * updated logic filter. * fix formatting issue. * Fix docstring overrides * Changed nearest to ffill for reindex. * Fixed my unit test file * Fixed the 0 on the docstring, per @kanderso's rec * Update the power_ac variable to signal, per @mdeceglie's request. * Removed trailing whitespace. * Update the scripts to handle all of the docstring issues. * Reverted naming conventions to power_ac from signal on filtering, updated to 'signal' in plotting. * tune_filter_plot docstring edit * docstring clean-up * address remaining items from review * changelog clean-up * unrestrict pandas version * Final updates from Matt on the logic-based filter. * changed from value error to index error for scrambled index on unit test. * update number of warnings thrown. * remove brackets for fixture function call in unit tests. * Updated unit test time series. * updated required min pandas installation (to calculate resample.quantile--from pandas 0.24.0 * Dropped back down min pandas version and updated +tested pd.Grouper method for pandas daily resampling with quantile. * removed brackets from unit test. * Made updates to the unit tests + doc strings per @mdeceglie's recs. * Updated the jupyter notebooks to handle the new clipping filters. Co-authored-by: Kirsten Perry <kperry@nrel.gov> Co-authored-by: kperry2215 <kirsten.perry2215@gmail.com> Co-authored-by: Kevin Anderson <kevin.anderson@nrel.gov> Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> * add 3.10 to pytest matrix * quote versions so 3.10 isn't interpreted as 3.1 * don't use requirements-min on py3.10 * update most of requirements.txt for py3.10 * drop 3.6 from test matrix, use requirements-min with 3.7 instead * bump requirements-min to versions that have py37-compatible wheels * bump tables version again * mirror new minimum dep versions in setup.py * update various references to supporting python versions * create v2.1.2 changelog file * create v2.1.3 changelog * bump scikit-learn to 1.0.2 in requirements.txt * more changelog * use more robust warning testing * tweaks * switch to pure pyplot calls * rerun availability nb * changelog * add testing section to changelog * fix erroneous comment Co-authored-by: Michael Deceglie <mdeceglie@users.noreply.github.com> Co-authored-by: cdeline <chris.deline@nrel.gov> Co-authored-by: Michael Deceglie <Michael.Deceglie@nrel.gov> Co-authored-by: Mark Mikofski <bwana.marko@yahoo.com> Co-authored-by: Kirsten Perry <70228568+kperrynrel@users.noreply.github.com> Co-authored-by: Kirsten Perry <kperry@nrel.gov> Co-authored-by: kperry2215 <kirsten.perry2215@gmail.com>

kandersolar added 3 commits January 30, 2020 08:55

Drop 2.7 and add 3.7 and 3.8 to testing, update docs.

d430324

creating DatetimeIndex directly is deprecated, switch to pd.date_range

fc19191

require pandas < 1.0.0

38be734

mdeceglie mentioned this pull request Jan 30, 2020

Add Changelog to sphinx docs #134

Merged

kandersolar added 2 commits January 30, 2020 15:31

bump requirements.txt numpy to 1.17.3 for testing on py3.8

dacee0e

more requirements.txt updates for py3.8 wheel availability

1860e06

kandersolar marked this pull request as ready for review January 31, 2020 18:58

mdeceglie approved these changes Jan 31, 2020

View reviewed changes

kandersolar added 2 commits January 31, 2020 14:35

Merge branch 'development' into drop_py27

fc67f76

Update v2.0.0.rst

31184ce

kandersolar merged commit b1b3528 into development Jan 31, 2020

kandersolar added this to the Version 2 milestone Mar 3, 2020

kandersolar deleted the drop_py27 branch March 9, 2020 20:08

kandersolar mentioned this pull request Mar 26, 2020

Update requirements #158

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Drop py2.7 and add 3.7 and 3.8 to testing #135

Drop py2.7 and add 3.7 and 3.8 to testing #135

kandersolar commented Jan 30, 2020

kandersolar commented Jan 30, 2020

mdeceglie commented Jan 30, 2020

kandersolar commented Jan 30, 2020

mdeceglie left a comment

Drop py2.7 and add 3.7 and 3.8 to testing #135

Drop py2.7 and add 3.7 and 3.8 to testing #135

Conversation

kandersolar commented Jan 30, 2020

kandersolar commented Jan 30, 2020

mdeceglie commented Jan 30, 2020

kandersolar commented Jan 30, 2020

mdeceglie left a comment

Choose a reason for hiding this comment