add integrations for testing, documentation, linting #3
Labels
infrastructure
integrations, packaging, etc.
Milestone
Comments
|
In terms of CI/testing what should we use? Options include travis (full VM, ubuntu), appveyor (VM, windows, linux), circle-ci (container, linux, mac), azure pipelines (container or vm, windows, linux, mac). This package will primarily run in linux containers, so maybe start there. |
|
I heard someone say good things about azure pipelines somewhere. Travis is well known and dependable. Appveyor is annoying in pvlib. I prefer codecov to coveralls, but not strongly. I've been happy enough with stickler for flake8 in pvlib. |
wholmgren
changed the title
|
I have |
|
in a |
lboeman
pushed a commit
to lboeman/solarforecastarbiter-core
that referenced
this issue
Jan 7, 2020
* start adjust serialization * fix and text regex * update docstrings
alorenzo175
added a commit
that referenced
this issue
Jan 9, 2020
* make metricresult * update figures using view on one cds * add func to export script, div, svg to plot obj * more report gen * add metric block * more simpler * adjust serialization * adjust templates * further simplify * make more general report figure * add html total table * remove old templates * adj timezone * add javascript to download metrics cds as csv * better dwnld * create a metadata cds, link fxobs pairs to value cds using integer index (#1) * create a metadata cds, link fxobs pairs to value cds using integer index * use numpy instead of list comprehension to calculate indices for filter * move reference forecast to ProcessedForecastObservation * add other likely parameters to datamodel * add validation result, valid point count to processed forecast observation * refactor preprocessing (#2) * move and refactor ProcessedForecastObservation generation * fix name change * update whatsnew * fix underline * proof of concept using python logger to get report messages * add schema to timeseries serialization (#3) * start adjust serialization * fix and text regex * update docstrings * make calculator work with ForecastAggregate objects * fix api tests * adjust logging of errors * add name for processedfxobs * return localized empty series/frame from api * test preprocessing and remove unnecessary interval length check * fix tests * remove old tests/add skeleton to test report main * Figure testing (#4) * test figures.py * don't set timezone in dummy test data * correct type of metrics attribute from list to tuple * add new tests for datamodel * remove dead code * datamodel test coverage * flake8, little more coverage * add test to ensure behavior from _special_field_processing remains * tweak datamodel testing * separate basefilter from other filter tests, test cds creation without expected data * remove basefilter skip logic * correct model type in basefilter test Co-authored-by: Tony Lorenzo <atlorenzo@email.arizona.edu> * fix calculator tests * adjust hour range * update bokeh requirement, skip tests w/ selenium * flake8, test fix Co-authored-by: Leland Boeman <lboeman@email.arizona.edu>
alorenzo175
added a commit
that referenced
this issue
Feb 3, 2020
* start Refactor reports (#291) * make metricresult * update figures using view on one cds * add func to export script, div, svg to plot obj * more report gen * add metric block * more simpler * adjust serialization * adjust templates * further simplify * make more general report figure * add html total table * remove old templates * adj timezone * add javascript to download metrics cds as csv * better dwnld * create a metadata cds, link fxobs pairs to value cds using integer index (#1) * create a metadata cds, link fxobs pairs to value cds using integer index * use numpy instead of list comprehension to calculate indices for filter * move reference forecast to ProcessedForecastObservation * add other likely parameters to datamodel * add validation result, valid point count to processed forecast observation * refactor preprocessing (#2) * move and refactor ProcessedForecastObservation generation * fix name change * update whatsnew * fix underline * proof of concept using python logger to get report messages * add schema to timeseries serialization (#3) * start adjust serialization * fix and text regex * update docstrings * make calculator work with ForecastAggregate objects * fix api tests * adjust logging of errors * add name for processedfxobs * return localized empty series/frame from api * test preprocessing and remove unnecessary interval length check * fix tests * remove old tests/add skeleton to test report main * Figure testing (#4) * test figures.py * don't set timezone in dummy test data * correct type of metrics attribute from list to tuple * add new tests for datamodel * remove dead code * datamodel test coverage * flake8, little more coverage * add test to ensure behavior from _special_field_processing remains * tweak datamodel testing * separate basefilter from other filter tests, test cds creation without expected data * remove basefilter skip logic * correct model type in basefilter test Co-authored-by: Tony Lorenzo <atlorenzo@email.arizona.edu> * fix calculator tests * adjust hour range * update bokeh requirement, skip tests w/ selenium * flake8, test fix Co-authored-by: Leland Boeman <lboeman@email.arizona.edu> * datamodel.__check__metrics(): support probabilistic forecasts (#306) * datamodel.__check__metrics__(): support prob fx Update to support checking metrics for evaluating probabilistic forecasts. * datamodel: fix __check_metrics__() and tests * flake8 corrections (for line length) * Add note to `docs/whatsnew` * Remove unnecessary `print()` * tests: correct typo ("strict", not "stric") * Spelling typos and use `if fx in <list>` * Forgot `type()` around `fx` * flake8: line indentation * Try `isinstance` with prob fx, then det fx check * Switch to nest if-if statement * add reports template.py testing (#305) * add reports template.py testing * flake8 * remove test of download script macro * Docstrings (#307) * add reports template.py testing * update figures, template docstring * add parameters to non-private datamodel objects, add sphinx decorators * add reports modules to docs * add reports api changes to whatsnew * flake8 * correct docstrings * remove data_object from ForecastAggregate docstring Co-authored-by: Tony Lorenzo <atlorenzo@email.arizona.edu> * fix scatter limits w/ nans closes #282 (#311) * Log errors on failed SVG export (#313) * Log errors on failed SVG export closes #310 * update api doc * try installing selenium and pandoc in azure * alter phantom path * use npm to install phantomjs * fix install cmd * metrics: test timezone behavior (#314) * metrics/preprocessing: test resample with timezone diff Test `resample_and_align()` to ensure that the local timezone conversion is correct when there is a difference between the local timezone (which will be taken from the `ReportMetaData` object) and the timezone of the fx/obs pair. * metrics/calculator/tests: test timezone changes Revise the tests to also check that the metrics per category respect the timezone of the report (e.g. if the fx/obs are originally in Central time, but the report is in Pacific, then the RMSE per hour of the day should be given with the hours converted to Pacific time). Note: the timezone is provided via a `ReportMetaData` object, which is used when calling the `metrics` code. * add checksum function to utils and add field to report metadata (#316) * Data Preprocessing Results and Validation Tracking and Reports (#320) * Fix typo in error message in quality mapping check if flagged function * Added ValidationResults in metrics.preprocessing's process_forecast_observations function for Observations Unavailable (after apply validation), Observations Ignored from Resampling and Alignment, Forecasts Ignored from Alignment, Total Observations Ignored * Fixed problem in metric.process_forecast_observations when using ForecastAggregate * Added new PreprocessingResult datamodel to store preprocessing results and added dictionary of results for forecast and observation during preprocessing.resample_and_align to record undefined/nan values and values lost during alignment * flake8 updates * flake8 changes for noqa for readability * Fixed an issue with the TOTAL FLAGGED VALUES DISCARDED PreprocessingResult if validation fails and setting the count to -1 to indicate failure in that case * Fix to merge with 1.0beta4 * Adding category blurbs constants in datamodels and adding them to the rendering * Adding validation with quality flag table and some other aesthetic changes to the total metrics table * Added data preprocessing section and table to report html body * Updated 1b4 whatsnew with preprocessing and validation report details * Added PreprocessingResult to api.rst * Updated test macro for metrics table * Added rendering tests for preprocessing and validation result tables and added empty PreprocessingResult and ValidationResult to the conftest RawReport * flake8 modifications * Added category blurbs to expected kwargs in template test * Added tests in preprocess_forecast_observations to check types for ValidationResult and PreprocessingResult and few other name/bug fixes * Adding PR number 320 to relevant 1.0beta4 whatsnew * Added missing space in the preprocessing result list fixture * Fixed a bug with the PreprocessingResult list for multiple forecast-observation pairs in metrics.preprcessing.process_forecast_observations * Created separate metric's tables for displaying forecasts in the vertical columns (total only right now) and horizontal columns (all other categories) including separate tests * Add missing report tests (#318) * add status kwarg to post_raw_report * add report tests closes #312 * make figures work with empty metrics * flake8 * fix listhandler setup * fix __check_units__ with no args * Slight modification of Report (#324) * refactor datamodel for report_parameters * report api alterations * adjust figures * other updates to new reportstruct * update docs * add infer timezone from agg test * sort version keys * add bar_subdivisions fig name * fix issues w/ rebasing * fix json encoding error on numpy int * template failed reports and report dicts (#325) * make versions a tuple instaed of dict * make sure the failed raw report can be loaded into datamodel obj * support templating failed reports * add auto reload to pending page * add bad status test * add selenium to requirements * adjust failure message * make sure deserialize timeseries is localized * add docstring * use figure_type * clarify report cli * remove pyarrow requirement * move hijack loggers to utils * remove redundant report_to_html_body * mostly docstring updates * flake8 Co-authored-by: Leland Boeman <lboeman@email.arizona.edu> Co-authored-by: David P. Larson <1169109+dplarson@users.noreply.github.com> Co-authored-by: Adam Wigington <ajwigington@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Probably want the following integrations:
The text was updated successfully, but these errors were encountered: