1.0.0b5.rst

1.0.0b5 (April 27, 2020)

This is the fifth 1.0 beta release.

Highlights include:

• support for event forecasts and reports
• using plotly to generate report figures
• support for normalized metrics in the reports
• an option to account for uncertainty using a deadband in reports
• support for forecast skill in reports

See below for the full list of changes.

API Changes

• Added :pysolarforecastarbiter.reports.figures which contains modules for creating Bokeh and Plotly report figures.
• Added :pysolarforecastarbiter.reports.figures.plotly_figures to create report figures using plotly (359)(388)
• :pysolarforecastarbiter.reports.figures.bokeh_figures.output_svg now takes a :pyselenium.webdriver.remote.webdriver.WebDriver as an optional argument (345)
• Add metrics for deterministic event forecasts :pysolarforecastarbiter.metrics.event.probability_of_detection, :pysolarforecastarbiter.metrics.event.false_alarm_ratio, :pysolarforecastarbiter.metrics.event.probability_of_false_detection, :pysolarforecastarbiter.metrics.event.critical_success_index, :pysolarforecastarbiter.metrics.event.event_bias, and :pysolarforecastarbiter.metrics.event.event_accuracy. (347) (348)
• Add functions to get the time range of values available in the API: :pysolarforecastarbiter.io.api.get_observation_time_range, :pysolarforecastarbiter.io.api.get_forecast_time_range, :pysolarforecastarbiter.io.api.get_probabilistic_forecast_constant_value_time_range. (369)
• Start and end are now optional parameters for the solararbiter referencedata update CLI command (369)
• :pysolarforecastarbiter.vavlidation.tasks.validate_timestamp has been replaced with :pysolarforecastarbiter.vavlidation.tasks.validate_defaults which also adds a nighttime flag (382)
• Removed normalizer keyword argument from :pysolarforecastarbiter.metrics.calculate_metrics and :pysolarforecastarbiter.metrics.calculate_deterministic_metrics. Normalization is now determined by the attributes of the datamodel objects. (379)
• Many functions in :pysolarforecastarbiter.metrics.deterministic gain a keyword argument error_fnc for specifying a function that calculates the nominal difference forecast - observation. Among other things, this is useful for specifying a deadband. (378)
• Added support for probabilistic forecasts with :pysolarforecastarbiter.metrics.calculator.calculate_metrics modifications and :pysolarforecastarbiter.metrics.calculator.calculate_probabilistic_metrics addition. (315) (266) (367)
• Added api function to get all values of a :pysolarforecastarbiter.datamodel.ProbabilisticForecast with addition of :pysolarforecastarbiter.io.api.get_probabilistic_forecast_values. (367)
• Removed ref_pair from :pysolarforecastarbiter.metrics.calculator.calculate_metrics and ref_fx_obs from :pysolarforecastarbiter.metrics.calculator.calculate_deterministic_metrics and :pysolarforecastarbiter.metrics.calculator.calculate_probabilistic_metrics. Instead, use the reference forecast attributes of :pysolararbiter.ForecastObservation and :pysolararbiter.ProcessedForecastObservation to supply reference forecast metadata and values. (386, 387)
• :pysolarforecastarbiter.metrics.preprocessing.resample_and_align now requires input reference forecast time series (optionally None), and also returns processed reference forecast time series (or None). (386, 387)
• Added capability to import reference data from DOE ARM sites in the :pysolarforecastarbiter.io.reference_observations.arm module. Data can be initialized and updated via the Solar Forecast Arbiter CLI referencedata command.(390)
• Added support for event observations and forecasts with :pysolarforecastarbiter.datamodel.Observation modifications and :pysolarforecastarbiter.datamodel.EventForecast addition. (357) (361) (399)
• Added support for evaluating event forecasts with :pysolarforecastarbiter.metrics.calculator.calculate_metrics modification, :pysolarforecastarbiter.metrics.calculator.calculate_event_metrics addition and :pysolarforecastarbiter.metrics.preprocessing.resample_and_align modifications. (357) (380) (361)
• Added support for plotting timeseries of event observations and forecasts to :pysolararbiter.plotting.timeseries.make_basic_timeseries. (394)

Enhancements

• Report plots are now created using Plotly. Bokeh figure generation moved to :pysolarforecastarbiter.reports.figures.bokeh_figures for users still interested in bokeh figures.(359)
• Report timeseries plots no longer draw lines over missing data points (351).
• Double clicking a forecast or observation in the legend of report timeseries plots will toggle the display of all other timeseries. (253)
• Clean up any PhantomJS drivers created to render SVGs (344) (349)
• Reference data updates can now proceed from the last value in the API to avoid any artificial data gaps (369) (353)
• GHI validation now includes checks on if a period is cloud-free and will be flagged with CLEARSKY (210) (382)
• All data validation now includes the generation of the NIGHTTIME flag (293) (382)
• Metrics automatically determine normalization based on Observation type. AC power observations are normalized by AC capacity; DC power by DC capacity. Normalized metrics set to nan for all other variables. (370) (379)
• Metrics 'mae', 'mbe', 'rmse', 'mape', 'nmae', 'nmbe', 'nrmse', 's' may now be calculated using a deadband. The deadband is specified as a percentage of the observations. The error forecast - observation is set to 0 within the deadband. The deadband is controlled using the :pysolarforecastarbiter.datamodel.ForecastObservation and :pysolarforecastarbiter.datamodel.ForecastAggregate uncertainty argument. None implies no deadband and a float sets the deadband. Additionally, for :pysolarforecastarbiter.datamodel.ForecastObservation, the string 'observation_uncertainty' may be supplied to set the deadband equal to :pysolarforecastarbiter.datamodel.Observation.uncertainty. (358, 378)
• Reports can now report the forecast skill metric (s) when a reference forecast is provided. The metadata table of forecasts and observations now includes reference forecasts. (386, 387)
• Reports now support evaluating event forecasts. (380) (361)

Bug fixes

• Fix using deprecated 'M' string for a month offset in :pysolarforecastarbiter.io.reference_observaions.srml.fetch (341) (342)
• Fix CLI report generation when status not set in report metadata (345)
• Fix bug with APISession list functions when only a single value is available. (367)
• Fix type errors in observation validation by ensuring all time series values from :pysolarforecastaribter.io.api functions are coerced to float and converting to float before validation (252) (384)
• Enforce metric order consistency. (352) (396)
• Eliminate most warnings from test suite. (385) (395)
• :pysolarforecastarbiter.metrics.deterministic.forecast_skill now returns 0 if both forecast and reference forecast errors are 0. (395)
• Clarify report data validation section text and table. (413) (422)

Contributors

• Will Holmgren (wholmgren)
• Leland Boeman (lboeman)
• Cliff Hansen (cwhanse)
• Tony Lorenzo (alorenzo175)
• Justin Sharp (MrWindAndSolar)
• Aidan Tuohy
• Adam Wigington (awig)
• David Larson (dplarson)