v0.31: Forecasting API & data annotations

Summary

Version v0.31 of FlexMeasures adds API endpoints to control forecasting and (brings back) visual annotations on time series data.

Here are other notable features:

• Schedules can now be returned in a custom resolution 
• The upload form (on the sensor page is improved): allow uploading data with different units (different resolution already supported), data can have gaps and better error handling
• Step-by-step replay in graphs
• The SwaggerUI API page can now be customized (incl. link to signup page)
• The 2FA code form (step of login) has improved UX
• JSON attributes (on assets, sensors) are now searchable (by code)
• Python3.9 has been deprecated
• We started using Copilot agents in PRs (for development)

What's Changed

• chore: upgrade dependencies for v0.31 by @nhoening in #1847
• Return meaningful results from the Train-Predict pipeline compute method by @BelhsanHmida in #1822
• Chore(deps): Bump urllib3 from 2.5.0 to 2.6.0 in requirements by @dependabot[bot] in #1850
• add OpenAPI general info & description, add two config setting for hosts by @nhoening in #1851
• Chore/no more SingleBlockManager warnings by @Flix6x in #1858
• fix: Fixed broken UI flex model fields by @joshuaunity in #1860
• Fix: duplicate beliefs by storing non-floored belief_time in forecasting pipeline by @BelhsanHmida in #1855
• feat: make listing public assets on account page optional by @nhoening in #1872
• Clean Base.html by @joshuaunity in #1868
• chore(db): Migrate JSON fields to JSONB, so they can be used to look up sensors, attributes, sources by @nhoening in #1867
• Feat/add trigger forecasting api endpoint by @BelhsanHmida in #1813
• Chore: more resilient docker compose setup by @nhoening in #1790
• Feat/less verbose error logs by @Flix6x in #1883
• add FLEXMEASURES_SIGNUP_PAGE option and improve OpenAPI intro by @nhoening in #1889
• Feat/index assets given a root node and/or depth by @Flix6x in #1874
• Fix sensor creation form by @nhoening in #1876
• Feat/add fetch forecasts endpoint by @BelhsanHmida in #1823
• Fix/multiple temperature sensors in chart by @Flix6x in #1890
• Fix/keep around finished forecasting jobs longer by @Flix6x in #1870
• add fields param to the asset-listing endpoints, controlling fields in response by @nhoening in #1884
• feat (CLI): schedule at a certain belief time by @Flix6x in #1871
• Feat/CLI scheduling dry-run by @Flix6x in #1892
• Fix/new sensors in server timezone by @Flix6x in #1894
• Two UX improvements on the 2FA form by @nhoening in #1875
• Step by Step replay graph by @joshuaunity in #1898
• Fix/delete beliefs source filter by @BelhsanHmida in #1903
• feat: allow plugins to register forecasters by @Flix6x in #1863
• Fix/include public assets on account page by @Flix6x in #1899
• Editing sensor timezone from UI by @joshuaunity in #1900
• fix: enforce canonical order before concatenation by @Flix6x in #1905
• fix: pass start and end dates to getAssetKPIs() on graphs page by @nhoening in #1909
• Add unit field for sensor data upload form by @joshuaunity in #1836
• Fix: flex-context dialogue is empty when flex-context has two booleans with the same value by @nhoening in #1907
• Bring back CLI tests to GitHub Actions by @nhoening in #778
• Docs/fix workflow and coverage badges by @Flix6x in #1913
• CI/doctests included in coverage by @Flix6x in #1914
• Include supported but un-editable flex-config fields in modal dialogues (UI editors) by @joshuaunity in #1915
• Add save buttons to flex-config modal dialogues by @joshuaunity in #1916
• Extra security layer for checking breached passwords by @joshuaunity in #1919
• Fixed broken toast message with sensor data upload form by @joshuaunity in #1921
• Fix/return validation errors instead of db errors by @Flix6x in #1923
• feat: upload allows gaps in data by @BelhsanHmida in #1918
• Fix/mutate job state when using flexmeasures jobs run-job by @Flix6x in #1929
• Fix/single job exception handler by @Flix6x in #1930
• feat: let custom schedulers return custom results by @Flix6x in #1933
• fix: simplify_index should not change the input frame inplace by @Flix6x in #1931
• Feat/show zero values in bar charts by @Flix6x in #1932
• Feat/variable peak pricing by @Flix6x in #1835
• Feat/register custom scheduler by name by @Flix6x in #1924
• Add test-specialist agent configuration for FlexMeasures testing patterns by @Copilot in #1940
• Create agent coordinator and initial specialist roster with codebase-specific domain knowledge by @Copilot in #1942
• Add YAML frontmatter to agent files for GitHub recognition by @Copilot in #1944
• fix: stroke encoding showed undefined color field by @Flix6x in #1936
• Fix: StorageScheduler to correctly combine multiple sessions targeting the same sensor by @Ahmad-Wahid in #1948
• feat: allow seeing the complete datetime in audit log by @joshuaunity in #1949
• feat: show statistics of the job queueing system by @Flix6x in #1838
• Feat/schedule resolution by @Flix6x in #1857
• feat: support storing the aggregate power schedule by @Flix6x in #1736
• Fix/sensor only fields in swagger by @Flix6x in #1951
• Agent instructions: Enforce dev environment usage and commit discipline by @Copilot in #1945
• Fix API test isolation: set fs_authn_via for Flask-Security session auth by @Copilot in #1960
• CI/setup agent test environment by @Flix6x in #1962
• Fix/syntax for copilot-setup-steps.yml by @Flix6x in #1963
• CI/agent lessons from PR 1961 by @Flix6x in #1964
• Do not support Py3.9 or mention it as the minimum requirement by @nhoening in #1965
• Add latest dependabot alert suggestions by @nhoening in #1959
• Fix/annotations on sensor page by @Flix6x in #1967
• Docs: fix the ReadTheDocs pipeline by @nhoening in #1972
• feat/default datasource types by @nhoening in #1938
• Docs/commitments section by @Flix6x in #1849
• Feat/optional training start by @Flix6x in #1917
• Feat/trigger field order by @Flix6x in #1982
• feat: allow update of parent asset on UI by @joshuaunity in #1957
• Feat: Add resolution to sensors list on asset context page by @nhoening in #1986
• Test: Add forecasting config tests by @BelhsanHmida in #1987
• Fix JSON serialization error on forecasting job pages by @Copilot in #1990
• Fix/scheduling/sign by @Flix6x in #1348

Full Changelog: v0.30.0...v0.31.0