Make source optional in the input field of the `AggregatorReporter` #819

victorgarcia98 · 2023-08-14T06:39:41Z

Description

This PR makes the source an optional field in the inputs of the AggregatorReporter. This is convenient because in many cases sensors only have one source.

I agree to contribute to the project under Apache 2 License.
To the best of my knowledge, the proposed patch is not based on code under GPL or other license that is incompatible with FlexMeasures

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

…eport-from-data-source

…eporting/make-source-optional-aggregator-reporter

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

victorgarcia98 · 2023-08-14T06:45:45Z

Should this PR include a changelog entry?

flexmeasures/data/models/reporting/aggregator.py

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

…eporting/make-source-optional-aggregator-reporter

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Flix6x

Great! Don't forget the changelog entry for 0.15.1.

flexmeasures/data/models/reporting/tests/test_aggregator.py

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

…gator-reporter

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Flix6x

The test docstring is still slightly misleading imo. And might one of the fixes apply to other reporters, too?

flexmeasures/data/models/reporting/tests/test_aggregator.py

documentation/changelog.rst

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

…eporting/make-source-optional-aggregator-reporter Signed-off-by: Victor <victor@seita.nl>

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

…tional-aggregator-reporter' into feautre/reporting/make-source-optional-aggregator-reporter

flexmeasures/data/models/reporting/tests/test_aggregator.py

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

…/make-source-optional-aggregator-reporter Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

…ws (#808) * add CostReporter with tests Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename CostReporter to profit reporter Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * improve validation Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix comments Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Add `is_profit` to `ProfitOrLossReporter` Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Call a `Reporter` saved in the DB via CLI (#818) * add DataSource field to flexmeasures add report Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add new command to create basic sources for the available reporters. Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Switch off HiGHS logs for `LOGGING_LEVEL=INFO` (#824) * add log toggling for HiGHS Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename command + add changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * use for loop in flexmeasures add sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: typo Signed-off-by: F.N. Claessen <felix@seita.nl> * refactor: make 'kind' choice singular, so it matches the key in data_generators Signed-off-by: F.N. Claessen <felix@seita.nl> * docs: add inline todo Signed-off-by: F.N. Claessen <felix@seita.nl> * refactor: get db from where we normally get it Signed-off-by: F.N. Claessen <felix@seita.nl> * style: black Signed-off-by: F.N. Claessen <felix@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl> Co-authored-by: F.N. Claessen <felix@seita.nl> * Make source optional in the input field of the `AggregatorReporter` (#819) * add DataSource field to flexmeasures add report Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: use source and make it an optional field Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix tests Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add test + fix case of multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add new command to create basic sources for the available reporters. Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set sensor in ouput bdf Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set resolution Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * update docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename command + add changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * allow passing multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * extend changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * clarify docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * check sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * use for loop in flexmeasures add sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> Signed-off-by: Victor <victor@seita.nl> * rename is_profit to profit_is_positive Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: `highs` string is in `solver_name` (#826) * fix: `highs` string is in `solve_name` using `in` Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * chore: update changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Fix: public sensors to show (#830) Fix two bugs dealing with public assets. * fix: public sensors have None owner Signed-off-by: F.N. Claessen <felix@seita.nl> * docs: changelog entry Signed-off-by: F.N. Claessen <felix@seita.nl> * fix: searching for annotations on public assets Signed-off-by: F.N. Claessen <felix@seita.nl> * refactor: more explicit if-statement Signed-off-by: F.N. Claessen <felix@seita.nl> * docs: add second fix to changelog Signed-off-by: F.N. Claessen <felix@seita.nl> --------- Signed-off-by: F.N. Claessen <felix@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl> Signed-off-by: Victor <victor@seita.nl> Co-authored-by: F.N. Claessen <felix@seita.nl> Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

…regatorReporter` (#819) * add DataSource field to flexmeasures add report Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: use source and make it an optional field Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix tests Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add test + fix case of multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add new command to create basic sources for the available reporters. Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set sensor in ouput bdf Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set resolution Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * update docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename command + add changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * allow passing multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * extend changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * clarify docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * check sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * use for loop in flexmeasures add sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> Signed-off-by: Victor <victor@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl>

* add CostReporter with tests Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename CostReporter to profit reporter Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * improve validation Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add DataSource field to flexmeasures add report Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: use source and make it an optional field Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix tests Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add test + fix case of multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Add --id and --show-attributed field to flexmeasures show data-sources. Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add sensors Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add new command to create basic sources for the available reporters. Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set sensor in ouput bdf Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set resolution Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add reporter tutorial Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * update docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename command + add changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * allow passing multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * extend changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * clarify docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix comments Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * check sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * use for loop in flexmeasures add sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Add `is_profit` to `ProfitOrLossReporter` Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Call a `Reporter` saved in the DB via CLI (#818) * add DataSource field to flexmeasures add report Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add new command to create basic sources for the available reporters. Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Switch off HiGHS logs for `LOGGING_LEVEL=INFO` (#824) * add log toggling for HiGHS Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename command + add changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * use for loop in flexmeasures add sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: typo Signed-off-by: F.N. Claessen <felix@seita.nl> * refactor: make 'kind' choice singular, so it matches the key in data_generators Signed-off-by: F.N. Claessen <felix@seita.nl> * docs: add inline todo Signed-off-by: F.N. Claessen <felix@seita.nl> * refactor: get db from where we normally get it Signed-off-by: F.N. Claessen <felix@seita.nl> * style: black Signed-off-by: F.N. Claessen <felix@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl> Co-authored-by: F.N. Claessen <felix@seita.nl> * fix gramar and typos Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add CLI changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename kind reporting to reporter Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * expand on the description of the fm add report command Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename function create_sensor_asset Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * untitlelize words Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename sensor names Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * group data sources by type Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Make source optional in the input field of the `AggregatorReporter` (#819) * add DataSource field to flexmeasures add report Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: use source and make it an optional field Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix tests Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add test + fix case of multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add new command to create basic sources for the available reporters. Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set sensor in ouput bdf Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * set resolution Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * update docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename command + add changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * allow passing multiple sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * extend changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * clarify docstring Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * check sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * use for loop in flexmeasures add sources Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> Signed-off-by: Victor <victor@seita.nl> * untitlelize sensor names Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename is_profit to profit_is_positive Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: `highs` string is in `solver_name` (#826) * fix: `highs` string is in `solve_name` using `in` Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * chore: update changelog Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Fix: public sensors to show (#830) Fix two bugs dealing with public assets. * fix: public sensors have None owner Signed-off-by: F.N. Claessen <felix@seita.nl> * docs: changelog entry Signed-off-by: F.N. Claessen <felix@seita.nl> * fix: searching for annotations on public assets Signed-off-by: F.N. Claessen <felix@seita.nl> * refactor: more explicit if-statement Signed-off-by: F.N. Claessen <felix@seita.nl> * docs: add second fix to changelog Signed-off-by: F.N. Claessen <felix@seita.nl> --------- Signed-off-by: F.N. Claessen <felix@seita.nl> * sort data sources by type Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * move changelog entries to v.0.16.0 Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * remove type column Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * use loss_is_positive instead of is_profit Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * grammar and typos Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add link to *Pandas* Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * styling: rephrasings and one more typo Signed-off-by: F.N. Claessen <felix@seita.nl> * add new dependency vl-convert-python Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * fix: equal_v out of bound Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * rename Losses to Costs Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * doc change Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * update CLI changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add warning message in the tutorial Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add changelog entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * add `a to entry Signed-off-by: Victor Garcia Reolid <victor@seita.nl> * Update toy-example-reporter.rst small typo Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> * rename attributes to asset_attributes Signed-off-by: Victor Garcia Reolid <victor@seita.nl> --------- Signed-off-by: Victor Garcia Reolid <victor@seita.nl> Signed-off-by: Victor <victor@seita.nl> Signed-off-by: F.N. Claessen <felix@seita.nl> Signed-off-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com> Co-authored-by: F.N. Claessen <felix@seita.nl> Co-authored-by: Felix Claessen <30658763+Flix6x@users.noreply.github.com>

victorgarcia98 added 6 commits August 13, 2023 20:38

add DataSource field to flexmeasures add report

292c893

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

fix: use source and make it an optional field

d53bcdb

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Merge branch 'feature/reporting/cost-reporter' into feature/cli/add-r…

0788a5f

…eport-from-data-source

Merge branch 'feature/cli/add-report-from-data-source' into feautre/r…

dd44355

…eporting/make-source-optional-aggregator-reporter

fix tests

3e48cdd

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

add test + fix case of multiple sources

0300af2

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

victorgarcia98 self-assigned this Aug 14, 2023

victorgarcia98 requested a review from Flix6x August 14, 2023 06:45

victorgarcia98 marked this pull request as ready for review August 14, 2023 06:45

Flix6x reviewed Aug 14, 2023

View reviewed changes

flexmeasures/data/models/reporting/aggregator.py Show resolved Hide resolved

flexmeasures/data/models/reporting/aggregator.py Outdated Show resolved Hide resolved

flexmeasures/data/models/reporting/aggregator.py Show resolved Hide resolved

victorgarcia98 added 2 commits August 16, 2023 15:07

add new command to create basic sources for the available reporters.

851de9c

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Merge branch 'feature/cli/add-report-from-data-source' into feautre/r…

c92e587

…eporting/make-source-optional-aggregator-reporter

victorgarcia98 requested a review from Flix6x August 16, 2023 13:19

set sensor in ouput bdf

faa1881

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

victorgarcia98 mentioned this pull request Aug 16, 2023

Validate sensor in BeliefsDataFrame constructor SeitaBV/timely-beliefs#140

Closed

Flix6x approved these changes Aug 16, 2023

View reviewed changes

flexmeasures/data/models/reporting/tests/test_aggregator.py Outdated Show resolved Hide resolved

victorgarcia98 added 4 commits August 17, 2023 11:28

set resolution

1e418b6

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

update docstring

8b27f71

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Merge branch 'main' into feautre/reporting/make-source-optional-aggre…

da6b3a4

…gator-reporter

add changelog entry

7692e94

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Flix6x reviewed Aug 18, 2023

View reviewed changes

flexmeasures/data/models/reporting/tests/test_aggregator.py Outdated Show resolved Hide resolved

documentation/changelog.rst Outdated Show resolved Hide resolved

Flix6x added this to the 0.15.1 milestone Aug 18, 2023

victorgarcia98 added 2 commits August 18, 2023 11:05

Merge branch 'main' into feature/cli/add-report-from-data-source

29c1e8c

rename command + add changelog

33a9ca2

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

victorgarcia98 changed the title ~~Make source optional in the input field of theAggregatorReporter~~ Make source optional in the input field of the AggregatorReporter Aug 18, 2023

victorgarcia98 requested a review from Flix6x August 18, 2023 11:49

victorgarcia98 added 3 commits August 18, 2023 15:27

allow passing multiple sources

9298e90

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Merge branch 'feature/cli/add-report-from-data-source' into feautre/r…

4b1e969

…eporting/make-source-optional-aggregator-reporter Signed-off-by: Victor <victor@seita.nl>

extend changelog

a595c91

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Merge remote-tracking branch 'origin/feautre/reporting/make-source-op…

5247d83

…tional-aggregator-reporter' into feautre/reporting/make-source-optional-aggregator-reporter

Flix6x reviewed Aug 18, 2023

View reviewed changes

flexmeasures/data/models/reporting/tests/test_aggregator.py Outdated Show resolved Hide resolved

clarify docstring

43a8a7e

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

victorgarcia98 requested a review from Flix6x August 18, 2023 13:45

victorgarcia98 added 2 commits August 19, 2023 00:02

check sources

8e5cb23

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

use for loop in flexmeasures add sources

a5e3606

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Base automatically changed from feature/cli/add-report-from-data-source to feature/reporting/cost-reporter August 21, 2023 07:45

Merge branch 'feature/reporting/cost-reporter' into feautre/reporting…

f92d81a

…/make-source-optional-aggregator-reporter Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Flix6x approved these changes Aug 21, 2023

View reviewed changes

Flix6x added the Still Needs Manual Backport label Aug 21, 2023

victorgarcia98 merged commit 47cce5f into feature/reporting/cost-reporter Aug 21, 2023
3 checks passed

victorgarcia98 deleted the feautre/reporting/make-source-optional-aggregator-reporter branch August 21, 2023 15:58

Flix6x removed the Still Needs Manual Backport label Aug 25, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make source optional in the input field of the `AggregatorReporter` #819

Make source optional in the input field of the `AggregatorReporter` #819

victorgarcia98 commented Aug 14, 2023 •

edited

Loading

victorgarcia98 commented Aug 14, 2023

Flix6x left a comment

Flix6x left a comment

Make source optional in the input field of the AggregatorReporter #819

Make source optional in the input field of the AggregatorReporter #819

Conversation

victorgarcia98 commented Aug 14, 2023 • edited Loading

Description

victorgarcia98 commented Aug 14, 2023

Flix6x left a comment

Choose a reason for hiding this comment

Flix6x left a comment

Choose a reason for hiding this comment

Make source optional in the input field of the `AggregatorReporter` #819

Make source optional in the input field of the `AggregatorReporter` #819

victorgarcia98 commented Aug 14, 2023 •

edited

Loading