642 add report cli command#659

Merged

Flix6x merged 100 commits intomainfrom

642-add-report-cli-command-2

May 29, 2023

Contributor

victorgarcia98 commented May 2, 2023

This PR introduces the command flexmeasures add report which allows to create reports (custom calculations defined in classes of type Reporter) and save them to the database or export them to Excel or CSV. It also includes some flags to define the time scope of the report in some predefined ways: --last-hour, --last-day, ...

Closes #642.

victor and others added 30 commits

April 14, 2023 10:50


          Creating Reporter and PandasReporter classes with their corresponding…

32e300c

… report_config schemas.

Signed-off-by: victor <victor@seita.nl>


          Added Tibber Reporter.

ce17568

Signed-off-by: victor <victor@seita.nl>


          - Fixing wong DA Price value.

848e39a

- Renaming BWV and EB to english words.
- Simplifying calculation (pandas pipeline).
- Adding units to sensors.
- Changing units from EUR/kWh to EUR/MWh
- Adding assert to check maximum error
- deserialize_report_config -> deserialize_reporter_config
- Warning when a string starting with  `@` is used in the method query or eval.
- Making process_pandas_args, process_pandas_kwargs and apply_transformation private methods.

Signed-off-by: victor <victor@seita.nl>


          Updating VAT units.

d9867fc

Signed-off-by: victor <victor@seita.nl>


          - Attatching report to sensor

1c9de43

- Output type of compute is BeliefDataFrame
- Added a global input resolution to schem
- ISO datetime and timedeltas
- start, end and input_resolution are considered serialized when passed to the method compute
- assert to check that result resolution = sensor resolution

Signed-off-by: victor <victor@seita.nl>


          Fixing wrong arguments to search_beliefs method.

f61825b

Signed-off-by: victor <victor@seita.nl>


          FIxing wrong type conversion logic.

b01f5c3

Signed-off-by: victor <victor@seita.nl>


          Small reporter fixes (#647)

487233c

* No return value

Signed-off-by: F.N. Claessen <felix@seita.nl>

* typo

Signed-off-by: F.N. Claessen <felix@seita.nl>

* plural

Signed-off-by: F.N. Claessen <felix@seita.nl>

* indentation

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix return type annotations

Signed-off-by: F.N. Claessen <felix@seita.nl>

* format docstring example

Signed-off-by: F.N. Claessen <felix@seita.nl>

* add type annotations

Signed-off-by: F.N. Claessen <felix@seita.nl>

* predefine instance attributes

Signed-off-by: F.N. Claessen <felix@seita.nl>

* remove redundant variable

Signed-off-by: F.N. Claessen <felix@seita.nl>

* grammar

Signed-off-by: F.N. Claessen <felix@seita.nl>

* support aliases

Signed-off-by: F.N. Claessen <felix@seita.nl>

* grammar/typos

Signed-off-by: F.N. Claessen <felix@seita.nl>

* test exact match

Signed-off-by: F.N. Claessen <felix@seita.nl>

---------

Signed-off-by: F.N. Claessen <felix@seita.nl>


          Add superclass to Reporter that will be common to all three data gene…

f34f5e1

…rator classes: Reporter, Scheduler, Forecaster.

Signed-off-by: victor <victor@seita.nl>


          Merge remote-tracking branch 'origin/626-add-reporter-class' into 626…

1f8457e

…-add-reporter-class


          Add start, end, resolution, beliefs_after and beliefs_before to the `…

dd21c99

…compute` method signature.

Signed-off-by: victor <victor@seita.nl>


          Add FLEXMEASURES_DEFAULT_DATASOURCE config to be the feault datasourc…

58d405f

…e for data generators.

Signed-off-by: victor <victor@seita.nl>


          Fixing wrong input type.

bebc320

Signed-off-by: victor <victor@seita.nl>


          Rename DataGenerator class to DataGeneratorMixin

0dfe5f4

Signed-off-by: victor <victor@seita.nl>


          Reduce logging level from warning to debug.

7830e33

Signed-off-by: victor <victor@seita.nl>


          Merge branch 'main' into 626-add-reporter-class

e378ac6

Signed-off-by: victor <victor@seita.nl>


          Merge branch 'main' into 626-add-reporter-class


          Register Reporter to the app context.

6ab84df

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Allowing to use BeliefsDataFrame specific method in the schema.

79fe12f

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Merge remote-tracking branch 'origin/626-add-reporter-class' into 626…

a64550b

…-add-reporter-class


          Fixed wrong method. TODO: test with a plugin.

4c86878

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Using module name instead of the module object.

5ca52eb

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          use belief_time instead of beliefs_before and beliefs_after (#652)

12267d3

Signed-off-by: F.N. Claessen <felix@seita.nl>


          Merge remote-tracking branch 'origin/626-add-reporter-class' into 626…

723ed02

…-add-reporter-class


          Merge branch 'main' into 626-add-reporter-class

5a6590e


          Fixing example.

cc47742

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Fixing grammar.

cc11809

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Require at least 1 input sensor for the tb_query_config.

c50df17

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Merge branch 'main' into 626-add-reporter-class

cc8aa22


          Bug fix: compute function was overriding the variables to the default…

c36b904

… value (None)

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

Victor Garcia Reolid added 8 commits

May 18, 2023 08:14


          feat: add data_source property to shared mixin

86257f4

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          fix: potential bug -> multiple data source creation

d8bdd1e

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          fix: issubclass instead of isinstance

650ddd3

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          docs: clarify test_add_reporter docstring

9883b49

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          feat: add function to apply an offset chain

129a549

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          feat: add start_offset and end_offset to add_report

a90105d

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          feat: add start and end offsets

86bb329

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Merge branch 'main' into 642-add-report-cli-command-2

1a73c1a

victorgarcia98 requested a review from Flix6x

May 19, 2023 10:46

Victor Garcia Reolid added 3 commits

May 19, 2023 13:00


          docs: add entry for flexmeasures add report

11778a0

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          feat: create the CLI command flexmeasures show reporters (#686)

c412802

* feat: create show reporter command

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* style: add hint to list reporters

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* style: add module name
style: print empty string when the author is missing

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* style: change column order

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* docs: add entry for PR #686

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

* docs: add command flexmeasures show reporters

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

---------

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Merge remote-tracking branch 'origin/642-add-report-cli-command-2' in…

f2bd1b7

…to 642-add-report-cli-command-2

Flix6x requested changes

View reviewed changes

flexmeasures/cli/data_add.py Show resolved Hide resolved

flexmeasures/cli/utils.py Outdated Show resolved Hide resolved

flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved

flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved

flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved

flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved

flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved

flexmeasures/cli/data_add.py Outdated Show resolved Hide resolved

Victor Garcia Reolid and others added 2 commits

May 19, 2023 17:12


          docs: add entry to changelog

b0813c4

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          Small report cli fixes (#691)

1d5d7be

* style: capitalize CSV

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: reporter is no longer a required parameter

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: required parameters on top

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: fix parameter name in docstring

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: avoid converting dt type

Signed-off-by: F.N. Claessen <felix@seita.nl>

* feature: add test case with timezone

Signed-off-by: F.N. Claessen <felix@seita.nl>

* feature: add test case for Pandas structure

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: type annotations

Signed-off-by: F.N. Claessen <felix@seita.nl>

* fix: use parameter 'end'

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: fix type annotation

Signed-off-by: F.N. Claessen <felix@seita.nl>

* style: justified indentation

Signed-off-by: F.N. Claessen <felix@seita.nl>

---------

Signed-off-by: F.N. Claessen <felix@seita.nl>

Flix6x reviewed

View reviewed changes

documentation/changelog.rst Outdated Show resolved Hide resolved

Victor Garcia Reolid added 6 commits

May 19, 2023 17:19


          docs: update PR number in the changelog

db0a94c

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          fix: changing types

50e2dd3

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          test: move fixture to conftest

880232d

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          fix: handle empty result

954f8f0

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          style: fix inaccurate message to the user

459ad9c

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>


          refactor: remove last-X parameters

6d0f3e0

Signed-off-by: Victor Garcia Reolid <victor@seita.nl>

victorgarcia98 requested a review from Flix6x

May 22, 2023 07:19

Flix6x approved these changes

View reviewed changes

Contributor

Flix6x left a comment

Nicely done!


          Merge branch 'main' into 642-add-report-cli-command-2

36152c4

Contributor Author

victorgarcia98 commented May 29, 2023

Thanks for the review!

I'm trying to merge but it's blocked by the condition to not merge with unresolved comments. Checking all the conversations in Files Changed, there's no unresolved comments so I'm selecting the option Dismiss Review to see if that allows the merge.

Contributor Author

victorgarcia98 commented May 29, 2023

Thanks for the review!

I'm trying to merge but it's blocked by the condition to not merge with unresolved comments. Checking all the conversations in Files Changed, there's no unresolved comments so I'm selecting the option Dismiss Review to see if that allows the merge.

Instead, I think it would better if any of @nhoening or @Flix6x merges this to bypass the blockage 😄

Flix6x merged commit cc40d37 into main

Flix6x deleted the 642-add-report-cli-command-2 branch

May 29, 2023 14:12

Flix6x mentioned this pull request

Reporting backend infrastructure #509

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Reporting Still Needs Changelog Entry