Skip to content

Support showing instantaneous sensor data with a custom event frequency#542

Merged
Flix6x merged 41 commits intomainfrom
resample-instantaneous-sensor-data
Dec 12, 2022
Merged

Support showing instantaneous sensor data with a custom event frequency#542
Flix6x merged 41 commits intomainfrom
resample-instantaneous-sensor-data

Conversation

@Flix6x
Copy link
Contributor

@Flix6x Flix6x commented Nov 27, 2022 •
edited
Loading

Upgrade timely-beliefs to expose new resampling functionality for instantaneous sensor data:

  • flexmeasures show beliefs can now handle showing (and saving) instantaneous sensor data and non-instaneous sensor data together.
  • /sensors/data now allows fetching instantaneous sensor data in a custom frequency, by using the "resolution" field. I've chosen to not (yet) expose a "frequency" field to the user API, but to keep only the "resolution" field. The resolution of the fetched data is now explicitly part of the return message, while the frequency is still implicit (i.e. the length of the data divided by the duration).

@Flix6x
Upgrade timely-beliefs
1d803c1 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x added this to the 0.12.0 milestone Nov 27, 2022
@Flix6x Flix6x self-assigned this Nov 27, 2022
@coveralls
Copy link
Collaborator

coveralls commented Nov 27, 2022 •
edited
Loading

Pull Request Test Coverage Report for Build 3674188427

  • 6 of 9 (66.67%) changed or added relevant lines in 1 file are covered.
  • 2 unchanged lines in 1 file lost coverage.
  • Overall coverage decreased (-0.02%) to 64.929%
Changes Missing Coverage Covered Lines Changed/Added Lines %
flexmeasures/api/common/schemas/sensor_data.py 6 9 66.67%
Files with Coverage Reduction New Missed Lines %
flexmeasures/api/common/schemas/sensor_data.py 2 78.64%
Totals Coverage Status
Change from base Build 3664805850: -0.02%
Covered Lines: 6644
Relevant Lines: 9596
💛 - Coveralls

@Flix6x Flix6x changed the title Upgrade timely-beliefs Support showing instantaneous sensor data with a custom event frequency Nov 28, 2022
Flix6x added 23 commits November 28, 2022 10:17
@Flix6x
Use requested event frequency instead of df.event_resolution, and com…
c1fe8b8 
…e up with default for instantaneous sensors

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Extend docstring for TimedBelief.search
d4f5ac3 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Return event resolution explicitly, because it may differ from the ev…
1f633da 
…ent frequency

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Make sure reindex is pass a (non-zero) timedelta instead of a possibl…
aa80336 
…e None value

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Remove old documentation notes concerning the use of the fm0 schema f…
60fc05e 
…or UDI events

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Clarify resolution check for posting sensor data
21d3157 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Update notation in API documentation
b267694 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Update example endpoint
713dd78 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Add more clarity about how the resolution field is interpreted for in…
c852a9e 
…stantaneous sensors

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Make SourceIdField importable and usable for API validation
a39c87a 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Allow filtering sensor data by source in API
a6c84db 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Make SourceIdField importable and usable for API validation
83dfe54 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Allow filtering sensor data by source in API
ca7d9a9 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Show source id in chart tooltip so users have somewhere to find them
d00be86 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Add test for /sensors/data [GET]
f4987db 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Test source field, and refactor test setup
764b7cf 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Test resolution field
d389c4a 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Test averaging event values when using resolution field
aa2eb22 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Add type annotations, flake8
d89c22c 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Remove double line breaks
62488c3 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
API changelog entry for introduction of 'source' field
eb8a951 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Missing punctuation
12537d3 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
API changelog entry for return message fix
5cd066c 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Add missing API documentation for optional fields
30f5dbb 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Update API documentation section on sources
d02dc94 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Move test to module using fresh db for each test to avoid session flush
eeb2e85 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Faster conversion of NaN values (we were just missing the dtype conve…
a5a150a 
…rsion)

Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Update inline comment
a050342 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Merge branch '398-support-optional-source-in-sensorsdata-get' into re…
442ea6e 
…sample-instantaneous-sensor-data

# Conflicts:
#	flexmeasures/api/common/schemas/sensor_data.py
@Flix6x
Update documentation refs
51139b5 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x changed the base branch from main to 398-support-optional-source-in-sensorsdata-get November 29, 2022 12:11
@Flix6x
Add test for fetching instantaneous sensor data
3224d53 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Sensor ownership is determined via asset ownership
a5a81b5 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Remove redundant flush
0f6e4d6 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x marked this pull request as ready for review November 29, 2022 13:04
@Flix6x
black
03470ac 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x requested a review from nhoening November 29, 2022 13:21
Base automatically changed from 398-support-optional-source-in-sensorsdata-get to main December 2, 2022 10:32
Flix6x added 2 commits December 2, 2022 11:42
@Flix6x
Merge branch 'main' into resample-instantaneous-sensor-data
4042d2a 
Signed-off-by: F.N. Claessen <felix@seita.nl>

# Conflicts:
#	flexmeasures/api/common/schemas/sensor_data.py
#	flexmeasures/api/v3_0/sensors.py
#	flexmeasures/api/v3_0/tests/conftest.py
#	flexmeasures/api/v3_0/tests/test_sensor_data_fresh_db.py
@Flix6x
Set up initial measurements together with the sensors
2a9e3f5 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x Flix6x mentioned this pull request Dec 2, 2022
Merged
Flix6x added 2 commits December 2, 2022 12:03
@Flix6x
Changelog entry
61866f1 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Typo
16e056e 
Signed-off-by: F.N. Claessen <felix@seita.nl>
nhoening
nhoening approved these changes Dec 9, 2022
View reviewed changes
@Flix6x
Correct sentence
9c5c6f1 
Signed-off-by: F.N. Claessen <felix@seita.nl>
@Flix6x
Merge branch 'main' into resample-instantaneous-sensor-data
3307ee0 
# Conflicts:
#	documentation/changelog.rst
#	requirements/app.in
#	requirements/app.txt
@Flix6x Flix6x merged commit 46213bd into main Dec 12, 2022
@Flix6x Flix6x deleted the resample-instantaneous-sensor-data branch December 12, 2022 10:38
@Flix6x Flix6x mentioned this pull request Dec 23, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@nhoening nhoening nhoening approved these changes

Assignees

@Flix6x Flix6x

Labels

API CLI Data

Projects

None yet

Milestone

0.12.0

Development

Successfully merging this pull request may close these issues.

3 participants

@Flix6x @coveralls @nhoening

Comments