New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue 259 synchronize how we collect data from data models #260
Issue 259 synchronize how we collect data from data models #260
Conversation
…dow into event_starts_after and event_ends_before
…me_window into beliefs_after and beliefs_before
…rizon_window into horizons_at_least and horizons_at_most
…to building up filter criteria and then applying those criteria at once
Still requires a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good overall, but I have a problem to understand a central piece of this. We can also do a call if that would be quicker to explain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So ... I might have missed it, but why do we need Sensor.search
, when Sensor.collect
gets the same parameters and is more powerful:
- can query multiple sensors if needed, and sum, if needed
- can resample
@@ -225,6 +225,8 @@ def test_query_beliefs(setup_beliefs): | |||
source = DataSource.query.filter_by(name="Seita").one_or_none() | |||
bdfs = [ | |||
TimedBelief.search(sensor, source=source), | |||
TimedBelief.search(sensor.id, source=source), | |||
TimedBelief.collect(sensor.name, source=source), | |||
sensor.search_beliefs(source=source), | |||
tb.BeliefsDataFrame(sensor.beliefs), # doesn't allow filtering |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So are all beliefs in setup_beliefs
from one source anyways? Then we are not really tresting the filtering.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True. We're using the filter only to test whether a FlexMeasures DataSource
, which subclasses the timely-beliefs BeliefSource
, results in a query that the database can still understand.
Neither exists. Are you instead suggesting to expand the capabilities of |
Sorry, I meant To me, your suggestion sounds like it leads to less confused questions like mine. If you also like that design and it's not too much work, go ahead. |
Merging the new TimedBelief.collect method with the existing TimedBelief.search method. This also entails renaming all occurences of TimedValue.collect to become TimedValue.search, in order to safeguard the goal of PR #260. * Rename old collect method to search * Merge new collect method into TimedBelief.search * Deprecate the sensors argument in favor of sensor * Small annotation revert in scope of project 9 * Fix deprecation of required argument
This PR synchronizes the function signatures of our old and new data collection methods, thereby supporting us in moving over data to our new data model. It introduces a new
TimedBelief.collect
method whose function signature is compatible with calls toTimedValue.collect
. To ensure equivalency, some query filter utils have been refactored to become filter criteria (i.e. the SQLAlchemy term for that which you pass to a query filter), and timely-beliefs 1.8.0 is on its way to support passing additional custom criteria for its search query filters.closes #259