Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement strata, enabling per-tick non-monotonic aggregations #94

Merged
merged 6 commits into from
Mar 5, 2022

Conversation

MingweiSamuel
Copy link
Member

@MingweiSamuel MingweiSamuel commented Mar 4, 2022

  • Users can now optionally specify a stratum number when adding a compiled subgraph, with the new add_..._stratified() methods. When not specified stratum number is zero.
  • Scheduler will run each stratum to fixed point before continuing to the next strata (loop to beginning after final stratum)
  • Scheduler keeps track of epoch -- number of times we've looped around strata.
  • New Hydroflow.run_stratum() method runs only the current stratum.
  • New Hydroflow.next_stratum() method increments to the next stratum with work available.

@MingweiSamuel MingweiSamuel force-pushed the strata2 branch 3 times, most recently from 03efd52 to 1140c8d Compare March 4, 2022 22:13
@MingweiSamuel MingweiSamuel changed the title Implement strata Implement strata, add per-tick enabling per-tick non-monotonic aggregations Mar 4, 2022
@MingweiSamuel MingweiSamuel marked this pull request as ready for review March 4, 2022 22:17
@MingweiSamuel MingweiSamuel changed the title Implement strata, add per-tick enabling per-tick non-monotonic aggregations Implement strata, enabling per-tick non-monotonic aggregations Mar 4, 2022
hydroflow/src/scheduled/graph.rs Outdated Show resolved Hide resolved
hydroflow/src/scheduled/graph.rs Outdated Show resolved Hide resolved
hydroflow/tests/groupby.rs Show resolved Hide resolved
hydroflow/src/scheduled/graph.rs Show resolved Hide resolved
hydroflow/src/scheduled/graph.rs Show resolved Hide resolved
hydroflow/src/scheduled/graph.rs Outdated Show resolved Hide resolved
@MingweiSamuel MingweiSamuel merged commit d39d44d into hydro-project:main Mar 5, 2022
@MingweiSamuel MingweiSamuel deleted the strata2 branch March 5, 2022 00:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants