Skip to content

differential-dataflow-v0.25.0

Choose a tag to compare

@github-actions github-actions released this 15 Jul 01:36
· 9 commits to master since this release

Added

  • operators::common: shared building blocks for differential operators — TimeHistory, bilinear_wave, tile_descriptions, KeyView, and discover_times. (#794)
  • TraceReader::batches_through, exposing the batches up to a given frontier for the tactic-driven operators. (#773)
  • formal/: machine-checked Lean models of differential's coverage and compaction arguments, plus a model-derived reference reduce tactic used as a differential oracle in tests. (#776, #780)

The following surface is new and experimental: it is public to enable experimentation, but the API is unstable and expected to change.

  • Chunk trait abstracting the container that backs collections, merge batching, and batches, with the columnar module retargeted onto it. (#744, #769, #778)
  • Public JoinTactic and ReduceTactic traits, letting callers supply per-batch join and reduce strategies; documented with asserted contracts. (#789, #790, #791)
  • Integer-proxy chunks: a backend-agnostic join and reduce over (key_hash, value_id) pairs. (#781)

Changed

  • Update to timely 0.31. (f7c799c)
  • Move layout onto the Cursor. Cursors now carry the key, value, time, and diff types and their containers as associated types; BatchReader and TraceReader retain only Time. Navigation becomes an optional cursor capability rather than a universal one. (#771, #772)
  • Generalize join and reduce into navigation-free drivers over pluggable tactics. join_core, reduce_abelian, and reduce_core gain type parameters for the tactic and output container. (#773, #790)
  • Rework MergeBatcher merge effort: weigh the geometric ladder by update counts, split Merger::account, and add Merger::len. (#767)
  • Name the shared key container in join and reduce to fix HRTB normalization for non-reference key containers. (#797, #798)
  • Reorganize the experimental columnar module: the 0.24 spill, batcher, builder, exchange, and arrangement submodules give way to layout, updates, trie_merger, collection, and trace. Many 0.24 public types (ValBuilder, ValPact, ValChunker, RecordedUpdates, Coltainer, the Spill / Fetch / SpillPolicy traits) are removed or relocated. (#769, #781)

Removed

  • The LayoutExt trait, and its supertrait bound on BatchReader, TraceReader, and Cursor. (#772)
  • TraceReader::Cursor, TraceReader::Storage, and BatchReader::Cursor associated types, along with BatchReader::cursor; CursorList<Batch::Cursor> is hard-wired instead. (#771)

This is a heavily breaking release.
Layout information moves off the trace and batch types onto the Cursor, which now carries the key, value, time, and diff types and their containers as associated types; BatchReader and TraceReader retain only Time.
Navigation becomes an optional cursor capability, and join and reduce are rebuilt as navigation-free drivers parameterized by the public JoinTactic / ReduceTactic traits, so join_core, reduce_abelian, and reduce_core gain tactic and container type parameters.
Downstream cursor, trace, and batch implementations must migrate to the associated-type form and drop the removed LayoutExt supertrait.
The experimental columnar module is reorganized and most of its 0.24 public surface relocated.
The Chunk trait, the JoinTactic / ReduceTactic traits, and the integer-proxy chunks are new and explicitly experimental: they are public to enable experimentation, but their APIs are unstable and expected to change.
Dependency tracking follows timely 0.31.