v0.8.0

Overview

• Capture operator no longer takes arguments. Items that flow through
  those points in the dataflow graph will be processed by the output
  handlers setup by each execution entry point. Every dataflow
  requires at least one capture.

• Executor.build_and_run() is replaced with four entry points for
  specific use cases:

  • run() for exeuction in the current process. It returns all
    captured items to the calling process for you. Use this for
    prototyping in notebooks and basic tests.

  • run_cluster() for execution on a temporary machine-local cluster
    that Bytewax coordinates for you. It returns all captured items to
    the calling process for you. Use this for notebook analysis where
    you need parallelism.

  • spawn_cluster() for starting a machine-local cluster with more
    control over input and output. Use this for standalone scripts
    where you might need partitioned input and output.

  • cluster_main() for starting a process that will participate in a
    cluster you are coordinating manually. Use this when starting a
    Kubernetes cluster.

• Adds bytewax.parse module to help with reading command line
  arguments and environment variables for the above entrypoints.

• Renames bytewax.inp to bytewax.inputs.

What's Changed

• New execution interface by @davidselassie in #19
• Container and Kubernetes related improvements by @miccioest in #24
• Restructure Python imports by @whoahbot in #22
• Restructure rust by @whoahbot in #25
• Renames test_run to test_execution by @davidselassie in #27
• Adds an "order" input helper and allows tumbling "event time" by @davidselassie in #26
• Maturin develop before running tests. by @whoahbot in #28
• Tests ability to interrupt execution by @davidselassie in #23
• Jupyter Notebook Anomaly Detection Example by @awmatheson in #30
• Debug operators by @blakestier in #29
• Running tests using whl file already build by @miccioest in #32
• More comprehensive readme by @awmatheson in #33
• modify inputs by @awmatheson in #35
• Remove Criterion benchmark by @whoahbot in #37
• Add a key param to the stateful_map operator. by @whoahbot in #36
• Runs doctests in CI by @davidselassie in #38
• API Docs by @davidselassie in #39
• Fixes sorted_window() to support items with identical times by @davidselassie in #41
• Apidocs templates by @konradsienkowski in #40
• Testable examples by @davidselassie in #34
• Examples metadata by @konradsienkowski in #43
• Update to 0.8.0 by @davidselassie in #45
• Runs CI on release publish by @davidselassie in #46

New Contributors

• @blakestier made their first contribution in #29
• @konradsienkowski made their first contribution in #40

Full Changelog: v0.7.1...v0.8.0