Skip to content

Release v0.16.0

Choose a tag to compare

View all tags
@Edwardvaneechoud Edwardvaneechoud released this 28 Aug 16:30
· 25 commits to main since this release
v0.16.0
551cedf
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Changes since v0.15.4.

Conditional branching in flows, a warm worker process pool, directory-scan mode on the file readers, alerts for scheduled runs, and a plain-Python walkthrough in the browser editor.

Gate — conditional execution

A new Gate node (#675) decides whether an entire branch runs. When the condition doesn't hold the gate itself succeeds but its downstream is deliberately skipped — a distinct state from failure, so runs stay green and post-run callbacks (Kafka offset commits) still fire. Turn on Add an else output for a one-node if/else router. Gates export as real if blocks in all three code exports, so a branching flow still round-trips to Python. See the Gate section of the Combine Nodes reference.

Gate node
The Gate node with an else output, driving an if/else split on the canvas

Warm worker process pool

The worker (#677) keeps spawned children alive between tasks instead of paying interpreter boot + imports on every offload — saves ~0.2s per task on macOS, ~0.4–0.7s on Windows. In practice: on Windows the designer feels instant where clicking around used to have a visible pause, and a 20-node flow can shave several seconds off its run.

On by default on Windows (4 warm children); off by default on macOS and Linux — opt in with FLOWFILE_WORKER_POOL_SIZE or resize live from the new admin-only Performance tab on the Compute page (the old Kernel Manager is now the Kernels tab).

Worker pool dashboard
The Performance tab on the Compute page — warm/busy members, tasks completed, live resize

Directory scan mode

read_csv, read_parquet, read_ipc (and their scan_* aliases) can point at a folder and read every matching file as one table (#674). New arguments scan_mode and include_file_paths on the Read node and the Python API. See the directory-scan section of the Reading Data reference.

Alerts on scheduled runs

Scheduled flows can tell you when something goes wrong. Get a message in Slack, Discord, Microsoft Teams, or any HTTPS webhook when a flow run fails, recovers, succeeds, or gets closed as orphaned (#684). Managed from a new Alerts tab in the Catalog — a channel is the webhook destination, an alert rule picks which outcomes go to which channel (per schedule, per flow, or account-wide). Requires the desktop or server build. Full setup in the new Alerts & Notifications guide.

Alerts tab
The Alerts tab with a channel, an account-wide rule, and recent deliveries

Plain-Python walkthrough (Flowfile Lite)

The browser editor's Code panel (#672, #678, #680) grows a step-by-step walkthrough of your own flow in two flavours: Plain Python (every table a list[dict], every node a for loop — no dataframe library) and Polars (same steps as one-liner calls). Numbered chips step through the pipeline; the Data tab shows the actual rows in/out at each step. Every node's settings panel also gains a "How would I write this myself?" section for its current settings.

Other

  • Context menu refactor (#673) — now works on edges, empty pane, and multi-node selections.
  • Flow API documentation (#682) — new Serve Flows as APIs guide.
  • README rewrite (#679, #683) — Superstore hero GIF, node count updated to 46.

Database

Migration 031_notifications — three new tables for the alerts feature. Reversible; existing rows unaffected.

Dev & infra

~10,200 new test lines across 33 files. Windows CI sharded into core-a, core-b, and worker (was one 37-min job). Linux/macOS CI runs with FLOWFILE_WORKER_POOL_SIZE=2 so the pooled path is what tests exercise. Version bumped to 0.16.0.

What's Changed

Full Changelog: v0.15.4...v0.16.0

Contributors

claude and Edwardvaneechoud
Assets 12
Loading