v0.22.0 Edit your flow as a notebook, on the canvas, or both 🎉

[Edwardvaneechoud]

0.22.0:

Changes since v0.21.1.

Flowfile now lets you change a flow visually and in a notebook at the same time, on the same screen. A while back I implemented typing your graph with the flowfile package: write ff.read_csv(...), chain a few steps, and open the result on the canvas. Once that worked, a truly bidirectional experience became the goal. This release delivers it: build in Python or on the canvas, whichever fits the moment, and the other side keeps up.

Every statement in the notebook is a node the canvas already understands, which is what makes an edit on either side an edit to the same graph.

The flow is a notebook now (#776, #784)

Open a flow, press Ctrl/Cmd+G and pick Notebook. The flow is shown as Python cells, import flowfile as ff, one cell per statement. Type a filter in a cell and push it: the node appears on the canvas, wired to its input. Add a sort on the canvas: the notebook gets its .sort(...) line. Run a cell and you see that node's rows, gates respected. Every push is one undo step.

Without a kernel, nothing in your cells is executed. Push reads the supported statements and applies them; a print or a loop points at its line and says it needs a kernel. Some nodes still show up as placeholders and are edited on the canvas.

There is a short video of the round trip on the notebook page in the docs.

Or run it on a real kernel (desktop and pip install) (#784, #787)

Pick a notebook kernel from the toolbar (a kernel with flowfile installed; the picker creates one for you) and the cells run as real Python: loops, imports, display(...), flowfile_ctx. Push reruns the cells in a fresh session and puts the flow they build on the canvas. The kernel is kept at arm's length: it holds no copy of the catalog database, never sees a stored secret, and anything that needs one (a database or cloud source, a secret-bearing custom node) is run by the app with the rows handed over. Docker deployments keep the notebook without a kernel.

Also in this release

• The Code panel is a resizable split beside the canvas with FlowFrame, Polars, Project and Notebook. The drawer keeps Settings and Results. (Feature/add notebook on kernel #784)
• Ten new Python API tutorials in the docs, from first steps to branching and flow parameters. (Add notebook to flow on canvas #776)
• Filters with many values use a native membership test, so a long in list is a lot faster. (Enhance filter expressions with native membership support and improve… #780)
• Showing a node's rows from a notebook cell no longer moves a change-tracking cursor or commits Kafka offsets. Only a writer does. (Hardening/remove db access in kernel #787)
• and, or and not on an expression now raise instead of silently dropping a condition. Use &, | and ~. (Fix/expr truthiness #783)
• Security: FastAPI 0.142 with a Starlette floor, the frontend and WASM dependency alerts cleared, and the log routes now require a token. (Add notebook to flow on canvas #776, Fix frontend and WASM Dependabot alerts #781, Fix/vulnerabilities backend #782)
• The in-browser editor ships as flowfile-editor 0.2.0 on npm. (Bump flowfile-editor to 0.2.0 #785, Publish flowfile-editor via npm trusted publishing #786)

Upgrade notes

• Migrations 034 and 035 run at startup. Nothing to do. (Feature/add notebook on kernel #784, Hardening/remove db access in kernel #787)
• Kernel images are 0.6.1. Older kernels must be rebuilt or pulled: they no longer sign their log posts and core rejects them. A notebook kernel must carry the same flowfile version as the app, so recreate it after updating.
• POST /logs/{flow_id} is gone, POST /raw_logs needs X-Internal-Token, and GET /logs/{flow_id} no longer accepts ?access_token=. Nothing in the repo used the old forms.
• In docker and package modes, custom-node save, delete and dry-run, and notebook plan and push, are admin-only.
• Known limit: on Apple Silicon, installing flowfile on the lite kernel fails because polars-grouper publishes no aarch64 Linux wheel.

Full Changelog: v0.21.1...v0.22.0

---

This discussion was created from the release v0.22.0 Edit your flow as a notebook, on the canvas, or both 🎉.