Skip to content

Release v0.15.4

Latest

Choose a tag to compare

@Edwardvaneechoud Edwardvaneechoud released this 13 Aug 12:00
· 3 commits to main since this release

Sampling options on data nodes, Excel write modes with concurrent-safe multi-sheet updates, smarter dashboard/visualization cache invalidation, and a batch of kernel lifecycle and flow-editor fixes.

Features

  • Sample node: random sampling (#660) — three methods: first (top-N, the default), random (uniform N rows), and random_fraction (uniform X%). Optional seed for reproducibility. Also available as FlowFrame.sample(), in code export, and in the WASM editor.
  • Excel write modes: overwrite, create, and update (#667) — output nodes targeting .xlsx now support three modes. overwrite is the existing behavior. create fails if the file already exists. update opens an existing workbook and replaces only the target sheet, so multiple output nodes can fill different tabs of one file. Concurrent writers to the same workbook are serialized with a per-workbook advisory file lock; writes land atomically via temp + os.replace. Legacy flows with the old new file mode auto-clamp to overwrite. Code export honors create's existence guard; update is runtime-only (openpyxl round-trip, not expressible in pure Polars).
  • Dashboard and visualization cache invalidation (#669) — catalog-reader nodes now probe live Delta versions at the start of every flow run and invalidate their cached result when the source changed, including out-of-band writes the catalog row never saw. Visualization session keys are version-addressed against the actual Delta log, so dashboard tiles and SQL-query charts refresh automatically when their underlying tables are updated. A new useDashboardRefresh composable gives dashboards a manual refresh button that clears frontend caches, force-refetches datasources, and bumps all tile nonces.
  • Targeted interrupts on shared kernels (#659) — cancelling a flow that shares a kernel with another running flow no longer interrupts the other flow's cell. Each execution carries a server-minted token; an interrupt only fires if that token owns the running cell.
  • Shipped kernel dependency manifest (#661) — kernel package inventories are now generated into a manifest that ships with the app (make kernel_manifest), instead of being read from repo sources absent from the wheel/Docker/sidecar. Fixes packaged installs falsely reporting "all packages present". Kernels whose contents can't be enumerated now report unknown instead of full.

Fixes

  • Excel workbook lock contention (#670) — the advisory lock now handles contention errors from concurrent writers more gracefully, preventing spurious failures when multiple output nodes race on the same workbook.
  • False "Cycle detected" on partially-connected flows (#664) — orphaned downstream nodes after disconnecting an edge no longer raise a cycle error. Unreachable nodes are skipped with a warning; the run-progress denominator no longer counts them.
  • Canvas copy/paste leaking into code editors (#663) — pasting text into a Formula/Polars/Python editor no longer silently duplicates a previously copied node on the canvas. Node copies use a clipboard sentinel; keyboard paste lands under the live pointer.
  • Kernel callbacks split across processes (#662) — API and log callback URLs were built independently, one with a hardcoded port. Desktop app kernels could route half their traffic to the wrong process. Single core_base_url() function now.
  • Startup kernel GC destroying live containers (#665) — a swallowed DB error could empty the registry and make GC delete every container and image. GC is now fail-closed, containers carry ownership labels, and FLOWFILE_KERNEL_GC=0 disables it.
  • Kernel shutdown duty transfer (#668) — a hard-killed core (crash, SIGKILL, OS restart) orphaned its kernel containers; the next core adopted them but refused to stop them at exit. Shutdown duty now transfers when the owning pid is no longer alive.
  • Catalog forgets last page (#666) — the sidebar Catalog icon now resumes the last sub-page visited in the session.
  • Dark-mode chip colors (#659) — catalog table editor badges no longer render white-on-white in dark mode.

Maintenance

  • Version bumped to 0.15.4.
  • New Make targets: kernel_manifest, check_kernel_manifest, check_kernel_data.
  • Kernel image tags moved to kernel/images.py (docker-SDK-free).
  • New env vars: CORE_PORT, FLOWFILE_KERNEL_GC.
  • New shared module: shared/excel_writer.py (cross-platform workbook locking + atomic writes).