Skip to content

Version 0.4

Latest

Choose a tag to compare

@jfathi jfathi released this 25 Apr 06:53
4e25023

Webber v0.4 — IterPromise, Callbacks, Metrics, Topology, Dashboard

This release introduces runtime observability, dynamic fan-out, and DAG persistence — making Webber viable for production workflows that need monitoring, templating, and flexible parallelism.

IterPromise — Dynamic Fan-Out

Spawn N parallel child tasks at runtime from a parent node's list output. The DAG graph is never mutated — fan-out is an executor-internal detail (collapse model):

      What you define                              DAG.execute()
      ───────────────                          ──────────────────────
    ┌─────────────────┐                      ┌───────────────────────┐
    │     source      │                      │        source         │
    │ "abc".split('') │                      │ returns ['a','b','c'] │
    └────────┬────────┘                      └──────────┬────────────┘
             │                                          │
             │  IterPromise(source)                ┌────┼────┐  fan-out
             │                                     │    │    │
        ┌────┴─────┐                            [a]▼ [b]▼ [c]▼
        │  worker  │                             ┌───┐┌───┐┌───┐
        │ str.upper│                             │"A"││"B"││"C"│  (parallel)
        └────┬─────┘                             └─┬─┘└─┬─┘└─┬─┘
             │                                     └────┼────┘
             │  Promise(worker)                         │  aggregate
             ▼                                          ▼
       ┌───────────┐                             ┌─────────────┐
       │ collector │                             │  collector  │
       │   print   │                             │["A","B","C"]│
       └───────────┘                             └─────────────┘
      3 nodes in graph                     3 nodes in graph (unchanged)
from webber import IterPromise

source = dag.add_node(lambda: ["a", "b", "c"])
worker = dag.add_node(str.upper, IterPromise(source))
collector = dag.add_node(print, Promise(worker))  # receives ["A", "B", "C"]
  • Multi-source zip: IterPromise(source_a, source_b, zip=True)
  • Resilient mode: IterPromise(source, partial=True) — failed sub-tasks become None
  • Sub-tasks execute in parallel via the existing ThreadPoolExecutor

Progress Callbacks

DAGCallbacks dataclass with 6 hook points for real-time execution observation. Zero overhead when unset.

Execution Metrics

DAGMetrics dataclass with per-task perf_counter timing, completion tracking, and summary statistics via executor.metrics.

Topology Export/Import

Save and restore DAG structure as JSON with callable registry. Supports explicit dict and module-based auto-resolution.

Execution Dashboard

Post-execution Vis.js browser visualization with node status coloring, timing sidebar, and summary stats.

Bug Fixes & Quality

  • Bare except clauses, comparison anti-patterns, dead code cleanup
  • 471 tests, no latency regression
  • 6 new code examples

We're actively seeking support! Reach out to https://github.com/WebberTeam or @jfathi