Skip to content
petehotteletPublic

About

An open-source framework for task-based agent swarms with dynamic parallelization, routing, and execution topology.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

4 stars

Watchers

0 watching

Forks

Repository files navigation

SMYTHE

An open-source framework for task-based agent swarms with dynamic parallelization, routing, and execution topology.

Rust preview 0.9.0 Rust CI status Rust 1.91 or later License: MIT

Quickstart · How it works · Measured results · Documentation

Smythe is a Rust framework for running task graphs with durable recovery. Define the work and its dependencies, execute ready tasks in parallel, and inspect the journal of outputs and costs. The native engine combines bounded concurrency, exact cost reservations, and recovery after a process interruption.

The Rust 0.9.0 preview runs local workflows with or without API keys. Model-driven planning, paid providers, tools, MCP, supervision and Jobs are roadmapped. Current scope.

Animated Noumenon rain showing the 192 authored Smythe glyphs

Noumenon is an example demonstrating parallel-processing: each glyph compiles as an independent task, and the Rust pipeline validates and exports the catalog to produce the screensaver.

192-glyph sheet · 256-glyph sheet · Individual SVGs · Noumenon screensaver · All materials

Quickstart

Build from this checkout with Rust 1.91 or later and a C build toolchain for bundled SQLite. The Rust packages are unpublished; the supported path is a source build. Run these commands from the repository root:

cargo fetch --locked
cargo build --release --locked --offline -p smythe-cli

Once dependencies are fetched, the examples run offline.

Run, inspect, and resume a graph

The bundled example graph contains two dependent tasks: research followed by summarize. The local OfflineProvider returns each task's goal, input, and references to dependency outputs, so you can inspect execution and recovery without a model call.

cargo run --release --locked --offline --bin smythe -- run --graph examples/native_graph.toml --id example --store smythe_artifacts/native-runs
cargo run --release --locked --offline --bin smythe -- inspect example --store smythe_artifacts/native-runs --events
cargo run --release --locked --offline --bin smythe -- output example research --store smythe_artifacts/native-runs
cargo run --release --locked --offline --bin smythe -- run --graph examples/native_graph.toml --id example --store smythe_artifacts/native-runs --resume

The first command executes both nodes and saves their results. inspect returns the saved state and events as JSON. Resuming the completed run returns the same output references with reused_nodes: 2, provider_calls: 0, and cost_nanousd: "0". Use output to load one verified value, or run --outputs inline to load all. Use a new ID for another fresh run, or --resume to reopen the existing one.

Graphs can be TOML or JSON. Set --workers to bound parallel work and --budget-nanousd to set the cost ceiling; their defaults are four workers and zero on a fresh run. Resume uses the saved ceiling; change it explicitly with control <id> budget <nanousd>. The bundled provider charges nothing.

Benchmark: Generate the Noumenon catalog

cargo run --release --locked --offline --bin smythe -- noumenon --glyphs 256 --workers 8 --out smythe_artifacts/native-glyphs
cargo run --release --locked --offline --bin smythe -- noumenon --glyphs 256 --workers 8 --out smythe_artifacts/native-glyphs --resume

A fresh run requires a new output directory. Full mode compiles the glyphs, renders at 16, 32, 64, and 128 pixels, checks topology and pairwise distinctness, and writes SVGs, catalog.json, validation.json, atlas.png, and report.json. Use --mode compile for compilation only. A completed resume validates the saved exports and makes no new provider calls.

Native commands and recovery · Examples.

How it works

The graph defines the work. Each node names a task and its dependencies. Smythe validates the graph before execution and dispatches a ready child as soon as its dependencies are accepted. Independent tasks share a bounded worker pool; an unrelated slow sibling does not hold up a ready branch. The Rust alpha executes graphs you supply. Generating a task-specific topology from a goal remains on the Rust roadmap.

The execution envelope governs the run. A single coordinator owns admissions, cost reservations, and state transitions. It records dispatch intent, stores outputs as immutable hashed blobs, and journals receipts and settlement in SQLite. Recovery can reuse recorded results and complete pending settlement; interrupted local work without a saved result may run again.

You need to… The Rust alpha provides
Define the workflow Validated TOML or JSON graphs with explicit dependencies
Bound parallel work Readiness-driven execution through a bounded Rayon worker pool
Account for cost Checked integer nanoUSD, reservations before dispatch, and idempotent settlement; current providers are local and zero-cost
Recover interrupted work SQLite journals, immutable output blobs, and integrity checks before reuse
Inspect or control a run Typed JSON errors with bounded sanitized diagnostics, explicit cursor paging, owner health, and persisted pause, continue, stop, retry and budget commands
Produce the glyph catalog Authored SVG compilation, raster validation, distinctness checks, and resumable exports
Embed the engine The smythe crate's Swarm, local Provider contract, store, and tracing types

Pause and stop prevent new admissions while active local calls drain. Recovery guarantees cover local process crashes. Remote effects, distributed workers, and power-loss guarantees are outside this alpha's scope.

The current native journal uses amended schema 3 and reads earlier schema-3 journals; schemas 1 and 2 require a new directory. Failed cost quotes are resumable, and invalidation gives affected nodes fresh retry allowance while attempt numbers keep increasing. Already-terminal quote/attempt failures retain their outcome. Resume checks semantic compatibility and saved outputs, so unrelated source edits do not invalidate a run. See the recovery contract.

Scheduling and accounting · Journal and run controls · Glyph acceptance.

Measured results

The journaled Rust alpha has no claimable performance headline yet. The retained Rust/Python comparison measures the earlier native preview. Its compilation clock excludes durability work now performed by the shared engine, so those timings do not describe this implementation.

Evidence What it establishes Scope
Original Rust preview comparison Matching SVG bytes across Rust and both Python backends; independent validation of native catalogs Historical diagnostic preview; different full-workflow clocks do not establish an end-to-end speedup
Native measurement guide How to record fresh CLI runs with source and binary provenance Diagnostic runner for the journaled engine; CPU and peak-memory measurements are still pending
Python benchmark collection Recorded framework, topology, artifact, and recovery results Evidence for the Python implementation, within each report's stated limits

The Python SVG workflow, Noumenon fan-out, interruption study, and Astra/Sol findings remain available with their protocols and records. They are not Rust performance claims.

Historical Python benchmark results

The following measurements and charts describe the Python implementation. They remain scoped to their linked protocols and are not Rust performance claims.

Framework efficiency

On a fixed three-stage pipeline, Smythe used 77% fewer tokens and 28% less wall time than CrewAI across five tasks and three repetitions per framework, with the same executor model and blind cross-vendor judging. It also recorded 6% less mean wall time than LangGraph, and the highest observed quality score: 9.73/10, versus 9.53 for both. Token counts describe model usage, not invoice savings.

Smythe uses 77 percent fewer mean tokens and 28 percent less mean wall time than CrewAI on the matched fixed-pipeline suite

Smythe, LangGraph, and CrewAI: observed blind quality, mean token counts, and mean wall time across 15 runs per framework

Recovery after interruption

After a hard kill, Smythe repeated 8 calls versus LangGraph's 32, a 75% reduction, in each of three matched repetitions. Both finished all 64 operations. Recovery protocol.

Three matched interruption tests: Smythe repeated 8 dispatches and LangGraph repeated 32 in every repetition; both finished all 64 operations

Generated execution topology

Across five task shapes, generated plans took 14% less wall time than a fixed pipeline, planning included. They used one node for a simple transformation and 5.3 on average for parallel research. Observed quality averaged 9.47/10 versus 9.33/10, within measured judge variation. Task-shape protocol and records.

Generated plans adapt node count to the task and reduce mean wall time by 14 percent, including planning, across the task-shape suite

The 200-workflow Astra/Sol study also reports the limits of generated plans: they increased mean time in both models on its ten synthetic tasks. The frozen rule accepted 191/200 workflows; human review accepted all eight disputed available answers. One missing usage receipt limits affected exact cost comparisons. All outcomes remain published.

Parallel artifact generation

Smythe compiles, validates at four sizes, compares every pair, and exports 256 SVG glyphs in 8.06 seconds median, 2.20× faster than its concurrency-one baseline with eight process workers. All 36 workflows pass with identical SVG and pixel hashes across repetitions, with zero API calls. Results and scope.

All 36 complete v2 workflows at 192 and 256 glyphs, with medians, ranges and one actual run's stages

Process workers trade more memory for shorter completion time:

Measured parent-plus-worker memory at each catalog size and concurrency

At concurrency 64, generating 192 glyphs took 21.7 seconds versus 1,133.3 seconds serially, 52.3× faster, and all 192 validated as unique. This is a controlled offline measurement with 5.8 seconds of simulated provider latency per call; every 64-, 128-, 192- and 256-node run produced complete sets of valid, unique tiles. Noumenon protocol and records.

Controlled offline Noumenon glyph generation at four graph widths, with all tiles valid and unique at every measured concurrency

Live GPT Image lanes generated the same 192 glyphs as transparent PNGs, and traced them into SVGs, three calls at a time within the account's image rate limit. Live lanes.

Jobs at 5,000 operations

A durable Jobs campaign was killed mid-run and recovered to 5,000 accepted artifacts. Safe resume preserved the 2,492 outputs already accepted and reissued none of them; eight interrupted operations needed explicit rerolls, and resuming the completed job made zero new calls. One offline campaign with identical fixtures, so this shows recovery correctness, not speed. Results and independent reconciliation.

One offline Jobs campaign: 2,492 accepted after the kill, 4,992 after safe resume, and 5,000 after eight explicit rerolls

All benchmarks, charts, and evidence status.

Project status

Rust workspace: 0.9.0, source preview. APIs and journal formats are experimental. The workspace contains smythe-core for graph and accounting contracts, smythe for the shared runtime, smythe-cli for the executable, smythe-noumenon for the glyph pipeline, and xtask for native measurements. Rust CI covers Linux and Windows.

The local engine is still being qualified. Quote/revision recovery, expanded simulation, bounded diagnostics, paging validation and report source coverage are implemented. Near-term work completes native checks, raster and measurement acceptance. See known preview limitations and the roadmap. Remote accounting and generated planning follow, then tools, supervision and Jobs. Learning remains after 1.0; optional telemetry belongs to the planned 1.0 interfaces.

The Python implementation, tests, packaging and comparison harnesses are retained in Git history at 12f1470. Historical benchmark records and the Noumenon artwork remain in this checkout. The eventual stable Rust release will be 1.0.0; this is the 0.9.0 preview.

The Noumenon screensaver is a separate project that renders the glyph catalog. Smythe's native command produces the catalog; it does not build or run the screensaver.

Documentation · Source builds · Native development checks · Contributing · Changelog · Security · MIT license.

About

An open-source framework for task-based agent swarms with dynamic parallelization, routing, and execution topology.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages