v0.25.0

Release 0.25.0

This span widens where Flowthru can read, write, and compute. Google Sheets and AWS S3 join the catalog, and Python steps can now train across multiple processes. There are also large improvements to the way steps like Python fold into design-time feedback: missing Python packages and how far pre-flight reaches are now things you declare and Flowthru checks for you, instead of surprising you mid-run.

What's New

Google Sheets

• Google Sheets Catalog Extension: You can now bind a Catalog Item to a native Sheets table with ItemFactory.Enumerable.GoogleSheets<TRow>(...), addressing it by a stable (spreadsheetId, table name) rather than a fragile cell range. Read coerces each cell to your schema property's declared type; write creates the table from the schema on first use and atomically replaces it each run, scoped to its own tab so sibling formula tabs are never clobbered. The extension owns no credentials — you build a SheetsService however you like and hand it to AddGoogleSheets, so auth lives in DI and the catalog never sees a secret. See Flowthru.Extensions.Google.Sheets.
  • GoogleSheets Starter: A complete sales-summary pipeline that reads a raw table, totals it by day, and writes the result back to a second table — running end-to-end with no Google account and no network by swapping in a shipped file-backed gateway. The same flow points at a real spreadsheet by changing only the gateway, not the catalog or steps. See GoogleSheets.

AWS S3

• AWS S3 Storage Medium: You can now read and write Catalog Items against s3:// paths. This registers a storage medium for the s3:// scheme, so any format you already use (Csv, Parquet, Json, Xml) works over S3 with no change — a Flow targets S3 just by writing an s3://bucket/key path on an Item. Bare and file:// paths still resolve locally; only s3:// routes to S3. Credentials resolve through the standard AWS chain, so Flowthru never loads or stores a secret. A write-probe at pre-flight PUTs and deletes a sentinel beside the target key, so a missing bucket or denied permission fails before any step runs instead of two hours in. UseLocalS3 swaps in a shipped file-backed stub for fully offline development against the same Flow. See Flowthru.Extensions.AWS.S3.

Python

• Distributed Python Training: You can now swap the process that launches your Python worker by registering an IPythonLauncher in DI before UsePython(). The default DirectPythonLauncher preserves the existing single-process behavior; two distributed launchers ship out of the box. TorchrunLauncher { NProcPerNode = N } spawns workers via torchrun for PyTorch DDP training across multiple processes on one host, and AccelerateLauncher dispatches through HuggingFace Accelerate to whichever backend your config selects. Your step function runs identically on every rank, and only rank 0 returns results to the C# catalog. A single distributed worker pool now serves every distributed step in a flow, so you no longer wire one executor per step.
  • Declared Python Package Requirements: You can now attach [PythonPackageRequirement(...)] to a launcher or capability class to declare the Python packages it depends on, and Flowthru enforces them at two phases. At design-time, an analyzer reads your uv.lock and flags a missing or wrong-version package as FTPY1501 / FTPY1502 right in your IDE — your uv.lock is wired automatically, with no .csproj setup. At pre-flight, the same checks run against the configured venv (FTPY3011 / FTPY3012). A missing Python dependency is now a red squiggle or a startup error instead of a crash deep in a run.
  • MnistDistributed Example: A two-step pipeline that trains a small CNN via TorchrunLauncher and then verifies it through the same worker pool, demonstrating multi-process training and pool reuse end-to-end. See MnistDistributed.

QOL

• Configurable Pre-Flight Depth: You can now choose how much pre-flight validation runs on each invocation with --validation-depth <none|hermetic|shallow|deep> (shallow stays the default). The new hermetic level performs the maximum validation that touches zero I/O — it confirms the flow is structurally sound and fully wired without opening a socket, file, or database. Pair --dry-run --validation-depth hermetic for an offline smoke test: validate everything knowable without external resources, then run nothing.

Bug Fixes

• Execution Defaults Configuration: Setting host-level execution defaults such as Parallelism had stopped working. You can again configure them imperatively with flowthru.ConfigureExecution(o => o.Parallelism = 4) or declaratively by binding the Flowthru:Execution section in appsettings.json, and a bad value (parallelism below 1) is caught at pre-flight rather than silently clamped.
• Multi-Worker Python Logging: When several Python workers ran at once, their output could interleave on the shared protocol stream and corrupt it. Each worker's output is now isolated, so distributed runs stay readable and the protocol stream stays clean.

❤️ Thank You

• Spencer Elkington