Skip to content

v0.18.2

Choose a tag to compare

@github-actions github-actions released this 18 May 04:09
· 83 commits to main since this release

Release 0.18.2

Python steps can now opt into the same smart-caching plan that C# steps got in 0.18 — mark a @step decorator with cacheable=True and the framework auto-derives a CodeVersion from the .py source, the project's lockfile, and the interpreter version. Cached Python steps short-circuit on warm runs just like cached C# steps do.

What's New

  • Python Step Caching: Adding cacheable=True to a @step(...) decorator opts the step into Flowthru's cache plan. The framework folds the .py file content, the project's resolved lockfile (uv.lockpoetry.lockrequirements.txtPipfile.lockpyproject.toml), and the interpreter version string into a stable CodeVersion. As long as the function is pure with respect to its inputs, a warm run skips the Python step entirely — no subprocess invocation, zero recorded duration, blue cell on the Mermaid diagram. No changes to the C# call site: pipeline.AddPythonStep(...) auto-derives the version when the registry has an entry for that (module, function) pair.
    • Reproducible-Build Identity: The version derivation hashes the .py source content (any logic edit invalidates), the resolved lockfile (any dependency bump invalidates), and the interpreter version string (any Python upgrade invalidates). Hashing the version string rather than the interpreter binary keeps the path fast on large managed installs.
    • FlowthruCoverage Demo: Both Python steps in the FlowthruCoverage example are now marked cacheable=True. A warm run that previously took ~30 seconds (dominated by the 25-second Plotly icicle render in generate_coverage_icicle.py) now finishes in under a second — roughly a 37× speedup. See Reporting/Steps for the opted-in Python functions.
    • KedroSpaceflightsPython Demo: Four Python steps in the KedroSpaceflightsPython example are now cacheable=True — the data preprocessing chain through the train/test split. Three of the four cache between runs; the fourth re-runs because its outputs are in-memory items that don't persist across processes by design.

Bug Fixes

  • Bare-String Decorator Outputs: The Python source generator's regex previously only matched outputs=["X", "Y"] (bracketed lists), silently ignoring the equivalent outputs="X" (bare string) form that the @step decorator has always accepted at runtime. Bare-string decorators are now parsed correctly, so @step(cacheable=True) opts in regardless of which output syntax you use.
  • FT2007 Severity: The diagnostic that fires when a @step decorator references an unknown schema is now Warning rather than Error. Catalog-label outputs (e.g. outputs="CoverageHeatmap" for an IItem<byte[]>) are legitimate when the consumer uses pipeline.AddPythonStep<TIn, TOut>; the named-factory path still surfaces a real compile error if you actually try to call into it with an unresolved type, so misuse isn't silenced — just promoted to the right call site.
  • Per-Package Targets Import: The in-repo build's auto-import of per-package targets files (Flowthru.Extensions.Python.targets, Flowthru.FUnit.targets) ran an item-expansion-in-a-property gate that evaluated to empty across MSBuild passes, so the targets file never loaded and the Python source generator never picked up .py files as AdditionalFiles. Imports are now gated on a plain Exists() check; each per-package targets file's own internal guards prevent unintended side effects on consumers that don't reference the package. Direct-ProjectReference consumers (those that don't use FlowthruInRepoSwap.props) can wire the targets file with a one-line <Import> — see the FlowthruCoverage csproj for the pattern.
  • Schema-Name Drift in Python Examples: Three Python decorators in the KedroSpaceflightsPython example referenced schema names that didn't match the C# [FlowthruSchema] records (CompanyRawSchemaCompanySchema, etc.). The mismatches surfaced only once the source generator started parsing those files; the names are now aligned.

🩹 Fixes

  • added python step cacheability (9de83fed)

❤️ Thank You

  • Spencer Elkington