v0.18.2
Release 0.18.2
Python steps can now opt into the same smart-caching plan that C# steps got in 0.18 — mark a @step decorator with cacheable=True and the framework auto-derives a CodeVersion from the .py source, the project's lockfile, and the interpreter version. Cached Python steps short-circuit on warm runs just like cached C# steps do.
What's New
- Python Step Caching: Adding
cacheable=Trueto a@step(...)decorator opts the step into Flowthru's cache plan. The framework folds the.pyfile content, the project's resolved lockfile (uv.lock→poetry.lock→requirements.txt→Pipfile.lock→pyproject.toml), and the interpreter version string into a stable CodeVersion. As long as the function is pure with respect to its inputs, a warm run skips the Python step entirely — no subprocess invocation, zero recorded duration, blue cell on the Mermaid diagram. No changes to the C# call site:pipeline.AddPythonStep(...)auto-derives the version when the registry has an entry for that(module, function)pair.- Reproducible-Build Identity: The version derivation hashes the
.pysource content (any logic edit invalidates), the resolved lockfile (any dependency bump invalidates), and the interpreter version string (any Python upgrade invalidates). Hashing the version string rather than the interpreter binary keeps the path fast on large managed installs. - FlowthruCoverage Demo: Both Python steps in the FlowthruCoverage example are now marked
cacheable=True. A warm run that previously took ~30 seconds (dominated by the 25-second Plotly icicle render ingenerate_coverage_icicle.py) now finishes in under a second — roughly a 37× speedup. See Reporting/Steps for the opted-in Python functions. - KedroSpaceflightsPython Demo: Four Python steps in the KedroSpaceflightsPython example are now
cacheable=True— the data preprocessing chain through the train/test split. Three of the four cache between runs; the fourth re-runs because its outputs are in-memory items that don't persist across processes by design.
- Reproducible-Build Identity: The version derivation hashes the
Bug Fixes
- Bare-String Decorator Outputs: The Python source generator's regex previously only matched
outputs=["X", "Y"](bracketed lists), silently ignoring the equivalentoutputs="X"(bare string) form that the@stepdecorator has always accepted at runtime. Bare-string decorators are now parsed correctly, so@step(cacheable=True)opts in regardless of which output syntax you use. FT2007Severity: The diagnostic that fires when a@stepdecorator references an unknown schema is nowWarningrather thanError. Catalog-label outputs (e.g.outputs="CoverageHeatmap"for anIItem<byte[]>) are legitimate when the consumer usespipeline.AddPythonStep<TIn, TOut>; the named-factory path still surfaces a real compile error if you actually try to call into it with an unresolved type, so misuse isn't silenced — just promoted to the right call site.- Per-Package Targets Import: The in-repo build's auto-import of per-package targets files (
Flowthru.Extensions.Python.targets,Flowthru.FUnit.targets) ran an item-expansion-in-a-property gate that evaluated to empty across MSBuild passes, so the targets file never loaded and the Python source generator never picked up.pyfiles asAdditionalFiles. Imports are now gated on a plainExists()check; each per-package targets file's own internal guards prevent unintended side effects on consumers that don't reference the package. Direct-ProjectReference consumers (those that don't useFlowthruInRepoSwap.props) can wire the targets file with a one-line<Import>— see the FlowthruCoverage csproj for the pattern. - Schema-Name Drift in Python Examples: Three Python decorators in the KedroSpaceflightsPython example referenced schema names that didn't match the C#
[FlowthruSchema]records (CompanyRawSchema→CompanySchema, etc.). The mismatches surfaced only once the source generator started parsing those files; the names are now aligned.
🩹 Fixes
- added python step cacheability (9de83fed)
❤️ Thank You
- Spencer Elkington