Duckle v0.5.7

Duckle v0.5.7 adds a pip-installable Python API with MCP onboarding, a Salesforce Bulk query source, faster streaming writes to MongoDB, and a batch of correctness fixes across sinks, derived columns, and concurrent runs.

New features

• pip install duckle (Python API + MCP) - a fluent Python builder that compiles to the same engine and pipeline format as the canvas. Write a pipeline in code, and the Python expressions compile to DuckDB SQL at plan time. The wheel bundles the headless runner and the MCP server, so uvx duckle quickstart scaffolds and runs a first pipeline, and a coding agent can build and run pipelines for you to verify on the same canvas.
• Salesforce Bulk query source src.salesforce.bulk - read migration-scale result sets over Bulk API 2.0, with the result-walk guarded and no-schema reads preserving text, completing the Bulk pair alongside the v0.5.6 sink. (#164, #196, #197)
• Faster MongoDB writes - snk.mongodb streams the upstream through newline-delimited JSON on disk instead of buffering the whole result set, cutting a 1,000,000-row load from about 30 s to 8.7 s, and fixing a DECIMAL-as-string bug on the way.

Fixes

• #203 - a parallel ctl.foreach running a code.python node no longer shares scratch files, so concurrent iterations cannot read or delete each other's input, script, or output. The run's unique database name is folded into the py-in / py-out / py-*.py paths, and a concurrency-4 regression test asserts each row keeps only its own value.
• #201 - src.duckdb no longer marks database as unconditionally required, and two related property-name bugs in the Python API were corrected.
• #89 - the macOS AI assistant failure turned out not to be code signing but the build extracting the llama.cpp dylib symlinks as empty files, so llama-server had nothing valid to load. The extractor now replays symlinks. This could not be exercised on a Windows development machine, so reports from macOS users are especially welcome.
• Sinks refuse writes they cannot honour - a sink no longer silently falls back to plain inserts when handed a write mode or upsert key it does not support. mode="overwrite" on MongoDB is accepted as an alias for replace; an unknown mode, or an upsert with no key columns, is rejected at validate time before any data moves; and conflictColumns: "id" is read the same as ["id"].
• Derived columns and secret redaction - deriving a column that already exists now replaces it in place instead of leaving the old value under the original name; and path, pattern, and spatial property keys are no longer mistaken for secrets, so the SQL from validate and explain runs as written.
• #199 - Python pipeline limit serialization, contributed by @mattfaltyn.

Download

Desktop apps and headless runners for Windows, macOS (Apple Silicon and Intel), and Linux (x64 and arm64) are attached below. Verify against SHA256SUMS.txt. Also on PyPI: pip install duckle.