Duckle v0.5.4

Duckle v0.5.4

SAP connectivity, a faster dbt loop, custom AI gateways, schema Autodetect, and a batch of reliability fixes from reported issues.

New connectors

• Native SAP sources. src.sap reads SAP OData v2 / v4 and CDS services, and src.sap.rfc calls classic RFC-enabled function modules over SOAP (RFC-over-SOAP), so SAP data lands in a pipeline without a separate middleware layer. Both carry proper SAP, OData, and GizmoSQL brand marks on the canvas.

Transforms & AI

• Custom OpenAI-compatible AI endpoints (#142). xf.ai.llm, xf.ai.classify, and xf.ai.embed gain a "Custom" provider with an optional base URL, endpoint path, and extra headers, so they can reach any OpenAI-compatible gateway (self-hosted, a proxy, or a corporate LLM endpoint). Existing pipelines are byte-for-byte unchanged.
• Multi-column Cast with per-column error handling (#144). A single Cast node now converts many columns at once, each with its own target type and format, and each with its own on-error behavior (inherit the node default, set NULL, or fail the run). A hard failure names the offending column.

Performance

• Faster xf.dbt loop: pre-warmed project parse (#146). dbt's dominant cost is parsing the project (Jinja + manifest), not running the model. When a dbt node sits behind upstream stages, Duckle now warms dbt's parse in the background while those stages run, so dbt run reuses a warm cache instead of paying a cold parse. The warm-up only touches the dbt project's target/ directory, never the run database. Set DUCKLE_DBT_PREWARM=0 to opt out.

Reliability & correctness

• Autodetect returns the real source schema, or a clear error (#148). "Autodetect from Source" now inspects the actual source, including driver and API sources (Oracle, SQL Server, ClickHouse, Snowflake, Databricks, and newly added connectors), and returns its real columns instead of a placeholder col_1 / col_2 / col_3. When it cannot reach the source it surfaces the real reason (bad credentials, unreachable host, unsupported source) rather than substituting a fabricated schema, and ${ENV:...} secrets are resolved before it runs. The duckle serve web editor uses the same inspection path.
• Portable pipeline artifacts across operating systems (#145). A pipeline built into a single-file artifact on one OS no longer bakes the build host's path into ${workspace}; the artifact keeps the placeholder and re-resolves it on the run host (honoring a DUCKLE_WORKSPACE override), so a Windows-built artifact runs correctly on Linux and vice versa.
• SQL Server: Encrypt toggle for legacy servers (#141). An Encrypt setting lets you disable TLS when connecting to older SQL Server instances whose handshake modern TLS rejects.
• SQL Server: XML columns and deep-in-table type conflicts (#141 follow-up). XML columns from a SQL Server source are now read in full instead of coming back empty, and schema inference scans the whole extract, so a column whose type only varies deep in a large table no longer aborts the load. A preview-grid tooltip also reveals the full text of long cells that the table view clips.
• SQL Server: Autodetect works for every column type (#141 follow-up). "Autodetect from Source" on a SQL Server source used to fail (and before that crash) on a table containing a sql_variant column or a CLR type such as geometry, geography, or hierarchyid: the driver cannot describe those types, and Autodetect probes with SELECT *, so it met them even on tables a normal run would not. Autodetect now reads the schema from the server's own catalog, so it returns the real name and type of every column regardless of type, and the preview renders spatial / variant / hierarchyid values as text. A plain three-int table (and everything else) autodetects cleanly again.
• In-app updater: checksum mismatch on a re-rolled build (#156). When a release's binaries were rebuilt but one build leg failed, the published SHA256SUMS.txt could keep the previous checksum for a changed binary, so the self-updater aborted every update with a "checksum mismatch" error. Checksums are now regenerated to match the binaries actually on the release, even after a partial rebuild, so the updater verifies and installs cleanly.
• Geospatial: shapefile .prj and native GEOMETRY (#150, #151). Writing a shapefile now emits its .prj projection sidecar (the coordinate system is preserved across the run instead of being dropped), and a GEOMETRY column read from a spatial source is detected as a first-class geometry type rather than plain text.
• Parquet: text columns no longer written with literal quotes (#140). JSON-typed columns coming from SQL Server are cast to VARCHAR before a Parquet write, so string values land unquoted.
• Truncate-and-insert write mode for Oracle and SQL Server (#138). Replace a table's contents in place without dropping it.

Canvas & data mapping

• One Join node (#153). The four separate inner / left / right / full join components are now a single Join node whose type is chosen from a dropdown, so the node's label can never disagree with the join it performs. Existing pipelines that reference the old join types keep working.
• Multi-column joins honored (#152). A Join with several key pairs now joins on all of them instead of collapsing to the first key, so multi-key joins return the correct rows.
• SQL name alias is emitted into the plan (#154). A node's SQL name now becomes a real view in the compiled plan and SQL export, so downstream steps and the exported code can reference it.
• Filter reject port de-duplicated (#120). The Filter node exposed two outputs that mapped to the same reject stream; it now has a single, clearly labelled reject output.

Smaller download

• Leaner desktop download. The four sidecar binaries embedded in the app (DuckDB, the Linux runner, the MCP server, and the LanceDB engine) are now zstd-compressed inside the executable and inflated on first use, with no change in behavior. That trims the download substantially, for example Windows x64 from about 152 MB to about 104 MB.

Editor & settings

• Opt-in unsigned DuckDB extension loading (#143). A Settings toggle allows loading unsigned / community DuckDB extensions; off by default. It applies immediately, no relaunch. Headless runs use DUCKLE_ALLOW_UNSIGNED_EXTENSIONS.
• Editable git remote (#139). Change a workspace's git remote URL from inside the app.
• Environment variables resolve in canvas and scheduled runs (#137). ${ENV:KEY} placeholders are honored consistently across an interactive Run and a scheduled run.
• Scheduler interval fixes (#135) so recurring runs fire on the configured interval.
• snk.rest form-encoded bodies. The REST sink form-urlencodes its body when the Content-Type calls for it (for example a Stripe-style endpoint).
• Plain-text / raw bodies for snk.rest and snk.webhook (#147). A new Body type option sends a raw text/plain body built from a ${column} template, so InfluxDB Line Protocol can be written straight to QuestDB (or any line-oriented HTTP endpoint) without a proxy. Each row renders to one line and the batch is newline-joined into a single request.
• Global font scaling for the whole editor, plus a cleaner connection-URL display.

---

Every fix above landed in the desktop app, the duckle serve web editor, and the headless duckle-runner (and, where relevant, the duckle-mcp catalog).