Skip to content

Duckle v0.6.0

Choose a tag to compare

@github-actions github-actions released this 08 Aug 08:05
· 8 commits to main since this release

A multimodal AI data store, an importer for legacy visual ETL jobs, a chat model you choose, and two geometry transforms that finally have the second input they always needed.

New

Pixeltable, read and write (#223). src.pixeltable reads a table, optionally narrowed by a Pixeltable where expression, a column subset and a limit. Versioned reads work by passing myapp.media:3. snk.pixeltable inserts into a table that already exists, or creates one from the incoming rows.

The exchange runs over Parquet on both legs, which is the one format both sides already do well: Pixeltable exports and Duckle ingests with read_parquet; on the way back Duckle writes Parquet that Table.insert accepts directly. Nothing is serialised a row at a time.

Pixeltable is a Python library with no Rust client, so the desktop app provisions a private Python for it with uv the first time a node needs it. Nothing is installed into your own environment, and deleting <app-data>/pixeltable/ removes it completely. It is fetched on demand rather than at startup because it is a large dependency tree for a connector most workspaces never touch.

Verified end to end against a real pixeltable 0.7.1 install: a pipeline reading a 10-row table with t.score > 0.4 and writing the result back returned exactly the five rows the filter should keep.

Choose the assistant's model, from 14. Setup installed one hardcoded 1.5B model, which is the right default on a laptop and the wrong one on a workstation with a GPU. The catalogue now spans 469 MB to 9.9 GB: Qwen2.5 Coder 0.5B through 14B, Qwen3 4B and 8B, Llama 3.2 1B and 3B, Phi-3.5 Mini, Mistral 7B v0.3, Qwen2.5 7B, Gemma 2 9B, and DeepSeek Coder V2 Lite. Each carries its real download size, read from the Hugging Face file listing rather than estimated, and an honest note on what it needs to run well.

Every entry was checked to resolve before being offered, so the picker cannot hand you a file that 404s halfway through a multi-gigabyte download. One candidate was dropped for being a gated repo that needs a token.

The first binaries published under this tag did not actually show the picker: the catalogue, the command and the argument were all in place, but the install card never rendered a control, so setup silently installed the default. These binaries are a re-roll that fixes it. If you downloaded before this was corrected, take the current file.

Import jobs from legacy visual ETL tools. Reads the XML those jobs are stored as and produces a Duckle pipeline. Measured on a real 44-job corpus: all 44 parse, and 211 of 216 nodes map, the only refusal being a site-specific custom component.

Three things deliberately do not convert, because a silent guess is worse than a stated gap. Encrypted passwords become ${ENV:...} placeholders. Connections whose credentials live outside the job file are reported rather than half-imported. Anything with no equivalent is imported as a labelled placeholder, so the shape of the job survives on the canvas instead of quietly losing a step.

CI for your pipeline repo. duckle-runner is now published as a release asset, and there are ready-to-copy workflows for GitHub Actions and GitLab CI under docs/ci/. They gate every push and pull request on duckle-runner validate, which compiles pipelines to SQL without opening a source, writing a sink, or needing credentials or a network.

This catches the failure nothing else does: rename a column in one commit, keep referencing the old name in another, and the two merge cleanly with no conflict. No file lock would have prevented it. Only a compile finds it.

The desktop binary cannot serve this job, so do not substitute it: Duckle-linux-x64 is a Tauri binary with libwebkit2gtk and the GTK stack in DT_NEEDED, resolved at load time, so it cannot start on a runner with no GUI libraries. The published runner is static musl with no shared-library dependencies and runs on alpine or in a scratch container.

Fixed

Excel date columns work without configuration (#225). An Excel-native date is stored as a serial number, and because a declared schema reads every cell as text, it reached the cast as the string 46037 and failed with "invalid date field format". Serial conversion existed but only ran when a column's format was set to the literal excel, which autodetect never sets and nothing in the interface suggests, so it was unreachable in practice. Date and timestamp columns with no format now try the text date first and fall back to the Excel epoch, so both shapes load with nothing to configure. Round-tripped a real workbook: a cell holding 2026-01-15 reads back as 46037 and now yields 2026-01-15, while text dates are untouched. Thanks to @OmarMustaafa for the sample file and the exact error.

Clip and Erase keep the coordinate system (#218 follow-up). Writing either result to a Shapefile produced no .prj: correct geometry, no error, missing projection. DuckDB carries a geometry's CRS in the column's type, and both transforms cast the input to the unparameterised GEOMETRY, discarding it before the sink could write it. The overlay functions propagate CRS from their arguments, so the input is simply no longer cast. Verified by writing real Shapefiles, which now carry the projection. One consequence worth knowing: the input must be a real geometry rather than WKT text, so a text column needs Create Geometry upstream. Reported by @OmarMustaafa while testing the fix above.

The assistant's model picker now exists. Described above: the choice was reachable from the API and from nothing else. The install card carries a dropdown, shows each model's real size and note, and preselects the recommended model rather than the smallest.

Pixeltable can be provisioned. The connector needs a private Python, and the function that creates it was only reachable from a command nothing called. An existing install was detected at startup, so this was invisible to anyone who had one, but a first-time user could not get one: the node fell back to the system python3, which does not have the library. Both run paths now provision it the moment a pipeline actually contains a Pixeltable node, which is what its description always promised.

Both were found by auditing every command and bridge export for a caller, rather than by anyone hitting them.

Clip and Erase can actually be wired up (#217, #218). Both shipped in v0.5.9 as two-layer overlays, and the engine required the second layer, but the palette declared only one input. The canvas decides which connections to offer from exactly that list, so there was no port to attach the clip or erase layer to: the node could be placed and configured and never run.

Both now offer a second input labelled clip layer / erase layer, like Spatial Join. Behaviour is unchanged: the second layer is still dissolved with ST_Union_Agg before the operation, every attribute of the input layer is preserved, and features left with nothing are dropped.

Thanks to @OmarMustaafa for reporting it on both issues with screenshots. A test now pins this contract for every component whose builder requires a second input, and it was checked by removing a port again and confirming it fails naming the component. A green build did not catch the original fault, because nothing compared what the engine requires against what the palette offers.

Node ids no longer collide. Adding a folder or duplicating an item minted an id from the clock alone, so two of the same kind created in the same millisecond could share one. Both now use the same timestamp-plus-random scheme as the rest of the canvas. That property is what makes pipeline JSON safe to merge: two branches adding nodes to one pipeline combine cleanly rather than colliding.

Downloads

Your machine File
Windows (most PCs) Duckle-windows-x64.exe
Windows on ARM (Surface, Snapdragon) Duckle-windows-arm64.exe
macOS, Apple silicon (M1 and later) Duckle-macos-arm64
macOS, Intel Duckle-macos-x64
Linux x86-64 Duckle-linux-x64
Linux ARM64 Duckle-linux-arm64
CI / headless validation duckle-runner-linux-x64

No installer and no admin rights: it is a single self-contained binary. Verify what you downloaded against SHA256SUMS.txt.

First run. Windows SmartScreen shows "unknown publisher" because the binary is not code-signed yet; choose More info then Run anyway. On macOS the file needs chmod +x and, because it is not notarised, xattr -d com.apple.quarantine ./Duckle-macos-arm64. On Linux it needs chmod +x and WebKitGTK present (sudo apt install libwebkit2gtk-4.1-0 on Debian or Ubuntu).

384 components, 366 available today.