DOPE Rust engine 1.1.0

DOPE Engine 1.1.0 — 2026-10-28

Source: 15e12aa3dc33851655580f6aa7c9c66353143f23, tree 4f76daed5165ac20ed649a42ec000cb1c715497b.
Release checklist: PR #346, reviewed head 86eaf983b99ad84d248704e1d955e12ec071ccce.
Intended annotated tag: dope-engine-v1.1.0.

This engine update adds optional and progressive landmark conditioning, Rust table preparation, and an additive 512-coordinate encoder API. The published model-weight release remains v1.0.0.

• Optional and progressive landmarks. --landmark-mode none|fast|progressive|full and --landmark-subsample N expose the evidence policy. None/fast skips native landmark probes. Progressive starts with no landmark evidence and adds the macro, larger, and fine groups using shared fold preparation. Its default sample size is 300; full remains unsampled unless requested. The bundle records finite values, an observation mask, status, sampling, and source/options identity. Missing evidence stays unavailable.
• Rust preparation and compatibility. Static descriptors describe the complete training partition; sampled landmark and building-block evidence uses the selected physical rows. Default Cap32V1 preparation preserves the released class policy, while explicit LosslessV1 supports larger FIT vocabularies. The full conditioning recipe is named full-report-v1; its former legacy-full-v1 digest mapping is recorded in the qualification evidence. Existing 629-coordinate APIs remain available.
• Resident 512-coordinate API. MlcV3Model::{encode_descriptor_512_tensor, encode_features_512_tensor, encode_conditioning_512} returns the bit-identical first 512 coordinates of the 629-coordinate encoding on the same backend, avoiding the unused graph-feature computation. CUDA-enabled source builds support resident tensors; cross-backend comparisons use numerical tolerances. This additive API uses the existing checkpoint and requires no new training.
• Preparation improvements and diagnostics. Exact-output table preparation, descriptors, building-block probes, and landmark reductions reduce repeated work. Stage-timing APIs expose preparation costs. Tests cover sampled conditioning, mixed columns, signed zeros, nonfinite-input handling, and lossless class vocabularies.
• CLI and integration documentation. Interactive terminal summaries, the quick start, and the Rust composition guide document the supported host/tensor APIs and landmark flags. The reviewed release checklist supplies the final installer tag/help behavior and a single dated Engine 1.1.0 changelog section.
• Dependency repair. The merged zstd update addresses RUSTSEC-2026-0391/0398/0403 in the affected workspaces. The release retains locked dependency inventories and upstream notices.

The prebuilt package is the Linux x86-64 GNU CPU executable plus build, dependency, license, checksum, and Sigstore provenance sidecars. It contains no Python runtime, libtorch, CUDA runtime, model weights, or sample wire packet. GPU-enabled builds are separate source builds; local CUDA parity evidence does not turn the CPU package into a CUDA binary.

The installer still defaults to the combined v1.0.0 release. For the Engine 1.1.0 binary, use the explicit engine-only route below and fetch model weights separately. GitHub CLI 2.102.0 or newer is required for the documented verification policy; this host's gh 2.62.0 does not support --source-ref. Use a current patched GitHub CLI release. The installer compatibility/version diagnostic and independent engine/model tag support are a separate follow-up; the instructions below do not imply those repairs are already in this source.

Engine-only installation after release verification:

bash scripts/install.sh --tag dope-engine-v1.1.0 \
  --source-digest 15e12aa3dc33851655580f6aa7c9c66353143f23 --require-attestation --no-weights
dope mlc-v3 fetch-weights --tag v1.0.0 --preset multitask --json

Use scripts/install.sh from the exact source commit above. Engine-tag attestation binds the binary and checksum inventory to that commit. The separate weight fetch verifies the compiled-in SHA-256 for released multitask weights: d0b6960328d1955386632ff78b512e3add53a9b0256db7bbf067e2e0be5f0c6c (1,781,800 bytes). Neither command was run as part of this preparation task.

Qualification remains bounded. The ≥5× cold fast/full latency target is not passed across the measured CUDA cells. Retention evidence identifies unavailable development tables and stratification arms; it is not a JopeDime policy-quality or world-class predictive result. Candidate ecded022 remains outside the released default pending owner registration/rights decisions. This is an uncertified engine release: E03 and the sealed certification work remain parked, and no privacy, de-identification, HIPAA, production-authority, or new model-promotion claim is made. Sigstore provenance establishes build identity, not those scientific or operational properties.

Source-bound references:

• Changelog
• Optional-landmark qualification and its limitations
• Signed engine package and verification contract
• Rust composition and downstream integration
• Changes since the previous signed engine

DOPE Rust engine dope-engine-v1.0.0

Signed CPU Rust engine for Linux x86-64. Verify binary and SHA256SUMS attestations before installation. Signing establishes build identity; it does not certify model quality or privacy. See docs/signed-engine-release.md.

DOPE 1.0.0 (uncertified research release)

DOPE 1.0.0 is an uncertified research release. It claims no tier certification, sealed-holdout result, formal differential privacy, HIPAA de-identification or other privacy guarantee. Canonical tasks E03–E06 (development pilot, signed precommit, sealed confirmation, independent reproduction) are still open; they gate a future certified release, governed by docs/release.md.

What is in it

• DOPE engine 1.0.0 (engine/): a pure-Rust compiler and sampler. dope fit train.csv --target y writes a few-kilobyte safetensors program that stores no rows; dope sample draws training rows from it. CI tests the CPU paths only; the CUDA and Metal builds are not exercised in CI.
• Bounded artifact decoding: before it allocates model arrays, the engine decoder enforces admission limits (64 MiB stored and inflated, a 1 MiB header and metadata, at most 64 sections and 4096 features). It checks every size with overflow-safe arithmetic and refuses cyclic copula trees. Malformed or over-limit files return an error. The limits leave format version 1 and the encoder's bytes unchanged.
• Real-world benchmark (bench/real-world/): 16 public UCI and PMLB tables, three auditor families, the equal-byte xz control, SDV and ARF generators, the research kernel, and ablations. All 480 declared cells are recorded; 16 failed cells (timeouts and budget misses) stay in every denominator.
• DOPE 1.0 manuscript (paper/dope1*.tex), with every number generated from the benchmark records.
• Research kernel (dope-kernel, root crate) keeps version 0.3.0-alpha.1, along with all of its recent work (see CHANGELOG.md).

Measured results (generated, not hand-typed)

Measured on 16 public tables (UCI and PMLB), held-out real test rows. Robust retention is the worst of three auditor families (linear/logistic, CatBoost, MLP) trained only on the method's rows; medians over tables with 95% bootstrap intervals.

Method	Median bytes	Robust retention	Tables ≥ 0.9
DOPE, 2 KiB budget	2.0 KiB	0.905 [0.815, 0.939]	8/16
DOPE, 4 KiB budget	3.8 KiB	0.909 [0.843, 0.932]	9/16
DOPE, 16 KiB budget	5.7 KiB	0.933 [0.869, 0.965]	10/16
DOPE research kernel (dope-kernel fit-table)	1.1 KiB	0.677 [0.484, 0.844]	3/16
Real rows, xz-compressed to 4 KiB	4.0 KiB	0.340 [-4.012, 0.918]	5/16
Real rows, xz-compressed to 64 KiB	50.8 KiB	0.983 [0.950, 0.998]	12/16
Gaussian copula (SDV defaults)	82.0 KiB	0.282 [0.094, 0.447]	1/16
ARF (arfpy defaults)	673.1 KiB	0.705 [0.000, 0.856]	3/16
TVAE (SDV defaults) (7 tables)	247.7 KiB	-0.460 [-1.361, 0.611]	0/7
CTGAN (SDV defaults) (7 tables)	712.9 KiB	-1.897 [-3.060, -0.473]	0/7
All real rows, xz-compressed (reference)	52.2 KiB	0.999 [0.996, 1.000]	16/16

Transfer Description Length at τ = 0.9: DOPE reaches it on 11/16 tables (median 1.9 KiB); xz-compressed real rows reach it on 16/16 (median 50.8 KiB).

At equal bytes, wherever DOPE fits the budget, it beats xz-compressed real rows in 60 of 92 paired (table, budget) comparisons and loses 32 (wins/ties/losses by budget: 1 KiB 11/0/1, 2 KiB 14/0/2, 4 KiB 11/0/5, 8 KiB 10/0/6, 16 KiB 10/0/6, 64 KiB 4/0/12). DOPE could not fit within 1 KiB on 4 tables: the schema plus the leanest program alone exceeded that budget.

Against the repository's research kernel (median 1.1 KiB) on the 14 regression and binary tables it supports, DOPE's robust retention wins/ties/losses are at 1 KiB 9/0/5, Hodges–Lehmann +0.010 [-0.394, +0.126]; at 2 KiB 12/0/2, Hodges–Lehmann +0.166 [+0.047, +0.291]; at 4 KiB 11/0/3, Hodges–Lehmann +0.160 [+0.045, +0.306]. On the 2 multiclass tables the kernel refuses the target or fits its class codes as a regression; its row above counts them as measured.

These numbers are copied verbatim from bench/real-world/results/readme-block.md, which bench/real-world/render.py generates from results/confirm/summary.json.

Known limitations

• The engine CLI and the research kernel's facade are both named dope; keep only one on PATH (docs/install.md).
• bench-check checks the rendered surfaces against summary.json, but it does not recompute summary.json from records.jsonl.
• report.py ranks with a sort that is not stable, so the self-audit Spearman coefficient can differ in its sixth decimal across environments.
• The tag is annotated but not signed, and no binaries are attached. Build from the tagged source with cargo build --release --manifest-path engine/Cargo.toml.