Skip to content

DOPE 1.0.0 (uncertified research release)

Choose a tag to compare

@jepsontaylor jepsontaylor released this 22 Oct 04:31
· 108 commits to main since this release

DOPE 1.0.0 is an uncertified research release. It claims no tier certification, sealed-holdout result, formal differential privacy, HIPAA de-identification or other privacy guarantee. Canonical tasks E03–E06 (development pilot, signed precommit, sealed confirmation, independent reproduction) are still open; they gate a future certified release, governed by docs/release.md.

What is in it

  • DOPE engine 1.0.0 (engine/): a pure-Rust compiler and sampler. dope fit train.csv --target y writes a few-kilobyte safetensors program that stores no rows; dope sample draws training rows from it. CI tests the CPU paths only; the CUDA and Metal builds are not exercised in CI.
  • Bounded artifact decoding: before it allocates model arrays, the engine decoder enforces admission limits (64 MiB stored and inflated, a 1 MiB header and metadata, at most 64 sections and 4096 features). It checks every size with overflow-safe arithmetic and refuses cyclic copula trees. Malformed or over-limit files return an error. The limits leave format version 1 and the encoder's bytes unchanged.
  • Real-world benchmark (bench/real-world/): 16 public UCI and PMLB tables, three auditor families, the equal-byte xz control, SDV and ARF generators, the research kernel, and ablations. All 480 declared cells are recorded; 16 failed cells (timeouts and budget misses) stay in every denominator.
  • DOPE 1.0 manuscript (paper/dope1*.tex), with every number generated from the benchmark records.
  • Research kernel (dope-kernel, root crate) keeps version 0.3.0-alpha.1, along with all of its recent work (see CHANGELOG.md).

Measured results (generated, not hand-typed)

Measured on 16 public tables (UCI and PMLB), held-out real test rows. Robust retention is the worst of three auditor families (linear/logistic, CatBoost, MLP) trained only on the method's rows; medians over tables with 95% bootstrap intervals.

Method Median bytes Robust retention Tables ≥ 0.9
DOPE, 2 KiB budget 2.0 KiB 0.905 [0.815, 0.939] 8/16
DOPE, 4 KiB budget 3.8 KiB 0.909 [0.843, 0.932] 9/16
DOPE, 16 KiB budget 5.7 KiB 0.933 [0.869, 0.965] 10/16
DOPE research kernel (dope-kernel fit-table) 1.1 KiB 0.677 [0.484, 0.844] 3/16
Real rows, xz-compressed to 4 KiB 4.0 KiB 0.340 [-4.012, 0.918] 5/16
Real rows, xz-compressed to 64 KiB 50.8 KiB 0.983 [0.950, 0.998] 12/16
Gaussian copula (SDV defaults) 82.0 KiB 0.282 [0.094, 0.447] 1/16
ARF (arfpy defaults) 673.1 KiB 0.705 [0.000, 0.856] 3/16
TVAE (SDV defaults) (7 tables) 247.7 KiB -0.460 [-1.361, 0.611] 0/7
CTGAN (SDV defaults) (7 tables) 712.9 KiB -1.897 [-3.060, -0.473] 0/7
All real rows, xz-compressed (reference) 52.2 KiB 0.999 [0.996, 1.000] 16/16

Transfer Description Length at τ = 0.9: DOPE reaches it on 11/16 tables (median 1.9 KiB); xz-compressed real rows reach it on 16/16 (median 50.8 KiB).

At equal bytes, wherever DOPE fits the budget, it beats xz-compressed real rows in 60 of 92 paired (table, budget) comparisons and loses 32 (wins/ties/losses by budget: 1 KiB 11/0/1, 2 KiB 14/0/2, 4 KiB 11/0/5, 8 KiB 10/0/6, 16 KiB 10/0/6, 64 KiB 4/0/12). DOPE could not fit within 1 KiB on 4 tables: the schema plus the leanest program alone exceeded that budget.

Against the repository's research kernel (median 1.1 KiB) on the 14 regression and binary tables it supports, DOPE's robust retention wins/ties/losses are at 1 KiB 9/0/5, Hodges–Lehmann +0.010 [-0.394, +0.126]; at 2 KiB 12/0/2, Hodges–Lehmann +0.166 [+0.047, +0.291]; at 4 KiB 11/0/3, Hodges–Lehmann +0.160 [+0.045, +0.306]. On the 2 multiclass tables the kernel refuses the target or fits its class codes as a regression; its row above counts them as measured.

These numbers are copied verbatim from bench/real-world/results/readme-block.md, which bench/real-world/render.py generates from results/confirm/summary.json.

Known limitations

  • The engine CLI and the research kernel's facade are both named dope; keep only one on PATH (docs/install.md).
  • bench-check checks the rendered surfaces against summary.json, but it does not recompute summary.json from records.jsonl.
  • report.py ranks with a sort that is not stable, so the self-audit Spearman coefficient can differ in its sixth decimal across environments.
  • The tag is annotated but not signed, and no binaries are attached. Build from the tagged source with cargo build --release --manifest-path engine/Cargo.toml.