Skip to content

v0.6.0

Choose a tag to compare

@physwkim physwkim released this 06 Jul 01:44
· 138 commits to main since this release

Changed

  • tomoxide-gui now depends on rsplot 0.5.0 from crates.io (was the
    siplot git dependency pinned to a rev). The plotting crate and its GitHub
    repo were renamed siplot → rsplot (and the sibling EPICS engine
    sidm → rsdm) and published to crates.io; the GUI switches to
    rsplot = "=0.5.0" with a commented-out [patch.crates-io] for local dev.
    Mostly a use siplot → use rsplot identifier rename. The one behavioural
    gap: the local siplot carried an unpushed commit making a Plot2D
    crosshair read out the pixel value under the cursor ("x, y, value"); rsplot
    0.5.0 instead exposes the value via the higher-level ImageView
    (value_changed, silx PositionInfo "Data"). The Data sinogram inspector and
    the Tune single-slice preview were migrated Plot2D → ImageView to keep the
    value readout (the preview's ColormapDialog is replaced by ImageView's
    interactive colorbar). GUI build/clippy/35 tests green on CPU and CUDA. The
    Data sinogram inspector and Tune preview ImageView layout was fixed and
    verified on-screen: the ImageView position-info bar is emptied and the value
    readout is pinned to the bottom so the image fills exactly the available
    height, keeping the sinogram's resizable bottom panel stable (an earlier
    version let it grow into / collapse away from the projection browser).

Added

  • CLI --progress_json (recon, recon_steps) — one flushed JSON line
    per completed output chunk on stdout
    ({"start":s,"end":e,"total":nz,"secs":t}; global slice range, full output
    slice count, wall-clock seconds since run start), implemented as a thin
    VolumeWriter tee. The multi-GPU shard orchestrator forwards the flag and
    lets children inherit stdout, so shard lines stream through with global
    ranges against one total. Machine progress for wrappers — the GUI Run
    screen tails these from its subprocess runs. Runtime-only (not a config
    key); progress lines are exactly the stdout lines starting with {.
  • Cooperative cancellation for the chunked drivers — new
    pipeline::CancelToken (Clone-able atomic flag) attachable via
    ReconSteps::with_cancel; all three drivers (run, run_streaming,
    run_streaming_pipelined_range) check it at chunk boundaries and stop with
    the new Error::Cancelled. Cancellation truncates: chunks already written
    stay on disk and the writer finalizes the partial output.
  • io::InMemoryWriter — a VolumeWriter that collects the reconstruction
    into a shared in-memory InMemoryVolume (the Arc<Mutex<…>> handle
    survives the pipelined driver consuming the writer) with an optional
    on_chunk progress callback. Backing store for GUI previews.
  • tomoxide::config (feature config) — the CLI's TOML Config moved
    into the library behind a default-off feature (optional serde/toml
    deps) so GUI recipes and CLI configs are one format. Gains three fields:
    lamino_angle, dtype, and output (base path; each writer adds its own
    suffix). The CLI gained the matching --output flag, --config now feeds
    all three, and the multi-GPU z-shard fan-out forwards the resolved output
    path to its children.
  • io::read_h5_frame — read one [ny, nx] frame of a 3-D HDF5 stack as
    f32 through the reader's full dtype dispatch (u8/i8/u16/i16/u32/i32/f32/
    f64). Real beamline stacks are usually uint16; the GUI projection browser
    reads through this (rsplot's own HDF5 loader handles 4/8-byte floats only).
  • io::read_h5_sizes — the (n, ny, nx) shape probe paired with
    read_h5_frame, so a volume browser can size its frame list without
    reading any data.
  • Vo 2018 single-method stripe variants through config/CLI — the
    long-implemented StripeMethod::VoSort/VoFilter/VoLarge/VoDead/
    VoFit are now selectable as --remove_stripe vo-sort | vo-filter | vo-large | vo-dead | vo-fit, with per-method config fields/flags
    (vo_sort_*, vo_filter_*, vo_large_*, vo_dead_*, vo_fit_*;
    median sizes use the fw_level convention 0 = tomopy auto) and
    multi-GPU shard forwarding.
  • tomoxide-gui M1 (offline preview loop) — new repo-internal but
    workspace-excluded crate (rsplot is edition-2024/rust-1.92; workspace
    membership would raise the repo's effective MSRV above 1.82) implementing
    docs/GUI.md M1: a single worker thread owns the Engine and all HDF5
    handles (!Send); Data (DXchange open + metadata, projection browser —
    frames load through io::read_h5_frame so uint16 beamline stacks render,
    with the colormap scaled to the stack's raw-count range — theta plot, raw
    sinogram inspector), Tune (single-slice preview through
    run_streaming_pipelined_range into io::InMemoryWriter, parameter panel
    with auto-recon, A/B pin compare), Center (Vo / entropy /
    phase-correlation / SIFT auto methods, ±0.5/±0.25 px tweak, hand-off to
    Tune), and recipe save/load (recipe file = CLI config TOML plus a [gui]
    table the CLI ignores). tomoxide-gui [FILE] [--mode <mode>] opens a
    dataset and/or picks the starting mode from the command line, and Tune
    fires the first preview of a fresh dataset by itself (the auto toggle
    still gates re-runs on parameter changes).
  • GUI Tune — λ sweep (L-curve regularization tuner). For algorithms whose
    reg_par[0] is a regularization strength λ (tv, grad, tikh,
    pml_*, ospml_*), a new worker Job::LambdaSweep reconstructs the preview
    slice once per λ across a log-spaced grid and scores each on the L-curve —
    data residual ‖A x − b‖₂ (the fidelity term, via sim::project) vs the
    reconstruction's isotropic TV seminorm (roughness). A floating window shows
    the per-λ montage over the L-curve; the max-distance-to-chord corner is the
    suggested λ, a click picks any point, and "Use selected λ" writes it to
    reg_par[0]. The guide is the L-curve corner, not a sharpness auto-pick:
    sharpness falls monotonically with λ and real data has no ground truth
    (docs/BENCHMARKS.md §10), so the choice stays the user's. Verified finite
    and λ-varying on both CPU and the CUDA device-resident 1-slice path.
  • GUI design document (docs/GUI.md) — design for a tomoxide-gui
    desktop application built on rsplot (egui + wgpu) and rsdm (EPICS PVA):
    offline workflow (dataset browsing, single-slice tune loop with A/B
    compare, center finding with a write_center sweep montage, subprocess
    full-volume runs, output browsing) plus a tomostream-style live streaming
    mode, with the prioritized list of library additions it requires.

Added

  • ext_pad — truncated-projection support extension for iterative
    methods.
    Real samples routinely overhang the field of view, so the
    projections don't end at zero; an iterative forward model whose support is
    the detector-width grid then dumps that inconsistency into a huge FOV-edge
    ring and background offset that swamp the (intact) interior — on real
    800-wide data a 10-iteration CGLS correlated only 0.56 with fbp full-frame
    while agreeing 0.99 in the interior. With ReconParams::ext_pad (CLI
    --ext_pad, config ext_pad, GUI Tune "extend FOV" — on by default in the
    GUI) the sinogram is edge-replicate extended by ncols/4 per side, the
    solve runs on the wider grid, and the central crop is returned; the wrapper
    sits above the backend dispatch so CPU/CUDA/wgpu see the identical extended
    problem. Real-data result: CGLS-10 full-frame correlation with fbp 0.56 →
    0.996, interior display contrast restored to fbp's level. Off by default in
    the library (tomopy-parity semantics unchanged; ~2.25× cost per iteration).

Changed

  • Analytic reconstructions now emit the physical μ — the shared
    make_fbp_filter base ramp is the physical |ω| inversion filter (peak
    0.5 at Nyquist) instead of tomopy/tomocupy's doubled ramp (peak 1), and
    recon::gridrec's ramp_scale drops its matching empirical ×2. Every
    analytic method (FBP, linerec, fourierrec, lprec, gridrec, on all backends)
    therefore reconstructs the attenuation μ per pixel-unit rather than 2×μ —
    the same scale the iterative solvers converge to, so fbp→iterative
    warm-starts are now scale-consistent. All analytic output amplitudes are
    halved
    ; downstream code that hard-coded the old scale (rescale windows,
    8/16-bit export ranges) must adjust. Cross-method and cross-backend ratios
    are unchanged (both sides halve). Pinned by tests/analytic_amplitude.rs (a
    unit disk reconstructs to core mean ≈ 1.0). This is a deliberate departure
    from both upstreams, whose absolute analytic amplitude is itself
    convention-dependent (tomopy gridrec ≈ 1.16×μ, tomocupy ≈ 4/π×μ).
  • Iterative reconstructions now converge to the physical μ — the
    forward/back-projector pair used by the iterative solvers is the plain
    line-integral Radon transform W and its pure adjoint Wᵀ on every
    backend (CPU, CUDA, wgpu); the π/nproj FBP angular-quadrature weight
    previously baked into both operators is now passed by the analytic FBP
    call sites only, where it belongs. A converged SIRT/CGLS/MLEM/… solve of
    W x = p therefore lands on the attenuation per pixel-unit (pinned by the
    new tests/iterative_amplitude.rs against an analytic disk sinogram)
    instead of (nproj/π)·μ — e.g. ≈ 143× smaller at 450 projections —
    matching ART/BART, which always solved the ungained ray equations. Iterate
    trajectories are unchanged up to that overall scale for the self-scaling
    methods (SIRT/CGLS/MLEM/OSEM/PML/OSPML); for grad/tikh/tv the fixed
    step and regularization now act on the physical scale, so hand-tuned
    reg_par values from before may need retuning. The grad/tv host and CUDA
    gain-compensation machinery (adj_scale/fwd_gain_inv) is deleted.
    Analytic (FBP/gridrec/…) outputs are unchanged.
  • GUI preview autoscale is percentile-robust — image colormaps scale to
    the 0.5–99.5 % range instead of the absolute min/max, so a handful of
    extreme pixels (e.g. the FOV-edge ring iterative methods produce on
    truncated-FOV data) no longer own the whole gray range and flatten the
    interior structure.
  • sift-center is pure Rust — find_center_sift now runs on the
    lowe-sift crate instead of the opencv binding, dropping the system
    OpenCV + clang build requirement entirely. The uint8 normalization stays
    bit-exact vs numpy; the SIFT stage is an independent implementation of
    Lowe's paper, so recovered shifts land within ~0.034 px and the center
    within 0.008 px of the cv2 golden (tolerances in sift_center_parity
    updated from float-noise to algorithmic bounds). The feature now needs
    rustc ≥ 1.92 (above the 1.82 MSRV, which is unchanged for default builds).
  • tomoxide-gui builds with sift-center on by default (alongside
    cuda), so the Center screen's SIFT method is always available with no
    extra system packages. A feature-gated smoke test pins the SIFT center
    call chain end-to-end.
  • HDF5/TIFF writers are zero-copy — H5Writer/TiffWriter handed each
    chunk through an elementwise gather copy before writing; a standard C-layout
    chunk (what every driver produces) is now passed straight to the write call
    (as_slice), with the gather kept only as a non-contiguous fallback. The
    gather was the H5 writer's dominant cost (~2× the raw file write at 512³).
  • The HDF5 reconstruction output is finalized without fsync — new
    VolumeWriter::finalize hook (called once by every driver on the success
    path); H5Writer implements it with rust-hdf5 0.3.1's close_no_sync
    (complete, valid HDF5; durability left to OS page-cache writeback, matching
    the TIFF/Zarr writers). Dropping the writer without finalize (error paths)
    still closes durably. Also removes the H5 writer's per-chunk flush()
    (a documented no-op in rust-hdf5) and its stale "durable partial output"
    doc claim. Combined effect on a 512³ fbp streaming recon (1 GPU): h5 output
    1.61 s → 0.84 s, on par with tiff (0.83 s), bit-identical output.
    rust-hdf5 0.3.2 supplies the second half of that win: its create no longer
    ftruncates a brand-new empty file — that truncate armed ext4
    auto_da_alloc, whose implicit writeback inside the final close(2)
    (~325 ms at 512³) silently defeated close_no_sync.
  • The HDF5 writer unlinks a stale output before creating — re-running a
    reconstruction over an existing .h5 output now lands on a fresh inode
    instead of truncating the old file, so the overwrite rerun no longer pays
    the auto_da_alloc writeback either (was +0.44 s end-to-end at 512³). The
    unlink trades away rust-hdf5's lock-before-truncate protection for this one
    regenerable output file.
  • The Zarr writer emits <f4 chunk bytes zero-copy on little-endian
    targets (bytemuck safe Pod cast — bytemuck core is now an unconditional
    dependency, its derive feature still gated behind gpu-wgpu); the
    per-element to_le_bytes gather remains only for big-endian targets and
    non-contiguous callers.

Fixed

  • CUDA analytic reconstruction of a single slice was silently all-zero
    (and an odd Fourierrec slice count a hard error). The z-bilinear
    back-projection kernel samples slice pairs, so it needs a ≥2-slice batch,
    and cfunc_fourierrec packs slice pairs, so it needs an even one — but a
    1-slice job (GUI preview, recon --start_row R --end_row R+1) built the
    streaming handle at capacity 1 and the one-shot path handed the kernels the
    raw count. Both now pad the batch with zero rows up to the kernel domain
    (≥2, even for Fourierrec) and drop the pad rows from the output, reusing
    the existing partial-chunk machinery; FourierReconstruct::reconstruct
    likewise zero-pads an odd stack instead of erroring. Multi-slice outputs
    are unchanged (single-row CLI recon is bit-identical to the same row of a
    multi-row run).
  • CUDA iterative reconstruction of a single slice was garbage (the same
    batch-domain family as the analytic fix above, via the other kernel pair):
    the device-resident solvers and the FilteredBackproject/ForwardProject
    wrappers share the z-bilinear projection kernels, so a 1-slice problem
    forward-projected to zero and the solve iterated on nothing — a GUI Tune
    preview of sirt/tv showed garbage. IterativeReconstruct::solve and both
    wrappers now duplicate the slice into a 2-slice problem (exact: the z-interp
    weights sum to 1 on identical rows, and EM ratios stay finite where zero-pad
    rows would 0/0) and drop the duplicate; the 1-slice solve equals the same
    slice of a multi-slice solve.
  • fourierrec output was uniformly π·nd² smaller than every other method
    (read as "all zeros" on real data — ~10⁻⁸ at nd = 800) on all three
    backends: the deapodization missed the Δθ = π/nang angular quadrature
    weight and compensated the inverse-FFT normalization against the wrong
    reference amplitude. Invisible to every parity test because they compare
    Pearson-style (scale-invariant) or fourierrec-to-fourierrec; the best-fit
    fbp/fourierrec amplitude is now pinned ≈1 by a regression test. Host
    (phi_amp = π·nd²/nang), CUDA (divphi ×π/4 over the unnormalized cuFFT
    inverse), and wgpu (deapodize norm = π/4) all land on the unified
    fbp/tomopy scale; cross-backend ratios are unchanged.
  • gridrec disagreed with fbp/fourierrec on real data and sat on an
    arbitrary amplitude.
    Three defects in one method: (1) its output was
    never masked to the detector-width disk, so gridding leakage outside the
    field of view dominated the frame (corr vs fbp 0.36 on real 800-wide data
    while agreeing 0.97 inside a 0.9-radius disk); (2) its radial FFT was
    zero-padded, so the nonzero borders of real (truncated-FOV/absorbance)
    projections became a hard step that rang across the FOV-edge annulus — it
    now edge-replicates the padding exactly like FbpFilter::apply; (3) its
    ramp weight and deapodization were unnormalized (the Kaiser–Bessel pair's
    W/I₀(β) constant was dropped and no polar density compensation applied),
    leaving a size-dependent scale (~2600× below fbp at n=128) — samples now
    carry 2π·|ρ|/nang and the true KB constant, landing on the unified
    fbp/tomopy amplitude. Pinned by amplitude and truncated-projection
    regression tests; real 800-wide data now: gridrec↔fourierrec corr 0.992,
    gridrec↔ramp-fbp corr 0.963, scales within 3 %.
  • recon --start_row/--end_row was silently ignored by the whole-volume
    paths
    (algorithms without a streaming handle — gridrec and the iterative
    set, everything on CPU/wgpu — plus --algorithm chains): a 1-row request
    read and wrote all rows. Both whole-volume branches now read only the
    requested detector-row band and write it at its global slice offset
    (matching the streaming/shard semantics); an explicit row range under
    laminography is rejected (the tilt couples all rows) instead of dropped.

Documentation

  • docs/BENCHMARKS.md §10 — FBP vs iterative accuracy against a known-truth
    phantom.
    Settles whether FBP's sharper look is accuracy or contrast using a
    synthetic Shepp–Logan phantom (piecewise-const + a TV-adversarial textured
    variant), inverse-crime-mitigated (generate at 2×, bin detector, recon at 1×),
    with a transmission-Poisson noise sweep. Finding: a properly-regularised
    iterative recon (TV, λ tuned to the noise) is closest to truth in every regime
    (≈ halves FBP NRMSE at 450 views), FBP is the noise-robust floor whose extra
    detail is contrast not accuracy, and unregularised fixed-iter CGLS is best at
    high SNR but worse than FBP under noise. Also documents that real-data dense
    references are method-dependent (FBP vs iterative disagree at r ≈ 0.73–0.81),
    qualifying the §1 quality-proxy caveat.