Skip to content

v0.7.0

Choose a tag to compare

@physwkim physwkim released this 17 Jul 23:26
· 57 commits to main since this release

Changed

  • tomoxide-gui's live-test dev-dependency epics-pva-rs is now 0.24
    (was 0.21). The in-process test server and rsdm's internal pvAccess client
    (^0.21 at the pinned rsdm) are coupled only by a TCP address, so the two
    copies coexist and talk wire protocol — verified by the live source tests.
  • tomoxide-gui now depends on rsplot / rsdm =0.5.2 (was =0.5.0).
    0.5.1 adds the VolumeRaycaster widget the XANES 3-D chemical-map view uses
    and 0.5.2 hardens it; the GUI builds against crates.io again, with the
    local-checkout [patch.crates-io] block commented out.

Added

  • recon::center::judge_sweep — a focus sweep's verdict, not just its
    argmax.
    Returns SweepVerdict::{Resolved, Railed, Ambiguous, Flat}, and only
    Resolved carries a value (SweepVerdict::resolved()); best() exists to
    show the winner on a plot or montage, never to adopt it. Every caller — align,
    the GUI's centre and tilt steps — now refuses rather than reports a number the
    curve did not establish, which matters because a bad axis silently poisons the
    tilt search that consumes it.
    The verdict is one uniform test — does this lobe own at least 20 % of the
    curve's height? — asked of the winner and of every other lobe. It replaces an
    edge check, which measurement showed is blind to both ways a real sweep fails.
    On the aligned pouch scan (known axis 396, tilt 44°), swept ±40 px: on one
    probed slice the curve carries three lobes and the highest is 417 — 21 px
    wrong, interior, beating the truth by 0.34 %, so no edge check can fire
    (Ambiguous, naming 396 as the strongest rival at 55 % prominence). On
    another the curve instead rises across the whole window and turns over one
    sample short of the edge at 435 — an interior maximum with no rival to
    contradict it, and still nothing but the range running out (Railed). Both are
    the same fact: the winner owns no lobe of its own. Narrowed to ±8 px around a
    prior, the same curve resolves 395.75 at 83 %. (These sweeps predate the
    z-centring fix below, which found the probe's slice index offset by
    (rh − nz)/2 from the volume z it claimed — the curve shapes and their
    verdicts, not the planes' names, are what these measurements establish.)
  • tomoxide align — the alignment workflow as a subcommand, plus the two
    library pieces it stands on. recon::center::find_center_rings is the ring
    estimator from docs/LAMINOGRAPHY_ALIGNMENT.md §1: the 360° mean projection is
    a bullseye centred on the rotation axis, registered against its own mirror. It
    reports a prominence — (peak − median)/MAD of the registration profile —
    and flags the scan when the rings never closed, which is the acquisition-time
    misalignment no reconstruction geometry repairs (measured: aligned scan 397.50
    at prominence 21.0 → trustworthy; misaligned scan 281.72 at 2.22 → flagged, and
    align stops there unless --force). cuda::center_probe / center_probe_sweep
    then refine it: cfunc_linerec's backprojection_try reconstructs one slice at
    N candidate centres in one launch off a single filtering. A probe slice
    equals the reconstruction exactly only when the shift is a whole number of
    columns (measured 2e-7…9e-7 of peak) and differs by ~1.6 % at half a column, so
    a naive sub-pixel sweep ranks integer offsets artificially sharp;
    center_probe_sweep removes that by construction, issuing one probe per
    fractional lattice, and never producing the biased slices at all. End to end on
    the aligned pouch scan, swept ±8 px around the axis read off the rings:
    --center 395.00 against a known 396. Around a prior is the whole contract —
    the same sweep widened to ±40 px grows three lobes and picks 417, and that is
    what judge_sweep above now refuses rather than returns.
    The two axes are searched by different machinery, and the asymmetry is physics,
    not an optimisation left undone. A fixed slice is sound for the centre — an
    in-plane shift does not move the in-focus layer, so one slice can rank the
    candidates — and unsound for the tilt, whose response is broad
    (~2 % per degree) while the in-focus layer moves in z with it (z_peak 800 →
    1120 as tilt went 40° → 58°), so a fixed slice scores the wrong plane by more
    than the tilt signal is worth. Measured: a fixed-slice tilt sweep returns 48° at
    one slice and rails to the top of any range at another. Hence
    recon::center::lamino_tilt_scan (--tilt_width / --tilt_step): one full
    reconstruction per candidate — that needs every (tilt, z) pair, which is
    exactly the work lamino_tilt_probe skips, so the probe buys nothing there and
    remains for the case its saving is real, a plane that is already known — scored
    by the max slice_focus over its slices, optionally inside the sample's z
    band
    (recon::center::SampleBand, --focus_z LO:HI) — the reference
    workflow's practice, and the guard wherever structured noise competes with
    the sample. A band fixed in z would miss the layer (it moves: the sample's
    spike sits at z 837 / 890 / 956 at tilts 40° / 44° / 48°), so the band is
    stated once, at the tilt it was read from, and carried into each candidate's
    volume through the detector rows — which belong to the data and do not move
    (v − nz/2 = cos(tilt)·(z − rh/2); the mapping reproduces those measured
    spikes to ~1 px, and that measurement is pinned as a unit test). End to end
    on the aligned pouch scan at 44°: the unbanded tilt curve over 36…52° is
    unimodal with its interior peak on the known 44° and a 3.0× span, every
    candidate's z_peak on the sample layer, the three measured spikes among
    them. The scan streams the volume, so its memory cost is one
    rh-tile regardless of depth, and it keeps what the reconstruction already
    computed: the winning in-band slice (§3 confirms by eye) and focus_by_z, the
    focus of every slice — the profile the next band is read off, so it is never
    truncated to the current one.
  • recon::center::pick_interior_max — the argmax of a sweep, plus whether the
    sweep earned the right to call it an optimum. A maximum on the first or last
    candidate is the range running out, and there is no way to tell from inside the
    range whether the true peak is at the edge or beyond it. Both align and the
    GUI refuse to adopt a railed pick; measured worth: it caught a real tilt scan
    railing at 47° on the pouch scan, and a centre sweep flat to 1.8 % and railing
    when pointed at an empty slice.
  • recon::center::slice_focus and mean_projection — the two measurements
    the alignment method is written in, as library functions rather than copies in
    each consumer. slice_focus is §2's score (mean |∇|² inside a 0.92-FOV disk;
    the disk matters — reconstruction geometry decides where the sampling leaves the
    detector, so scoring the full frame scores a boundary that moves with the
    parameter being scored). mean_projection is the bullseye find_center_rings
    measures, exposed because §1 makes looking at it step one and treats
    eye-vs-correlation disagreement as the misalignment flag — which needs the
    image, not only the number.
  • tomoxide-gui: laminography alignment on the Center screen (Beam:
    Parallel | Laminography). The toggle is not a display option: under a
    laminographic tilt the axis leans along the beam, so 0°/180° is not a mirror of
    the object and find_center_vo / _pc / _sift all lose the symmetry they
    assume (mirror registration scattered 395…607 against a known 396). It picks the
    estimator family valid for the beam. The three steps are the doc's, and each is
    a picture the CLI can only summarise as a number: 1 Rings shows the mean
    projection itself, so closed rings versus arcs that never close is a thing you
    see rather than a prominence you trust; 2 Centre is the probe sweep as a
    montage plus a focus curve with click-to-pick; 3 Tilt streams one result per
    full reconstruction — the in-focus slice into a montage, focus-vs-tilt, and
    focus-by-slice for the selected candidate — with a cancel, since it is minutes
    per candidate. Railed picks are shown as such and never adopted.
  • docs/LAMINOGRAPHY_ALIGNMENT.md — how to find the laminography rotation
    center and tilt.
    A field-tested recipe, validated against a scan whose center
    was already known independently. Leads with reading the center off the raw data
    before reconstructing anything: the mean of all projections over 360° is a
    bullseye of rings centred on the rotation axis, so the center column is
    readable by eye (automated flip-registration measured 397.5 vs a known 396),
    and rings that open into arcs instead of closing diagnose a scan that was
    mis-aligned at acquisition. Also documents the reconstruction sweep that
    follows (full projections; focus scored as the max over z, optionally confined
    to a sample band carried per tilt) and records the three methods that failed
    validation — 0°/180° mirror registration (the tilted axis breaks the mirror
    symmetry), 1-D column-profile symmetry (truncation destroys it), and
    sub-sampled projections with a fixed z band.
  • Fourier laminography (LamFourierRec) now honours --filter. The
    Fourier/USFFT lamino path applied a plain |f| ramp and ignored the selected
    apodisation, unlike tomocupy's LamFourierRec (which filters with
    args.fbp_filter, default parzen) and unlike tomoxide's own fbp/linerec lamino
    paths. The apodisation window is now folded into the projection ramp on all
    three paths (CPU golden, CUDA host, CUDA device) from a single window
    definition shared with make_fbp_filter. The default is parzen (tomocupy
    parity); pass --filter ramp for the previous plain-ramp behaviour.
  • Multi-GPU angle-split SIRT for laminography. Laminographic SIRT can now run
    across multiple GPUs by partitioning the projection angles into contiguous
    per-device chunks; each GPU holds the full volume and its angle chunk and
    contributes a partial tilted back-projection, with the correction all-reduced
    each iteration. The result is bit-identical to single-device SIRT.
  • CUDA laminography now streams out-of-core, across all GPUs. The tilted
    back-projector previously built the whole filtered stack in one shot on a
    single GPU, which overflowed the kernels' 32-bit index (nz·nproj·ncols ≈ 7.5e9 > 2³¹, SIGSEGV) and exceeded VRAM at production resolutions. It now
    mirrors tomocupy's laminography chunking: filter the stack once into host
    memory (nz-sub-chunked so the padded scratch stays under VRAM and the index
    ceiling), then a nested output-rh-tile × projection-angle-chunk loop uploads
    each filtered angle chunk and accumulates its back-projection into the tile.
    Multi-GPU shards the output rh axis across every selected device (the whole
    stack is filtered once, then per-device back-projection shards read it
    read-only). Chunk sizes are derived from free VRAM and the index ceiling. The
    fits-in-one-chunk case stays byte-identical to the previous single-shot path.
    The CLI now streams the result to disk as well: reconstruct_lamino_streaming
    (library + pipeline) emits the output one rh-tile at a time through a callback,
    so tomoxide-cli writes the volume tile-by-tile and never holds the whole
    reconstruction in host RAM. Tiles are computed a round of GPUs at a time and
    written from the main thread (the H5 writer is !Send), bounding host peak to
    the sinogram plus the filtered host stack plus one in-flight tile per device.
  • GUI: Live streaming reconstruction screen (tomoxide-gui, the seventh
    mode wired up — docs/GUI.md §2.6, milestone M3 first cut). Connects a
    tomoScanStream-style pvAccess projection stream through rsdm's headless data
    engine: NTNDArray frames land in a fixed-capacity ~180°-of-projections ring
    buffer (with rolling dark/flat), and a self-contained live thread
    reconstructs the selected Z (horizontal) slice every loop, re-reading the
    parameters each iteration (tomostream semantics — a center tweak or filter
    change applies on the next pass). Controls: PVA channel addresses
    (image/theta/companion width+height/optional dark+flat), slice index, live
    center + tweak, analytic algorithm, filter, Fourier-wavelet ring removal,
    buffer depth. rsdm delivers frames as a flat pixel array, so the detector
    width comes from a companion width PV (or a manual value) and the height is
    either a PV, a manual value, or derived from the frame length (len / width) — so only the width is ever required. The rotation angle likewise
    comes from a theta PV or a manual constant step per frame (deg/frame), for
    streams that publish no theta PV. When a theta PV is used each frame is paired
    with the latest scalar angle (no NTNDArray uniqueId is exposed). X/Y
    ortho panes need dedicated backprojection kernels (§6 #7) and are out of
    scope for this pass — Z-only, as the design's honesty note calls for. Verified
    end-to-end against an in-process epics_pva_rs::PvaServer (no beamline
    required).
  • tomoxide::xanes — per-voxel XANES peak-energy fitting (new xanes
    feature, off by default). Ports the reference txm_pal_core fit core
    (Levenberg–Marquardt quadratic / Gaussian white-line fits + Savitzky–Golay /
    median / 3-point / boxcar smoothers) to pure Rust with no PyO3. fit_peak_energy
    is the per-voxel core (smooth → locate peak → windowed curve fit → range-check
    → NaN on failure); fit_map is a rayon driver over a (E, z, y, x) f32
    volume view, parallel across z, that reads views and never materialises the
    full f64 stack (so a 40-energy 500³ volume streams a z-band at a time) and
    polls a CancelToken between slices. This is library-side milestone-M4 item
    #11.
  • tomoxide::xanes::MultiEnergyVolume — multi-energy stack reader (M4 #12)
    plus io::read_h5_band (coalesced [z0, z1) band hyperslab read, dtype-
    dispatching like read_h5_frame). A MultiEnergyVolume is a common-grid set
    of per-energy recon volumes — either separate files sharing a dataset key
    (tomoxide's own per-energy output, from_files) or one combined file with an
    explicit (energy, dataset) per energy (from_combined) — validated to one
    grid, sorted by energy, and read a z-band at a time into (E, band, ny, nx)
    f32 to feed fit_map without holding the full stack. Dataset keys are
    always caller-supplied; the reader never guesses a combined file's per-energy
    naming scheme.
  • tomoxide::xanes::write_peak_map_h5 + io::list_h5_datasets. The
    results writer emits a fitted map (peak_energies, energies, edge_jump,
    finite-voxel mask) in the layout the xanes_tools Python viewer reads; the dataset
    lister returns every key in a file so a combined-stack loader can discover
    per-energy volumes without guessing the writer's name formatting.
  • tomoxide::xanes — zone-plate magnification correction (M4 #14). Ports the
    reference magnification_correction.py: magnification_corr_factors derives a
    per-energy scale factor from the zone-plate focal-length model (focal length
    grows with photon energy, so each energy images the sample at a slightly
    different magnification), normalised to the first energy; apply_magnification
    rescales a (z, y, x) volume about its centre by that factor (bilinear over
    z/x, the y rotation axis untouched, zero-fill outside — matching
    scipy.ndimage.affine_transform(diag(cf, 1, cf), order=1, mode="constant")).
    MagnificationParams carries the zone-plate geometry with reference defaults.
  • GUI: XANES chemical-mapping screen (tomoxide-gui, a seventh mode). Loads
    a combined registered stack (registered.h5: an energies axis +
    reconstructions/{energy} volumes — per-energy keys discovered by parsing the
    dataset-name leaves, no formatting assumed), streams it a z-band at a time
    through xanes::fit_map on a cancellable background thread with a live
    progress bar, and browses the peak-energy map slice-by-slice (a click reads
    that voxel's spectrum; a histogram summarises the map). Fit controls cover
    method (quadratic / Gaussian), window width, energy range, smoothing, a
    mean-absorption mask threshold, and band size; the result saves via
    write_peak_map_h5. This is milestone-M4 item #13 (viewer); the stack it
    consumes is produced upstream (energy-looped recon + external registration).
    The central panel toggles between the 2-D slice browser and a 3-D direct
    volume rendering
    of the chemical map (rsplot's new GPU VolumeRaycaster:
    front-to-back alpha-composited ray-march, orbit / pan / wheel-zoom). The
    transfer function reuses the same viridis peak-energy colormap as the 2-D map
    for hue, with fitted voxels opaque and unfitted (masked / out-of-window)
    voxels transparent; an opacity slider scales the composite and the volume is
    decimated to a bounded edge so the GPU upload stays small (docs/GUI.md §2.7).
  • GUI: XANES per-energy magnification correction (tomoxide-gui). A
    side-panel toggle applies xanes::apply_magnification to each energy's volume
    before fitting, driven by editable zone-plate geometry (magnification, ZP
    diameter, outermost zone width) with a live preview of the resulting factor
    range. Because the correction scales the z axis it cannot be applied to the
    streamed z-bands independently, so enabling it reads the whole stack into
    memory and fits from the corrected copy; the default streaming path is
    unchanged (docs/GUI.md §6 #14).
  • GUI design: XANES spectroscopic-mapping screen (docs/GUI.md §2.7, a
    seventh mode). Post-reconstruction per-voxel white-line / edge-shift fitting
    over a multi-energy volume stack, reusing the existing txm_pal_core Rust
    fit core (Levenberg–Marquardt + rayon, no GPU); 2-D chemical maps +
    histogram + click-to-spectrum, with a results-HDF5 save interoperable with
    the xanes_tools Python viewer. New library additions — fitting-module port,
    multi-energy stack reader, energy-to-energy registration, zone-plate
    magnification prep — are catalogued in §6 (#11–#14) and milestone M4 (§7).
    Per-energy reconstruction reuses the existing CUDA streaming pipeline
    (run_streaming_pipelined_range) looped over energies with the current
    DXchange preprocessing unchanged — no new recon or prep code. Energy-to-energy
    3-D registration (SimpleITK rigid Mutual-Information) has no pure-Rust
    equivalent and stays an upstream Python step in v1.

Fixed

  • Laminography volume z-centring unified: a slice index — and the probes'
    sz — now names the same volume z in every algorithm.
    The linerec-family
    CUDA kernels (back-projection, both try-probes, and the forward projector)
    centred the volume z on nz/2, the detector-row midpoint, while the
    Fourier/USFFT path centres it on rh/2, the volume's own midpoint — so for a
    tilted axis (rh > nz) every Linerec/Fbp/SIRT laminography volume, and every
    probe slice, sat (rh − nz)/2 slices off the Fourierrec volume of the same
    data: 200 slices on the reference pouch scan (rh 1424, nz 1024). This is the
    defect behind the centre sweep's slice dependence — a probe pointed at the
    "middle of the volume" was in fact scoring 200 slices above it. Every kernel
    now takes the full reconstruction height and centres z on rh/2; the parallel
    beam passes rh == nz, so that path is bit-identical, and the tilt probe
    carries a per-tilt height (the in-focus height depends on the tilt). A new
    test (cuda_lamino_z_convention) pins the convention by placing a slab at a
    known off-centre z and requiring Linerec and Fourierrec to put its peak at
    the same volume z — it fails under the old centring. tomoxide align now
    defaults the centre-sweep slice to the --focus_z band's midpoint — the
    plane the sample occupies — falling back to rh/2 (--focus_z no longer
    requires --tilt_width). Validated on the reference scan: probed at the band
    midpoint (slice 890) the sweep resolves 395.00 against the known 396;
    probed at the empty volume midpoint the focus response is ~100× weaker and
    judge_sweep refuses the railed curve instead of reporting a number.
  • align and the GUI Center screen applied minus-log twice. Both preceded
    their reconstructions with normalize_dataset — which already ends in the
    minus-log — and then called minus_log again. −log of a line integral
    amplifies the noise floor everywhere the integral is near zero (air) while
    leaving the sample recognisable, so every recon these paths scored was
    quietly noise-inflated: on the aligned pouch scan the double-prepped volume
    put a pure-noise plane 1.7× above the eye-confirmed electrode plane
    (1.95e-5 vs 1.15e-5), which made whole-volume focus scoring unusable and
    was misdiagnosed as a defect of the focus metric itself. With the prep
    applied once, the same metric on the same scan puts the sample ≈6× above
    the noise floor, the unbanded tilt scan resolves the known 44° with a
    3.0× span (previously a 1.4 % spread with no identifiable optimum), and the
    rings' flip-registration reads 397.50 at prominence 21.0.
    normalize_dataset's doc now states the minus-log is included.
  • Fourier laminography reconstruction sign corrected. The Fourier/USFFT
    lamino path (recon::lamino::lamino and its CUDA analytic mirror) reconstructed
    dense material as negative attenuation — inverted versus the physical
    minus-log convention that linerec/fbp (direct back-projection) and SIRT (physical
    forward model) follow. The cause was a global -1 in tomocupy's irfftshiftc
    fft2d step, ported faithfully; it is a pure sign, independent of the fftshift
    centering. It is now dropped in fft2d_fwd and, symmetrically, in the paired
    fft2d_inv, so the reconstruction matches linerec/fbp/SIRT (positive material)
    while the fft2d_inv(fft2d_fwd) == id round-trip and CPU↔CUDA parity are
    preserved. Output now differs from tomocupy's raw fourierrec by a global sign
    only (magnitudes identical).