Skip to content

Many-grain density support (dense/streaky frames)#7

Merged
marinerhemant merged 2 commits into
mainfrom
feature/many-grain-density
Jul 23, 2026
Merged

Many-grain density support (dense/streaky frames)#7
marinerhemant merged 2 commits into
mainfrom
feature/many-grain-density

Conversation

@marinerhemant

Copy link
Copy Markdown
Member

Enables indexing of dense, streaky, many-grain frames — hundreds of overlapping grains
per image, no depth resolution — which the pipeline previously mishandled. This is the
core capability behind the July 2026 34-ID-E Ti-6Al-4V campaign (13 scans, ~150k frames).
Validated against multi-grain synthetic ground truth.

1. Robust blur sizing (laue_index/preprocess.py)

The auto matching-blur sigma was sized from the absolute minimum neighbour distance,
which collapses on dense frames: fragments of a single streak sit a few px apart, shrinking
sigma to ~1 px and leaving the 0.4° orientation grid unable to reach spots up to ~18 px from
a grid seed. Recovery dropped to ~15% on a 150-grain synthetic frame. Fragments of one
streak should merge — they are one grain's reflection — so size from the 10th percentile
of neighbour distances. This changes the default preprocessed output; golden fixtures updated
to match.

2. GaussSigmaMax parameter (laue_stream_utils.py + preprocess.py)

Optional cap on the auto sigma. On dense frames a large blur lights so much of the detector
that chance matches flood the coarse gate and bury true grains; iterative-peel drivers start
tight and widen per pass. Default 0.0 = no cap, so existing behaviour is unchanged unless
the param is set.

3. MAX_MATCHES 100k → 4M (LaueMatchingGPU.cu, LaueMatchingGPUStream.cu)

The kernel appends matches in arrival order, so overflow discards candidates arbitrarily —
true grains included — before the top-score merge can rank them
. 100k overflowed on
many-grain frames. 4M × 12 B × MAX_STREAMS is < 200 MB on device and pinned host.

4. Torn-row tolerance (read_solutions)

Tolerates truncated rows (daemon terminated mid-write on a dense frame), dropping malformed
rows with a warning rather than failing the whole parse.

Plus

GenerateSimulation.py gains multi-grain synthetic frames (orientations_to_recips) — the
known-orientation, known-count ground truth used to measure recovery above.

Tests

Char suite and the streaming regression tests (#5) pass. The .cu change needs the GPU
build (build-gpu) to verify — the CPU build does not exercise it.

Deliberately not included: the laue_torch README bullet and its torch/midas-hkls
requirements, held until the associated publications go through.

…eiling

Enables indexing of dense, streaky, many-grain frames (hundreds of overlapping
grains per image) that the pipeline previously mishandled. Validated against
multi-grain synthetic ground truth during the July 2026 34-ID-E beam time.

Three coupled changes:

1. laue_index/preprocess.py — robust spot spacing.
   The auto blur sigma was sized from the ABSOLUTE minimum neighbour distance,
   which collapses on dense frames: fragments of a single streak sit a few px
   apart, shrinking sigma to ~1 px and leaving the 0.4-deg orientation grid
   unable to reach spots up to ~18 px from a grid seed (recovery dropped to
   ~15% on a 150-grain synthetic frame). Fragments of the same streak SHOULD
   merge — they are one grain's reflection — so size the blur from the 10th
   percentile of neighbour distances instead of the minimum. This changes the
   default preprocessed output; the golden fixtures are updated to match.

2. GaussSigmaMax parameter (laue_stream_utils.py + preprocess.py) — optional
   cap on the auto sigma. On dense frames a large blur lights so much of the
   detector that chance matches flood the coarse gate and bury true grains;
   iterative-peel drivers start tight and widen per pass. Default 0.0 = no cap.

3. MAX_MATCHES 100k -> 4M (LaueMatchingGPU.cu, LaueMatchingGPUStream.cu).
   The kernel appends matches in arrival order, so overflow discards candidates
   ARBITRARILY — true grains included — before the top-score merge can rank
   them. 100k overflowed on many-grain frames. 4M x 12 B x MAX_STREAMS is
   < 200 MB on device and pinned host.

Also in laue_stream_utils.py: read_solutions tolerates torn/truncated rows
(daemon terminated mid-write on a dense frame), dropping malformed rows with a
warning rather than failing the whole parse.

Char suite and the streaming regression tests pass with these changes.
Add calc_recip_array / orientations_to_recips and accept a list of orientations,
so a single frame can superpose many grains' patterns. This is the synthetic
ground truth used to validate the many-grain density work — known orientations,
known count, to measure recovery.
@marinerhemant
marinerhemant merged commit 7e5fef4 into main Jul 23, 2026
2 checks passed
@marinerhemant
marinerhemant deleted the feature/many-grain-density branch July 23, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant