v0.2.0

graintrace 0.2.0

This release makes the NF/FF/EBSD experiment path work end to end inside graintrace: state the frame transform between datasets, reconstruct NF with a confidence mask and graph segmentation, mesh it with SCULPT, and match grains across datasets and load steps. It contains API and default changes, so read Breaking and behaviour changes before upgrading.

Highlights

• FrameTransform (new, graintrace.frame_transform, #61): a stated sample-frame change x' = Q(s⊙x) + t, g' = g Qᵀ, ε' = Q ε Qᵀ that moves positions, orientations and strain tensors consistently. Tilt is part of Q; s is a per-axis sign only (no scale). It has adapters for FF grain CSVs, point clouds, voxel grids and the headerless reconstruction_cpfe_ee.csv, plus inside_voxel_solid for building overlap masks. Inverse, composition and JSON records are exact. Orientation math goes through neml2.
• NF confidence mask (#62): min_confidence, max_point_distance and voxel_size_um in NearFieldMeshBuilder.reconstruct, NFGridConversion and the NF point-cloud gridding, so air and low-confidence points become void instead of being nearest-neighbour filled. The per-voxel confidence is written beside the grid.
• Graph (Leiden) segmentation in NearFieldMeshBuilder (#62): segmentation={"method": "graph", ...} with an optional in-plane downsample. The .mic reader also accepts MIDAS per-layer subfolders.
• Size-aware split detection (#63): FragmentationAnalyzer.detect_splits gains d_tol_mode="radius"/"radius_sum", assignment="one_to_one" (Hungarian), per-side recall with in_overlap_a/b masks, and a chance-link null (n_null).
• Segmentation without a grain-size cap (#64): FragmentationAnalyzer.segment no longer caps grain size by default. Small fragments are absorbed only into a neighbour of similar orientation (absorb_max_miso_deg). The adjacency re-merge is agglomerative and checks the merged cluster's spread. cluster_leak_check detects orientation leaks. seg_to_grain_table(trim_miso_deg=...) gives a trimmed grain mean.
• .ang converter (new, graintrace.ang_conversion.AngConverter, #56): converts EDAX/TSL .ang layers to .mic for NearFieldMeshBuilder, or to the merged x, y, z, Eul0..2 CSV for VoxelMeshBuilder. Layers are ordered by the parsed layer index.

Breaking and behaviour changes

• graintrace.nf.convert.pointcloud_to_fixed_grid now returns (grid, confidence) instead of grid.
• Passing nx/ny together with voxel_size_um raises ValueError.
• SCULPT --void_mat is now set by graintrace. SCULPT numbers spn values from 1, so the void (spn 0) is material 1. The previously documented --void_mat 0 meshed the void as a block. mesh_sculpt now passes --void_mat 1 when the grid has void and replaces a user value with a warning. Remove --void_mat from your sculpt_options.
• SCULPT meshes are mapped from SCULPT's voxel-index coordinates exactly. Earlier meshes were shrunk by one voxel per axis, and masked (void-bordered) grids were stretched over the full scan box.
• SCULPT block ids are renumbered to a contiguous 1..N. <mesh>_block_map.csv records each block's original SCULPT id and grain. Orientation row k belongs to block k+1.
• A phase column is no longer treated as a void marker implicitly. Opt in with void_marker_col.
• Isolated solid islands left unsegmented after cleanup become void instead of aborting write_spn.
• FragmentationAnalyzer.segment defaults: no grain-size cap, orientation-gated absorption, and a spread-checked re-merge. Segmentations may keep more small, distinct grains than before. percolation_max_frac=0.03 restores the previous behaviour exactly.
• grain_fragments uses gamma=0.25 when seg_kwargs gives none.
• orientation_helper.matrix_to_euler wraps φ1/φ2 to [0, 360) (or [0, 2π)) and resolves gimbal lock deterministically. The rotations are unchanged, but stored Euler values can differ numerically.
• VoronoiMeshBuilder(rotate_angles=...) now rotates orientations on the sample side (g Qᵀ). The old crystal-side update was a no-op for cubic crystals. orientation_active_convention is deprecated and ignored.
• FrameTransform rejects non-unit quaternions. detect_splits raises on negative or NaN tolerances, and treats NaN or negative radii as 0 with a warning.
• AngConverter defaults to TSL Bunge radians and raises when angles labelled as radians exceed 4π.

Fixes

• VoronoiMeshBuilder auto_rotate no longer raises on about 8% of inputs; the axis signs are aligned before handedness is restored.
• FrameTransform.mrp returns the short MRP set; it no longer blows up to NaN near 180° rotations.
• The chain-merge in adjacency_remerge without a cap is fixed (#64). Labels below 0 and the background label are left untouched consistently across segmentation helpers.
• Grain-orientation averaging folds crystal-symmetry variants first (#20).
• An unsegmented voxel no longer corrupts the SCULPT mesh. Discarded flood segments stay rejected (#33). Mesh blocks are matched to their own grain's voxels (#38).
• The flood out-of-domain mask is applied in place. Experiment rotation refuses a row-count mismatch. IPF block colouring defaults to Bunge degrees (#55).
• detect_splits keeps a parent's best candidate when no two children are adjacent (#37).
• Graph-affecting and weighting parameters are validated when resuming a segmentation checkpoint (#18, #22). The mutual-kNN neighbour count is clamped to N−1 (#16). DBSCAN summaries are labelled with their columns (#17). Top-k prune ties are resolved consistently, with a faster half-edge grouping (#39, #41).
• validate_topology refuses a broken tessellation at any verbosity (#42). GrainMatcher output files follow output_prefix (#40). Examples write to their own output folders (#36). Lazy-import errors name the install tier needed (#34).

Documentation

• New API page for FrameTransform, a .ang section in File formats, and updated configuration, pitfalls and tutorials (NF reconstruction, NF+FF, reorientation/fragmentation, voxel segmentation).
• The docs build now lists pandoc as a prerequisite (#35). The maths and tutorial pages are expanded (#54).

Upgrade notes

1. Drop --void_mat from sculpt_options, and re-mesh any masked NF grid: earlier meshes included the air as a block and were stretched.
2. Unpack grid, confidence = pointcloud_to_fixed_grid(...).
3. For NF data, use min_confidence (about 0.5–0.6) with graph segmentation instead of relying on a grain-size cap.
4. To move FF data into the NF frame, state the transform with FrameTransform instead of rotate_angles with orientation_active_convention.

v0.1.3

graintrace v0.1.3

A documentation-focused release. It adds a full documentation site (published to GitHub Pages) and lands the CPFE output-control and offline grid-resampling improvements. No breaking API changes; the pure-Python subset is unchanged.

Documentation (new)

• Documentation site at https://applied-material-modeling.github.io/graintrace/, built with Sphinx and deployed from CI (pydata theme with top-nav tabs: Getting started, Tutorials, Algorithms & theory, Guides, API).
• Notebook tutorials, one per workflow stage. The pure-Python ones (REI 2D/3D, REI comparison) execute at build time with real figures and carry an "Open in Colab" badge; native-stack tutorials (reconstruction, meshing, calibration, CPFE, grain tracking) are shown with a "requires the native stack" note.
• Algorithms & theory section: the model, equations, and algorithm behind each stage, with references and links to the tutorial and API.
• Guides: install (three tiers), quickstart, concepts and data formats, a full configuration reference, the MCP server, benchmarks, common pitfalls, and a docs-development page.
• API reference generated with autodoc; docstrings filled in on the major builders and runners (VoronoiMeshBuilder, VoxelMeshBuilder, NearFieldMeshBuilder, CPFESimulation, GridResampler, MaterialCalibration, IdentifyRareClusters).
• Docs CI: strict build plus doctest on every push/PR, and a "definition of done" rule (new code updates tests, docstrings, docs, CLAUDE.md, and the MCP layer).

CPFE & simulation

• Output-frequency controls for a run: grid_transfer (final / per_step / off), exodus_output (sync / per_step), and mesh_csv (sync / per_step / off), including a crisp per-element CSV written on the true mesh for full-fidelity rare-event identification.
• Offline grid resampler (GridResampler): regenerate regular-grid field CSVs from a saved CPFE Exodus after a cheap run, without re-solving.
• Configurable MPI launcher (mpiexec / srun) and automatic resolution of the NEML2 / libtorch library directories from the puma-opt repo layout.

Benchmarks

• bench_cpfe.py: --summarize sweep rollup, records ncore and output modes, and fails fast on GPU OOM.

Fixes

• Dependency check updated for the current pyzag adapter symbol.

v0.1.2

graintrace 0.1.2

Second maintenance release. Headlines: pyzag is now installed automatically, CPFE runs end-to-end on the rebuilt NEML2 v3 / PUMA stack, a no-CUBIT voxel hex mesher, bring-your-own NEPER, CI, a benchmark suite, and a substantial docs overhaul.

Packaging & install

• pyzag==2.0.0 is now a pip dependency — pip install graintrace installs it automatically (it's pure-Python). NEML2 stays a PUMA source build (the PyPI NEML2 wheel pins an incompatible pyzag, so it's intentionally not used).
• Three-tier install documented by capability: Python-only → NEML2-only PUMA path (no full MOOSE build) → full CPFE.
• NEPER is now bring-your-own via the new graintrace/neper_env.py resolver (NEPER env var → graintrace_tools.json → PATH), with an opt-in auto_install=True Linux ~/.local build. gmsh ships via pip.
• Environment standardized on the PUMA build env; dropped the environment.yml step.

Features

• CPFE runs end-to-end on the rebuilt NEML2 v3 / PUMA stack, pinned to moose_neml2_v3 (run_cpfe.i fix).
• No-CUBIT voxel mesher — VoxelMeshBuilder.mesh(mesher="voxel") dumps the grid straight to Exodus (one cube hex per voxel, scaled-Jacobian = 1), alongside the default SCULPT hex path.
• Taylor model grain-base optimization for material calibration.
• Stitching: optional NEPER-tessellation extent refinement in scan_tessellation.

Tooling & CI

• CI added: lint (pylint) + formatting (black) + tests with coverage (run_tests.yml, formatting_check.yml).
• Benchmark suite under benchmark/ (calibration, CPFE, graph segmentation) with a shared harness.

Documentation

• README: renamed subtitle "Grain Texture and Response Analysis for Crystal Exploration", broadened the description beyond CPFE, added a capabilities table and an examples table, and a repo-clone note (examples/benchmark/mwe_data don't ship on PyPI).
• FF meshing default corrected: SCULPT hex (from the voxelized reconstruction) is the default; NEPER/gmsh tet .msh is a fallback, and CPFE runs on either hex or tet. Propagated across README, CLAUDE.md, skills, and MCP recipes.

v0.1.1

Fixed

• HEDM stitching noise model is now physically consistent. Synthetic
  measurement error was previously injected as a single fractional noise_level
  applied multiplicatively to all of X, Y, Z, GrainRadius, and the three Euler
  angles. For centroids this made the absolute error scale with distance from
  the origin (zero at the origin, large far away) and put it in different units
  than the stitcher's absolute position_tolerance. Noise is now split into
  three independent, physically meaningful channels.
• Fixed a latent cost-normalization bug in ScanStitchingComparison: the -1
  "disable" sentinel was used as a divisor, which could invert the sign of a
  weighted cost term. Normalizers now guard on > 0, matching
  RegionBaseStitching.

Changed

CrystalGenerator.write_to_csv and hedm_zscan no longer accept
apply_noise, apply_noise_method, or noise_level. Use instead:

Old	New
noise_level (fractional, all columns)	position_noise_std (absolute, length units, additive on X/Y/Z)
	orientation_noise_std (degrees, proper misorientation)
	radius_noise (relative fraction of GrainRadius)
apply_noise=True/False	set any std > 0 to enable; all-zero = noiseless
(none)	noise_seed for reproducible noise

Each channel is applied only when its value is > 0, so the default is
noiseless (equivalent to the old apply_noise=False).

Added

• orientation_helper.perturb_orientation(euler, sigma_deg, ...): applies a
  proper random misorientation of a given angular std about a uniformly random
  axis (gimbal-free), replacing the old per-Euler fractional perturbation.

Migration

# before
cg.hedm_zscan(..., apply_noise=True, noise_level=0.01)

# after (physical units; tune to your experiment)
cg.hedm_zscan(
    ...,
    position_noise_std=3.0,      # micrometers
    orientation_noise_std=0.5,   # degrees
    radius_noise=0.05,           # 5% relative
    noise_seed=42,
)

v0.1.0

Initial released of graintrace