Refinement Module

[friedkitteh]

Description of the change

Methods to refine:

1. orientations
2. projection centers
3. both simultaneously

The initial orientations are stored in a CrystalMap, with the projection center estimates from an EBSDDetector. The refinement process is made into an optimization problem and uses the user-friendly and efficient numerical routines implemented in scipy.

Progress of the PR

• Docstrings for all functions
• Unit tests with pytest for all lines
• Clean code style by running black via pre-commit

Minimal example of the bug fix or new feature

>>> import kikuchipy as kp
>>> s = kp.data.nickel_ebsd_large(allow_download=True)
>>> s.remove_static_background()
>>> s.remove_dynamic_background()
>>> mp = kp.data.nickel_ebsd_master_pattern_small(projection="lambert", energy=20)
>>> r = sampling.get_sample_fundamental(
... resolution=4, space_group=mp.phase.space_group.number
... )
>>> detector = kp.detectors.EBSDDetector(
... shape=s.axes_manager.signal_shape[::-1],
... pc=[0.421, 0.7794, 0.5049],
... sample_tilt=70,
... convention="tsl",
... )
>>> sim = mp.get_patterns(
... rotations=r,
... detector=detector,
... energy=20,
... dtype_out=s.data.dtype,
... compute=True
... )
>>> xmap = s.dictionary_indexing(sim, n_slices=10, keep_n=10)
>>> refined_xmap = s.orientation_refinement(
... xmap=xmap, 
... master_pattern=mp, 
... detector=detector, 
... energy=20
... )

For reviewers

• The PR title is short, concise, and will make sense 1 year later.
• New functions are imported in corresponding __init__.py.
• New features, API changes, and deprecations are mentioned in the
  unreleased section in doc/changelog.rst.

[friedkitteh]

Okay! Finally ready for the first review. I will be offline while I relocate and expect to be back online sometime around July 9th.

[hakonanes]

Great! Good work. It'll be reviewed within then.

[hakonanes]

I've tried out the functionality on the "large" Ni EBSD data set included in the kikuchipy.data module (user guide) and on the single crystal Si pattern used in the PC calibration moving screen user guide), and based on a qualitative assessment, I'm really stoked about the results!

---

Normalized cross correlation scores after dictionary indexing to 333 227 simulated Ni patterns (used EMsoft orientation sampling):

Before refinement	After refinement

Histogram of the normalized cross correlation scores shown in the maps above:

---

Geometrical simulations of Si as done in the PC calibration user guide ("Reference frames", link above). Note that the initial orientation is determined from TSL, while the PC is solely determined by the moving screen technique. PC in TSL's convention (x*, y*, z*) and the normalized cross correlation score are shown in the figure titles.

Moving screen (0.5123, 0.8606, 0.4981), r = 0.0199	PC refinement (0.5234, 0.8302, 0.4978), r = 0.3675	Orientation and PC refinement (0.5177, 0.8466, 0.4867), r = 0.4090

---

Quick comments I have, which we should include in this PR, after trying the functionality out are these (note that a thorough review is underway):

1. Rescaling of experimental patterns to float32 should not be inplace, but of deep copies of them just before they are refined.
2. I think we should always print the refinement information, like method and desirable parameters, to stdout (in general just your standard print), so that the user, even though they don't care about those, still get the information.

Apart from this I have several other suggestions for improvements, but I think that should be left for later PRs when we start to use the refinement and get a feel for the workflow and what things to store where etc. I suggest we leave updates of the pattern matching notebook to a later PR.

[friedkitteh]

1. Rescaling of experimental patterns to float32 should not be inplace, but of deep copies of them just before they are refined.

Yep, I 100% agree with you on this.

2. I think we should always print the refinement information, like method and desirable parameters, to stdout (in general just your standard print), so that the user, even though they don't care about those, still get the information.

Do you mean something like:

Refining 10 000 orientations, using Nelder-Mead with fatol=0.0001, xatol=0.0001, ..., :
[########################################] | 100% Completed

[hakonanes]

Do you mean something like:

Refining 10 000 orientations, using Nelder-Mead with fatol=0.0001, xatol=0.0001, ..., :
[########################################] | 100% Completed

I was thinking more in the line of what HyperSpy does for their BaseSignal.decomposition() as shown in our multivariate analysis notebook from cell 7.

[hakonanes]

I'm really excited to try this out some more, great work!

There is a lot here, so this review is just the first of a few, I believe.

In general I agree with the structure and setup, and the requested changes relate more to specific details.

[hakonanes]

There's lots of work in this PR, and the functionality is a huge boost to kikuchipy. Thanks again, @friedkitteh!

I'll create an issue to keep track of suggested changes and minor improvements to refinement which we can do bit by bit in other PRs until a v0.5 release sometime in September.