Make EBSD refinement less restrictive regarding CrystalMap

[hakonanes]

Hello Hakon,

I could index my patterns with no issues, however an error is raised once I try to perform the refinement:

ValueError: The crystal map shape (28028,) and step sizes [0, 0] aren't compatible with the signal navigation shape (28028,) and step sizes (1.0,) (see EBSD.axes_manager)

This is how my patterns axes_maneger looks like:

s.axes_manager

<Axes manager, axes: (28028|155, 128)>
            Name |   size |  index |  offset |   scale |  units 
================ | ====== | ====== | ======= | ======= | ====== 
     <undefined> |  28028 |      0 |       0 |       1 | <undefined> 
---------------- | ------ | ------ | ------- | ------- | ------ 
     <undefined> |    155 |        |       0 |       1 | <undefined> 
     <undefined> |    128 |        |       0 |       1 | <undefined>

Originally posted by @tmostormujof in #525 (comment)

@tmostormujof, thank you for raising the issue. The error message is raised here:

kikuchipy/kikuchipy/signals/util/_crystal_map.py

Lines 47 to 55 in 87816c8

	compatible = xmap.shape == nav_shape and np.allclose(
	xmap_scale, nav_scale, atol=1e-6
	)
	if not compatible and raise_if_not:
	raise ValueError(
	f"The crystal map shape {xmap.shape} and step sizes {xmap_scale} aren't "
	f"compatible with the signal navigation shape {nav_shape} and step sizes "
	f"{nav_scale} (see `EBSD.axes_manager`)"
	)

when either (1) the EBSD navigation shape and CrystalMap shape aren't equivalent or (2) their step sizes aren't the same.

Regarding a (short term) solution for you: I'm a bit surprised that the CrystalMap step sizes are read as [0, 0], since I would think that the crystal map only had one navigation dimension (28028,) with a step size of 1 by default. What is your crystal map shape? It is not advisable to access or set private properties, but in this case I think it is the simplest solution. Could you try to update the spatial coordinate array of the crystal map xmap returned dictionary indexing by:

>>> from orix.crystal_map import create_coordinate_arrays
>>> xmap._x = create_coordinate_arrays(shape=(28028,), step_sizes=(1,))
# Continue with refinement

Regarding a long term solution: I believe this error is fairly common, and refinement does not use spatial correlation between patterns, so the step size is not critical. I suggest to drop the requirement that the step sizes are equivalent.

[tmostormujof]

Hello Hakon,

I apologize for so many questions. This short term solution raises another issue. The original xmap._x has the folloxing shape:

xmap._x
array([    0,     1,     2, ..., 28025, 28026, 28027])

When I use the create_coordinate_arrays function, it changes and it's formed by a dict and an int, see below:

from orix.crystal_map import create_coordinate_arrays
xmap._x = create_coordinate_arrays(shape=(28028,),step_sizes=(1,))
xmap._x
({'x': array([    0,     1,     2, ..., 28025, 28026, 28027])}, 28028)

What raises the error:

TypeError: '<' not supported between instances of 'int' and 'dict'

Coming from:

  File "C:\Users\Tomas\anaconda3\envs\PythonCPU\lib\site-packages\orix\crystal_map\crystal_map.py", line 898, in _step_size_from_coordinates
    unique_sorted = np.sort(np.unique(coordinates))

[hakonanes]

No need to apologize, it's I who gave you the incorrect code, it should be

>>> from orix.crystal_map import create_coordinate_arrays
>>> coordinates, step_sizes = create_coordinate_arrays(shape=(28028,), step_sizes=(1,))
>>> xmap._x = coordinates["x"]
# Continue with refinement

where we only pass the "x" array in the returned dictionary (and ignore the integer(s)), as you say (the documentation of the used function is here).

The original xmap._x has the folloxing shape

It seems like the original coordinate array is correct, so I am at a loss here. Is it possible for you to send me the crystal map as an HDF5 file, which can be saved as

>>> from orix import io
>>> io.save("crystal_map.h5", xmap)

[hakonanes]

I experience a similar issue myself when trying to refine an EBSD signal with one pattern:

>>> xmap
Phase  Orientations  Name  Space group  Point group  Proper point group     Color
    0    1 (100.0%)    al        Fm-3m         m-3m                 432  tab:blue
Properties: 
Scan unit: px
>>> xmap.shape
()
>>> s
<EBSD, title: , dimensions: (1|240, 240)>
>>> xmap_ref = s.refine_orientation(xmap=xmap, <other parameters>)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [113], in <cell line: 5>()
      3 # First refine orientations, then projection centers
      4 ref_kwargs = dict(detector=detector, master_pattern=mp_al, energy=energy)
----> 5 xmap_al_refined = s_cal_al.refine_orientation(xmap=xmap_al, **ref_kwargs)
      6 _, detector_al_ref = s_cal_al.refine_projection_center(xmap=xmap_al_refined, **ref_kwargs)

File ~/kode/kikuchipy/kikuchipy/signals/ebsd.py:1205, in EBSD.refine_orientation(self, xmap, detector, master_pattern, energy, mask, method, method_kwargs, trust_region, compute, rechunk, chunk_kwargs)
   1095 def refine_orientation(
   1096     self,
   1097     xmap: CrystalMap,
   (...)
   1107     chunk_kwargs: Optional[dict] = None,
   1108 ):
   1109     r"""Refine orientations by searching orientation space around
   1110     the best indexed solution using fixed projection centers.
   1111 
   (...)
   1203     refine_orientation_projection_center
   1204     """
-> 1205     self._check_refinement_parameters(xmap=xmap, detector=detector, mask=mask)
   1206     patterns = self._get_dask_array_for_refinement(
   1207         rechunk=rechunk, chunk_kwargs=chunk_kwargs
   1208     )
   1209     return _refine_orientation(
   1210         xmap=xmap,
   1211         detector=detector,
   (...)
   1220         compute=compute,
   1221     )

File ~/kode/kikuchipy/kikuchipy/signals/ebsd.py:2142, in EBSD._check_refinement_parameters(self, xmap, detector, mask)
   2135 def _check_refinement_parameters(
   2136     self,
   2137     xmap: CrystalMap,
   2138     detector: EBSDDetector,
   2139     mask: Union[np.ndarray, None],
   2140 ):
   2141     """Raise ValueError if EBSD refinement input is invalid."""
-> 2142     _crystal_map_is_compatible_with_signal(
   2143         xmap=xmap,
   2144         navigation_axes=self.axes_manager.navigation_axes[::-1],
   2145         raise_if_not=True,
   2146     )
   2147     sig_shape = self.axes_manager.signal_shape[::-1]
   2148     _detector_is_compatible_with_signal(
   2149         detector=detector,
   2150         navigation_shape=self.axes_manager.navigation_shape[::-1],
   2151         signal_shape=sig_shape,
   2152         raise_if_not=True,
   2153     )

File ~/kode/kikuchipy/kikuchipy/signals/util/_crystal_map.py:51, in _crystal_map_is_compatible_with_signal(xmap, navigation_axes, raise_if_not)
     47 compatible = xmap.shape == nav_shape and np.allclose(
     48     xmap_scale, nav_scale, atol=1e-6
     49 )
     50 if not compatible and raise_if_not:
---> 51     raise ValueError(
     52         f"The crystal map shape {xmap.shape} and step sizes {xmap_scale} aren't "
     53         f"compatible with the signal navigation shape {nav_shape} and step sizes "
     54         f"{nav_scale} (see `EBSD.axes_manager`)"
     55     )
     56 else:
     57     return compatible

ValueError: The crystal map shape () and step sizes (0,) aren't compatible with the signal navigation shape (1,) and step sizes (1.0,) (see `EBSD.axes_manager`)

[hakonanes]

My issue was fixed by removing the single navigation dimension of the EBSD signal with

>>> s2 = s.squeeze()
>>> s2
<EBSD, title: , dimensions: (|240, 240)>

and doing refinement on s2. That won't have any effect on your case, though... Will investigate further.

[hakonanes]

Terribly sorry, @tmostormujof, but I think I know why you see that error message...

I believe you see this warning before the error message is displayed:

/home/hakon/kode/kikuchipy/kikuchipy/signals/util/_crystal_map.py:40: UserWarning: The signal navigation axes must be named 'x' and/or 'y' in order to compare the signal navigation scale to the CrystalMap step sizes 'dx' and 'dy' (see `EBSD.axes_manager`)
  warnings.warn(

So, if you just name the one navigation dimension in your EBSD signal axes manager to "x"

>>> s.axes_manager
<Axes manager, axes: (2|240, 240)>
            Name |   size |  index |  offset |   scale |  units 
================ | ====== | ====== | ======= | ======= | ====== 
                 |      2 |      1 |       0 |       1 | <undefined> 
---------------- | ------ | ------ | ------- | ------- | ------ 
     <undefined> |    240 |      0 |       0 |       1 | <undefined> 
     <undefined> |    240 |      0 |       0 |       1 | <undefined> 

>>> s.axes_manager[0].name = "x"
>>> s.axes_manager
<Axes manager, axes: (2|240, 240)>
            Name |   size |  index |  offset |   scale |  units 
================ | ====== | ====== | ======= | ======= | ====== 
               x |      2 |      1 |       0 |       1 | <undefined> 
---------------- | ------ | ------ | ------- | ------- | ------ 
     <undefined> |    240 |      0 |       0 |       1 | <undefined> 
     <undefined> |    240 |      0 |       0 |       1 | <undefined> 

the refinement should run without encountering the error message.

I'll get this fixed in either a patch version 0.5.9 or in version 0.6, which I'm working toward releasing within a couple of weeks.

[hakonanes]

I'm able to reproduce the error with this minimal (yet realistic) working example:

import kikuchipy as kp
from orix import sampling


s0 = kp.data.nickel_ebsd_large(lazy=True)
s = s0.inav[:13, 0]
s.remove_static_background()
s.remove_dynamic_background()
s.compute()

mp = kp.data.nickel_ebsd_master_pattern_small(projection="lambert")
rot = sampling.get_sample_fundamental(3, point_group=mp.phase.point_group)

detector = kp.detectors.EBSDDetector(
    shape=s.axes_manager.signal_shape[::-1],
    pc=[0.421, 0.7794, 0.5049],
    sample_tilt=70,
    convention="edax",
)

# "Mess up" axes manager
s.axes_manager[0].name = ""

# Index
sim = mp.get_patterns(rotations=rot, detector=detector, energy=20)
xmap = s.dictionary_indexing(sim)

# Refine
xmap_ref = s.refine_orientation(
    xmap=xmap,
    detector=detector,
    master_pattern=mp,
    energy=20,
)

which raises this error

/home/hakon/kode/kikuchipy/kikuchipy/signals/util/_crystal_map.py:40: UserWarning: The signal navigation axes must be named 'x' and/or 'y' in order to compare the signal navigation scale to the CrystalMap step sizes 'dx' and 'dy' (see `EBSD.axes_manager`)
  warnings.warn(
Traceback (most recent call last):
  File "/home/hakon/miniconda3/envs/kp-dev/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3397, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-5-1ceae238fa6c>", line 1, in <cell line: 1>
    xmap_ref = s.refine_orientation(
  File "/home/hakon/kode/kikuchipy/kikuchipy/signals/ebsd.py", line 1205, in refine_orientation
    self._check_refinement_parameters(xmap=xmap, detector=detector, mask=mask)
  File "/home/hakon/kode/kikuchipy/kikuchipy/signals/ebsd.py", line 2142, in _check_refinement_parameters
    _crystal_map_is_compatible_with_signal(
  File "/home/hakon/kode/kikuchipy/kikuchipy/signals/util/_crystal_map.py", line 51, in _crystal_map_is_compatible_with_signal
    raise ValueError(
ValueError: The crystal map shape (13,) and step sizes [0, 0] aren't compatible with the signal navigation shape (13,) and step sizes (1.0,) (see `EBSD.axes_manager`)

[hakonanes]

@tmostormujof, you won't experience this error message in the next release v0.6 coming in a few weeks. To avoid the error message, see #526 (comment) or install the development version of kikuchipy from the develop branch.

[hakonanes]

And thank you very much for reporting it, so that we got it fixed!

[tmostormujof]

Hello Hakon,

I apologize to be unactive these couple of days, I've been quite busy. Thank you for getting that fixed, I could perform my refinements with no issues now!

[hakonanes]

No problem, I'm glad you got refinement working.

Note that the features and capabilities of the CrystalMap returned from EBSD.dictionary_indexing() and EBSD.refine_orientation() are extensively documented in the orix documentation (see e.g. the crystal map user guide).

kikuchipy is in continuous development, so your feedback is very much appreciated. Please open a new issue (or a discussion) if you encounter new issues or have a question!