doc/user_guide/pattern_processing.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "nbsphinx": "hidden"
   },
   "source": [
    "This notebook is part of the `kikuchipy` documentation https://kikuchipy.org.\n",
    "Links to the documentation won't work from the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pattern processing\n",
    "\n",
    "The raw EBSD signal can be empirically evaluated as a superposition of a Kikuchi\n",
    "diffraction pattern and a smooth background intensity. For pattern indexing, the\n",
    "latter intensity is usually undesirable, while for\n",
    "[virtual backscatter electron VBSE) imaging](virtual_backscatter_electron_imaging.rst),\n",
    "this intensity can reveal topographical, compositional or diffraction contrast.\n",
    "This section details methods to enhance the Kikuchi diffraction pattern and\n",
    "manipulate detector intensities in patterns in an\n",
    "[EBSD](../reference.rst#kikuchipy.signals.EBSD) signal.\n",
    "\n",
    "Most of the methods operating on EBSD objects use functions that operate on the\n",
    "individual patterns (`numpy.ndarray`). These single pattern functions are\n",
    "available in the [kikuchipy.pattern](../reference.rst#pattern) module.\n",
    "\n",
    "Let's import the necessary libraries and read the Nickel EBSD test data set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exchange inline for notebook or qt5 (from pyqt) for interactive plotting\n",
    "%matplotlib inline\n",
    "\n",
    "import hyperspy.api as hs\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import kikuchipy as kp\n",
    "\n",
    "\n",
    "s = kp.data.nickel_ebsd_small()  # Use kp.load(\"data.h5\") to load your own data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most methods operate inplace (indicated in their docstrings), meaning they\n",
    "overwrite the patterns in the EBSD signal. If we instead want to keep the\n",
    "original signal and operate on a new signal, we can create a\n",
    "[deepcopy()](http://hyperspy.org/hyperspy-doc/current/api/hyperspy.signal.html#hyperspy.signal.BaseSignal.deepcopy)\n",
    "of the original signal. As an example here, we create a new EBSD signal from a\n",
    "small part of the original signal:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s2 = s.deepcopy()\n",
    "np.may_share_memory(s.data, s2.data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Background correction\n",
    "\n",
    "### Remove the static background\n",
    "\n",
    "Effects which are constant, like hot pixels or dirt on the detector, can be\n",
    "removed by either subtracting or dividing by a static background via\n",
    "[remove_static_background()](../reference.rst#kikuchipy.signals.EBSD.remove_static_background):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s2.remove_static_background(operation=\"subtract\", relative=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "nbsphinx-thumbnail": {
     "tooltip": "Enhancement of Kikuchi bands in EBSD patterns"
    },
    "tags": [
     "nbsphinx-thumbnail"
    ]
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(13, 6), ncols=2)\n",
    "ax[0].imshow(s.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"As acquired\")\n",
    "ax[0].axis(\"off\")\n",
    "ax[1].imshow(s2.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[1].set_title(\"Static background removed\")\n",
    "ax[1].axis(\"off\")\n",
    "fig.tight_layout(w_pad=0-8)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, the static background pattern is assumed to be stored as part of the\n",
    "signal `metadata`, which can be loaded via\n",
    "[set_experimental_parameters()](../reference.rst#kikuchipy.signals.EBSD.set_experimental_parameters).\n",
    "The static background pattern can also be passed to the `static_bg` parameter.\n",
    "Passing `relative=True` (default) ensures that relative intensities between\n",
    "patterns are kept when they are rescaled after correction to fill the available\n",
    "data range. In this case, for a scan of data type `uint8` with data range\n",
    "[0, 255], the highest pixel intensity in a scan is stretched to 255 (and the\n",
    "lowest to 0), while the rest is rescaled keeping relative intensities between\n",
    "patterns. With `relative=False`, all patterns are stretched to [0, 255].\n",
    "\n",
    "<div class=\"alert alert-warning\">\n",
    "\n",
    "Warning\n",
    "\n",
    "The `Acquisition_instrument.SEM.Detector.EBSD` and `Sample.Phases` metadata\n",
    "nodes are deprecated and will be removed in v0.6.\n",
    "\n",
    "There are three main reasons for this change: the first is that only the static\n",
    "background array stored in the\n",
    "`Acquisition_instrument.SEM.Detector.EBSD.static_background` node is used\n",
    "internally, and so the remaining metadata is unnecessary. The background pattern\n",
    "will be stored in its own `EBSD.static_background` property instead. The second\n",
    "is that keeping track of the unnecessary metadata makes writing and maintaining\n",
    "input/ouput plugins challenging. The third is that the\n",
    "[EBSD.xmap](../reference.rst#kikuchipy.signals.EBSD.xmap) and\n",
    "[EBSD.detector](../reference.rst#kikuchipy.signals.EBSD.detector) properties,\n",
    "which keeps track of the\n",
    "[CrystalMap](https://orix.readthedocs.io/en/stable/reference.html#orix.crystal_map.CrystalMap)\n",
    "and [EBSDDetector](../reference.rst#kikuchipy.detectors.EBSDDetector) for a\n",
    "signal, respectively, should be used instead of the more \"static\" metadata.\n",
    "\n",
    "</div>\n",
    "\n",
    "The static background pattern intensities can be rescaled to each individual\n",
    "pattern's intensity range before removal by passing `scale_bg=True`, which\n",
    "will result in the relative intensity between patterns to be lost (passing\n",
    "`relative=True` along with `scale_bg=True` is not allowed)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Remove the dynamic background\n",
    "\n",
    "Uneven intensity in a static background subtracted pattern can be corrected by\n",
    "subtracting or dividing by a dynamic background obtained by Gaussian blurring.\n",
    "This so-called flat fielding is done with\n",
    "[remove_dynamic_background()](../reference.rst#kikuchipy.signals.EBSD.remove_dynamic_background).\n",
    "A Gaussian window with a standard deviation set by `std` is used to blur each\n",
    "pattern individually (dynamic) either in the spatial or frequency domain, set by\n",
    "`filter_domain`. Blurring in the frequency domain is effectively accomplished\n",
    "by a low-pass\n",
    "[Fast Fourier Transform (FFT) filter](#Filtering-in-the-frequency-domain). The\n",
    "individual Gaussian blurred dynamic backgrounds are then subtracted or divided\n",
    "from the respective patterns, set by `operation`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s3 = s2.deepcopy()\n",
    "s3.remove_dynamic_background(\n",
    "    operation=\"subtract\",  # Default\n",
    "    filter_domain=\"frequency\",  # Default\n",
    "    std=8,  # Default is 1/8 of the pattern width\n",
    "    truncate=4,  # Default\n",
    ")\n",
    "\n",
    "# _ means we don't want the output\n",
    "_, ax = plt.subplots(figsize=(13, 6), ncols=2)\n",
    "ax[0].imshow(s2.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"Static background removed\")\n",
    "ax[1].imshow(s3.inav[0, 0].data, cmap=\"gray\")\n",
    "_ = ax[1].set_title(\"Static + dynamic background removed\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The width of the Gaussian window is truncated at the `truncated` number of\n",
    "standard deviations. Output patterns are rescaled to fill the input patterns'\n",
    "data type range."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Get the dynamic background\n",
    "\n",
    "The Gaussian blurred pattern removed during dynamic background correction can\n",
    "be obtained as an EBSD signal by calling\n",
    "[get_dynamic_background()](../reference.rst#kikuchipy.signals.EBSD.get_dynamic_background):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bg = s.get_dynamic_background(filter_domain=\"frequency\", std=8, truncate=4)\n",
    "\n",
    "_, ax = plt.subplots(figsize=(13, 6), ncols=2)\n",
    "ax[0].imshow(s.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"As acquired\")\n",
    "ax[1].imshow(bg.inav[0, 0].data, cmap=\"gray\")\n",
    "_ = ax[1].set_title(\"Dynamic background\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Average neighbour patterns\n",
    "\n",
    "The signal-to-noise ratio in patterns in an EBSD signal `s` can be improved by\n",
    "averaging patterns with their closest neighbours within a window/kernel/mask\n",
    "with\n",
    "[average_neighbour_patterns()](../reference.rst#kikuchipy.signals.EBSD.average_neighbour_patterns):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s4 = s3.deepcopy()\n",
    "s4.average_neighbour_patterns(window=\"gaussian\", std=1)\n",
    "\n",
    "_, ax = plt.subplots(figsize=(13, 6), ncols=2)\n",
    "ax[0].imshow(s3.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"Static + dynamic background removed\")\n",
    "ax[1].imshow(s4.inav[0, 0].data, cmap=\"gray\")\n",
    "_ = ax[1].set_title(\"After neighbour pattern averaging\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The array of averaged patterns $g(n_{\\mathrm{x}}, n_{\\mathrm{y}})$ is obtained\n",
    "by spatially correlating a window $w(s, t)$ with the array of patterns\n",
    "$f(n_{\\mathrm{x}}, n_{\\mathrm{y}})$, here 4D, which is padded with zeros at the\n",
    "edges. As coordinates $n_{\\mathrm{x}}$ and $n_{\\mathrm{y}}$ are varied, the\n",
    "window origin moves from pattern to pattern, computing the sum of products of\n",
    "the window coefficients with the neighbour pattern intensities, defined by the\n",
    "window shape, followed by normalizing by the sum of the window coefficients. For\n",
    "a symmetrical window of shape $m \\times n$, this becomes\n",
    "<cite data-cite=\"gonzalez2017digital\">Gonzalez and Woods (2017)</cite>\n",
    "\n",
    "\\begin{equation}\n",
    "g(n_{\\mathrm{x}}, n_{\\mathrm{y}}) =\n",
    "\\frac{\\sum_{s=-a}^a\\sum_{t=-b}^b{w(s, t)\n",
    "f(n_{\\mathrm{x}} + s, n_{\\mathrm{y}} + t)}}\n",
    "{\\sum_{s=-a}^a\\sum_{t=-b}^b{w(s, t)}},\n",
    "\\end{equation}\n",
    "\n",
    "where $a = (m - 1)/2$ and $b = (n - 1)/2$. The window $w$, a\n",
    "[Window](../reference.rst#kikuchipy.filters.Window) object, can be plotted:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = kp.filters.Window(window=\"gaussian\", shape=(3, 3), std=1)\n",
    "w.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Any 1D or 2D window with desired coefficients can be used. This custom window\n",
    "can be passed to the `window` parameter in\n",
    "[average_neighbour_patterns()](../reference.rst#kikuchipy.signals.EBSD.average_neighbour_patterns)\n",
    "or [Window](../reference.rst#kikuchipy.filters.Window) as a `numpy.ndarray`\n",
    "or a `dask.array.Array`. Additionally, any window in\n",
    "[scipy.signal.windows.get_window()](https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.get_window.html)\n",
    "passed as a string via `window` with the necessary parameters as keyword\n",
    "arguments (like `std=1` for `window=\"gaussian\"`) can be used. To demonstrate the\n",
    "creation and use of an asymmetrical circular window (and the use of\n",
    "[make_circular()](../reference.rst#kikuchipy.filters.Window.make_circular),\n",
    "although we could create a circular window directly by calling\n",
    "`window=\"circular\"` upon window initialization):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w = kp.filters.Window(window=\"rectangular\", shape=(5, 4))\n",
    "w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w.make_circular()\n",
    "w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s5 = s3.deepcopy()\n",
    "s5.average_neighbour_patterns(w)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "w.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But this `(5, 4)` averaging window cannot be used with our `(3, 3)` navigation\n",
    "shape signal."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\">\n",
    "\n",
    "Note\n",
    "   \n",
    "Neighbour pattern averaging increases the virtual interaction volume of the\n",
    "electron beam with the sample, leading to a potential loss in spatial\n",
    "resolution. Averaging may in some cases, like on grain boundaries, mix two\n",
    "or more different diffraction patterns, which might be unwanted. See\n",
    "<cite data-cite=\"wright2015introduction\">Wright et al. (2015)</cite> for a\n",
    "discussion of this concern.\n",
    "\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Adaptive histogram equalization\n",
    "\n",
    "Enhancing the pattern contrast with adaptive histogram equalization has been\n",
    "found useful when comparing patterns for dictionary indexing\n",
    "<cite data-cite=\"marquardt2017quantitative\">Marquardt et al. (2017)</cite>.\n",
    "With [adaptive_histogram_equalization()](../reference.rst#kikuchipy.signals.EBSD.adaptive_histogram_equalization), the\n",
    "intensities in the pattern histogram are spread to cover the available range,\n",
    "e.g. [0, 255] for patterns of `uint8` data type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s6 = s3.deepcopy()\n",
    "s6.adaptive_histogram_equalization(kernel_size=(15, 15))\n",
    "\n",
    "_, ax = plt.subplots(figsize=(13, 6), ncols=2)\n",
    "ax[0].imshow(s3.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"Static + dynamic background removed\")\n",
    "ax[1].imshow(s6.inav[0, 0].data, cmap=\"gray\")\n",
    "_ = ax[1].set_title(\"After adaptive histogram equalization\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `kernel_size` parameter determines the size of the contextual regions. See\n",
    "e.g. Fig. 5 in\n",
    "<cite data-cite=\"jackson2019dictionary\">Jackson et al. (2019)</cite>, also\n",
    "available via\n",
    "[EMsoft's GitHub repository wiki](https://github.com/EMsoft-org/EMsoft/wiki/DItutorial#52-determination-of-pattern-pre-processing-parameters),\n",
    "for the effect of varying `kernel_size`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Filtering in the frequency domain\n",
    "\n",
    "Filtering of patterns in the frequency domain can be done with\n",
    "[fft_filter()](../reference.rst#kikuchipy.signals.EBSD.fft_filter). This method\n",
    "takes a spatial kernel defined in the spatial domain, or a transfer function\n",
    "defined in the frequency domain, in the `transfer_function` argument as a\n",
    "`numpy.ndarray` or a [Window](../reference.rst#kikuchipy.filters.Window).\n",
    "Which domain the transfer function is defined in must be passed to the\n",
    "`function_domain` argument. Whether to shift zero-frequency components to the\n",
    "centre of the FFT can also be controlled via `shift`, but note that this is\n",
    "only used when `function_domain=\"frequency\"`.\n",
    "\n",
    "Popular uses of filtering of EBSD patterns in the frequency domain include\n",
    "removing large scale variations across the detector with a Gaussian high pass\n",
    "filter, or removing high frequency noise with a Gaussian low pass filter. These\n",
    "particular functions are readily available via `Window`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pattern_shape = s.axes_manager.signal_shape[::-1]\n",
    "w_low = kp.filters.Window(\n",
    "    window=\"lowpass\",\n",
    "    cutoff=23,\n",
    "    cutoff_width=10,\n",
    "    shape=pattern_shape\n",
    ")\n",
    "w_high = kp.filters.Window(\n",
    "    window=\"highpass\",\n",
    "    cutoff=3,\n",
    "    cutoff_width=2,\n",
    "    shape=pattern_shape\n",
    ")\n",
    "\n",
    "fig, ax = plt.subplots(figsize=(22, 6), ncols=3)\n",
    "fig.subplots_adjust(wspace=0.05)\n",
    "im0 = ax[0].imshow(w_low, cmap=\"gray\")\n",
    "ax[0].set_title(\"Lowpass filter\", fontsize=22)\n",
    "fig.colorbar(im0, ax=ax[0])\n",
    "im1 = ax[1].imshow(w_high, cmap=\"gray\")\n",
    "ax[1].set_title(\"Highpass filter\", fontsize=22)\n",
    "fig.colorbar(im1, ax=ax[1])\n",
    "im2 = ax[2].imshow(w_low * w_high, cmap=\"gray\")\n",
    "ax[2].set_title(\"Lowpass * highpass filter\", fontsize=22)\n",
    "_ = fig.colorbar(im2, ax=ax[2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then, to multiply the FFT of each pattern with this transfer function, and\n",
    "subsequently computing the inverse FFT (IFFT), we use\n",
    "`fft_filter()`, and remember to shift the zero-frequency components to the\n",
    "centre of the FFT:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s7 = s3.deepcopy()\n",
    "s7.fft_filter(\n",
    "    transfer_function=w_low * w_high,\n",
    "    function_domain=\"frequency\",\n",
    "    shift=True\n",
    ")\n",
    "\n",
    "_, ax = plt.subplots(figsize=(13, 6), ncols=2)\n",
    "ax[0].imshow(s3.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"Static + dynamic background removed\")\n",
    "ax[1].imshow(s7.inav[0, 0].data, cmap=\"gray\")\n",
    "_ = ax[1].set_title(\"After FFT filtering\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that filtering with a spatial kernel in the frequency domain, after\n",
    "creating the kernel's transfer function via FFT, and computing the inverse FFT\n",
    "(IFFT), is, in this case, the same as spatially correlating the kernel with the\n",
    "pattern.\n",
    "\n",
    "Let's demonstrate this by attempting to sharpen a pattern with a Laplacian\n",
    "kernel in both the spatial and frequency domains and comparing the results\n",
    "(this is a purely illustrative example, and perhaps not that practically\n",
    "useful):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s8 = s3.deepcopy()\n",
    "w_laplacian = np.array([[-1, -1, -1], [-1, 8, -1], [-1, -1, -1]])\n",
    "s8.fft_filter(transfer_function=w_laplacian, function_domain=\"spatial\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from scipy.ndimage import correlate\n",
    "\n",
    "\n",
    "p_filt = correlate(s3.inav[0, 0].data.astype(np.float32), weights=w_laplacian)\n",
    "p_filt_resc = kp.pattern.rescale_intensity(p_filt, dtype_out=np.uint8)\n",
    "\n",
    "_, ax = plt.subplots(figsize=(13, 6), ncols=2)\n",
    "ax[0].imshow(s8.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"Filter in frequency domain\")\n",
    "ax[1].imshow(p_filt_resc, cmap=\"gray\")\n",
    "ax[1].set_title(\"Filter in spatial domain\")\n",
    "\n",
    "np.sum(s8.inav[0, 0].data - p_filt_resc)  # Which is zero"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note also that `fft_filter()` performs the filtering on the patterns with data\n",
    "type `np.float32`, and therefore have to rescale back to the pattern's original\n",
    "data type if necessary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Rescale intensity\n",
    "\n",
    "Vendors usually write patterns to file with 8 (``uint8``) or 16 (``uint16``) bit\n",
    "integer depth, holding [0, 2$^8$] or [0, 2$^{16}$] gray levels, respectively. To\n",
    "avoid losing intensity information when processing, we often change data types\n",
    "to e.g. 32 bit floating point (``float32``). However, only changing the data\n",
    "type with [change_dtype()](http://hyperspy.org/hyperspy-doc/current/api/hyperspy.signal.html#hyperspy.signal.BaseSignal.change_dtype) does not\n",
    "rescale pattern intensities, leading to patterns not using the full available\n",
    "data type range:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s9 = s3.deepcopy()\n",
    "\n",
    "print(s9.data.dtype, s9.data.max())\n",
    "\n",
    "s9.change_dtype(np.uint16)\n",
    "\n",
    "print(s9.data.dtype, s9.data.max())\n",
    "\n",
    "plt.figure(figsize=(6, 5))\n",
    "plt.imshow(s9.inav[0, 0].data, vmax=1000, cmap=\"gray\")\n",
    "plt.title(\"16-bit pixels w/ 255 as max intensity\", pad=15)\n",
    "_ = plt.colorbar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In these cases it is convenient to rescale intensities to a desired data type\n",
    "range, either keeping relative intensities between patterns in a scan or not. We\n",
    "can do this for all patterns in an EBSD signal with\n",
    "[kikuchipy.signals.EBSD.rescale_intensity()](../reference.rst#kikuchipy.signals.EBSD.rescale_intensity):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s9.rescale_intensity(relative=True)\n",
    "\n",
    "print(s9.data.dtype, s9.data.max())\n",
    "\n",
    "plt.figure(figsize=(6, 5))\n",
    "plt.imshow(s9.inav[0, 0].data, cmap=\"gray\")\n",
    "plt.title(\"16-bit pixels w/ 65535 as max. intensity\", pad=15)\n",
    "_ = plt.colorbar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Or, we can do it for a single pattern (`numpy.ndarray`) with\n",
    "[kikuchipy.pattern.rescale_intensity()](../reference.rst#kikuchipy.pattern.rescale_intensity):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = s3.inav[0, 0].data\n",
    "print(p.min(), p.max())\n",
    "p2 = kp.pattern.rescale_intensity(p, dtype_out=np.uint16)\n",
    "print(p2.min(), p2.max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also stretch the pattern contrast by removing intensities outside a range\n",
    "passed to `in_range` or at certain percentiles by passing percentages to\n",
    "`percentiles`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s10 = s3.deepcopy()\n",
    "print(s10.data.min(), s10.data.max())\n",
    "s10.rescale_intensity(out_range=(10, 245))\n",
    "print(s10.data.min(), s10.data.max())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s10.rescale_intensity(percentiles=(0.5, 99.5))\n",
    "print(s10.data.min(), s3.data.max())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(13, 5), ncols=2)\n",
    "im0 = ax[0].imshow(s3.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[0].set_title(\"Static + dynamic background removed\", pad=15)\n",
    "fig.colorbar(im0, ax=ax[0])\n",
    "im1 = ax[1].imshow(s10.inav[0, 0].data, cmap=\"gray\")\n",
    "ax[1].set_title(\"After clipping lowest/highest intensities\", pad=15)\n",
    "_ = fig.colorbar(im1, ax=ax[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This can reduce the influence of outliers with exceptionally high or low\n",
    "intensities, like hot or dead pixels."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## Normalize intensity\n",
    "\n",
    "It can be useful to normalize pattern intensities to a mean value of $\\mu = 0.0$\n",
    "and a standard deviation of e.g. $\\sigma = 1.0$ when e.g. comparing patterns or\n",
    "calculating the [image quality](feature_maps.ipynb#image-quality). Patterns can be\n",
    "normalized with\n",
    "[normalize_intensity()](../reference.rst#kikuchipy.signals.EBSD.normalize_intensity):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s11 = s3.deepcopy()\n",
    "np.mean(s11.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# s11.change_dtype(np.float32)  # Or pass `dtype_out` as below\n",
    "s11.normalize_intensity(num_std=1, dtype_out=np.float32)\n",
    "np.mean(s11.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "_, ax = plt.subplots(figsize=(13, 4), ncols=2)\n",
    "ax[0].hist(s3.data.ravel(), bins=255)\n",
    "ax[0].set_title(\"Static + dynamic background removed\")\n",
    "ax[1].hist(s11.data.ravel(), bins=255)\n",
    "_ = ax[1].set_title(\"After intensity normalization\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}