Ensure correct endian in numba_histogram

[jlaehne]

Description of the change

Makes sure that numba_histogram runs for big endian datatypes like >u2 or >f4 by converting data to little endian for execution.

Closes #2668

Progress of the PR

• Change implemented,
• extend to other numba functions,
• add tests,
• ready for review.

Minimal example of the bug fix or the new feature

import numpy as np
from hyperspy.misc.array_tools import numba_histogram

# works
arr = np.arange(100, dtype='<u2')
numba_histogram(arr, 5, (0, 100))

# works
arr = np.arange(100, dtype='u2')
numba_histogram(arr, 5, (0, 100))

# works
arr = np.arange(100, dtype='>u2')
numba_histogram(arr, 5, (0, 100))

[jlaehne]

Question is, whether this affects any other functions that run numba? Namely

• _linear_bin_loop in array_tools.py
• lowess in lowess_smooth.py
• _fast_mean and _fast_std in peakfinders2D.py

Probably, the would need a similar wrapper.

[francisco-dlp]

What about the following instead?

    if data.dtype.byteorder == '>':
       data = data.byteswap().newbyteorder()
    return _numba_histogram(data, bins, ranges)

It is the same, but easier to maintain...

[jlaehne]

Sure, no problem.

Should I add it for the other 4 numba-functions? I think we cannot rule out that someone calls them with the wrong endian. For the private functions I could just add it to the calling functions, only lowess would need a wrapper.

[ericpre]

The issue is that numba only support native endianess and this approach will not work on all platform. What's about:

        # Numba only supported native dtype
        # https://github.com/numba/numba/issues/2243
        if not data.dtype.isnative:
            data = data.astype(data.dtype.type

[jlaehne]

Yes, that should do the job, indeed.

[codecov]

Codecov Report

Merging #2678 (9452c96) into RELEASE_next_patch (c7a662c) will increase coverage by 0.04%.
The diff coverage is 100.00%.

@@                  Coverage Diff                   @@
##           RELEASE_next_patch    #2678      +/-   ##
======================================================
+ Coverage               76.62%   76.66%   +0.04%     
======================================================
  Files                     201      201              
  Lines                   29655    29666      +11     
  Branches                 6491     6495       +4     
======================================================
+ Hits                    22722    22743      +21     
+ Misses                   5174     5168       -6     
+ Partials                 1759     1755       -4     

Impacted Files	Coverage Δ
hyperspy/misc/array_tools.py	83.46% <100.00%> (+7.43%)	⬆️
hyperspy/misc/lowess_smooth.py	100.00% <100.00%> (ø)
hyperspy/signal.py	75.87% <0.00%> (+0.10%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7a662c...9452c96. Read the comment docs.

[ericpre]

Nice to take the opportunity to increase the coverage, thanks!

[jlaehne]

Nice to take the opportunity to increase the coverage, thanks!

Not much, but at least a few obvious ones.

In the end, I refrained from covering _fast_mean and _fast_std in peakfinders2D.py, because I could not trigger the error using the 'stat' peakfinder routine.