Architecture Detection

[bkmgit]

Currently, x86 architectures are classified as Haswell, Westmere or Fallback. If Haswell, availability of AVX2 instructions is assumed. This is not always correct, for example on Haswell based Celeron processors https://en.wikipedia.org/wiki/List_of_Intel_Celeron_processors#Haswell_based_Celerons_2. Consider querying the available CPU instructions that are available and then choosing the appropriate SIMD instruction set to use. A portable C++ library that can help with this is Pytorch cpuinfo, though it has many build dependencies. A similar python library is py-cpuinfo. Similar checks can be applied here, for example checking /proc/cpuinfo on Linux.

[lemire]

Consider querying the available CPU instructions that are available and then choosing the appropriate SIMD instruction set to use.

What makes you think that we do not do this already?

I am closing this issue.

If the reporter thinks that there is an issue with the library, I invite them to thoroughly document it, starting with an analysis of our source code.

[bkmgit]

Thanks for the code and for responding to issues. It takes some time to fully understand source of the error. Ran the regression tests on Celeron processor, and they fail since all architectures are built for x86 and then unless an architecture is specified, all available architectures that have been built are run by https://github.com/simdutf/simdutf/blob/master/tests/helpers/test.cpp

The header only example https://github.com/simdutf/simdutf/blob/master/singleheader/amalgamation_demo.cpp works as it does automatic detection of the architecture and uses the best available one, as would occur in normal use. Can contribute a modification to the testing harness to only run supported instruction sets.

[bkmgit]

To get failing tests, on a Celeron processor without AVX2, try:

git clone https://github.com/simdutf/simdutf
cd simdutf
mkdir build
cd build
cmake ..
make
ctest .

Please let me know if further documentation would be helpful.

When doing automated testing of libraries, the architecture may not be known in advance, so it would be helpful if the testing harness can check this, though if that further complicates maintenance, it is also possible to build a further external test harness that will check available architectures.

[lemire]

The code does not actually test whether the processor is haswell, it tests whether it supports the needed features.
It has been extensively tested across a wide range of architectures. We do not invite pull requests if you find a bug in how we handle processor feature detection.