Skip to content

Commit

Permalink
CI: Guard compile-time CPU features tests
Browse files Browse the repository at this point in the history
  This guard protects against any sudden unexpected changes that may adversely
  affect the compile-time SIMD features detection which could leave the SIMD code
  inactivated. Until now we have faced two cases:

  1. Hardening the compile-time test files of Neon/ASIMD features without checking
     the sanity of the modification leading to disabling all optimizations on aarch64.
     see gh-21747

  2. A sudden compiler upgrades by CI side on s390x that causes conflicts with the
     installed assembler leading to disabling the whole VX/E features, which made us
     merge SIMD code without testing it. Later, it was discovered that this code
     disrupted the NumPy build.
     see gh-21750, gh-21748
  • Loading branch information
seiko2plus committed Jun 15, 2022
1 parent 118e6c4 commit 91faeaa
Show file tree
Hide file tree
Showing 3 changed files with 42 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .github/workflows/build_test.yml
Expand Up @@ -61,6 +61,8 @@ jobs:
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11-dev"]
env:
EXPECT_CPU_FEATURES: "SSE SSE2 SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL"
steps:
- uses: actions/checkout@v3
with:
Expand Down
4 changes: 4 additions & 0 deletions .travis.yml
Expand Up @@ -34,6 +34,8 @@ jobs:
- DOWNLOAD_OPENBLAS=1
- NPY_USE_BLAS_ILP64=1
- ATLAS=None
# VSX4 still not supported by ubuntu/gcc-11
- EXPECT_CPU_FEATURES="VSX VSX2 VSX3"

- python: "3.8"
os: linux
Expand All @@ -46,6 +48,7 @@ jobs:
- DOWNLOAD_OPENBLAS=1
- NPY_USE_BLAS_ILP64=1
- ATLAS=None
- EXPECT_CPU_FEATURES="VX VXE VXE2"

# Wheel builders
- python: "3.8"
Expand All @@ -54,6 +57,7 @@ jobs:
virt: vm
env:
- CIBW_BUILD: cp38-manylinux_aarch64
- EXPECT_CPU_FEATURES: "NEON NEON_FP16 NEON_VFPV4 ASIMD ASIMDHP ASIMDDP ASIMDFHM"
install: python3 -m pip install cibuildwheel==2.4.0
script: |
cibuildwheel --output-dir wheelhouse
Expand Down
36 changes: 36 additions & 0 deletions tools/travis-test.sh
Expand Up @@ -103,6 +103,42 @@ run_test()
"import os; import numpy; print(os.path.dirname(numpy.__file__))")
export PYTHONWARNINGS=default

# This guard protects against any sudden unexpected changes that may adversely
# affect the compile-time SIMD features detection which could leave the SIMD code
# inactivated. Until now we have faced two cases:
#
# 1. Hardening the compile-time test files of Neon/ASIMD features without checking
# the sanity of the modification leading to disabling all optimizations on aarch64.
# see gh-21747
#
# 2. A sudden compiler upgrades by CI side on s390x that causes conflicts with the
# installed assembler leading to disabling the whole VX/E features, which made us
# merge SIMD code without testing it. Later, it was discovered that this code
# disrupted the NumPy build.
# see gh-21750, gh-21748
if [ -n "$EXPECT_CPU_FEATURES" ]; then
as_expected=$($PYTHON << EOF
from numpy.core._multiarray_umath import (__cpu_baseline__, __cpu_dispatch__)
features = __cpu_baseline__ + __cpu_dispatch__
expected = '$EXPECT_CPU_FEATURES'.upper().split()
diff = set(expected).difference(set(features))
if diff:
print("Unexpected compile-time CPU features detection!\n"
f"The current build is missing the support of CPU features '{diff}':\n"
"This error is triggered because of one of the following reasons:\n\n"
f"1. The compiler for somehow no longer supports any of these CPU features {diff}\n"
f"Note that the current build reports the support of the following features:\n{features}\n\n"
"2. Your code messed up the testing process! please check the build log and trace\n"
"compile-time feature tests and make sure that your patch has nothing to do with the tests failures."
)
EOF
)
if [ -n "$as_expected" ]; then
echo "$as_expected"
exit 1
fi
fi

if [ -n "$CHECK_BLAS" ]; then
$PYTHON ../tools/openblas_support.py --check_version
fi
Expand Down

0 comments on commit 91faeaa

Please sign in to comment.