Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[arm] Disable ACLE and UNALIGNED64_OK on armv7 and earlier when using cmake. #1030

Merged
merged 1 commit into from Aug 11, 2021

Conversation

mtl1979
Copy link
Collaborator

@mtl1979 mtl1979 commented Jul 21, 2021

  • ACLE was introduced in armv8
  • Support for unaligned 64-bit reads was added in armv8
  • Add status text in cmake output if unaligned 64-bit reads are enabled or disabled.

@codecov
Copy link

codecov bot commented Jul 21, 2021

Codecov Report

Merging #1030 (bd9156b) into develop (12a975a) will decrease coverage by 0.04%.
The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           develop    #1030      +/-   ##
===========================================
- Coverage    77.91%   77.86%   -0.05%     
===========================================
  Files           81       81              
  Lines         8389     8380       -9     
  Branches      1371     1364       -7     
===========================================
- Hits          6536     6525      -11     
- Misses        1320     1325       +5     
+ Partials       533      530       -3     
Flag Coverage Δ
macos_clang 68.04% <ø> (ø)
macos_gcc 66.75% <ø> (ø)
ubuntu_clang 71.83% <ø> (ø)
ubuntu_clang_debug 72.18% <ø> (ø)
ubuntu_clang_inflate_allow_invalid_dist 71.59% <ø> (ø)
ubuntu_clang_inflate_strict 71.82% <ø> (ø)
ubuntu_clang_mmap 71.81% <ø> (ø)
ubuntu_clang_msan 71.83% <ø> (ø)
ubuntu_clang_pigz 31.69% <ø> (ø)
ubuntu_clang_pigz_no_optim 37.83% <ø> (ø)
ubuntu_clang_pigz_no_threads 31.34% <ø> (ø)
ubuntu_clang_reduced_mem 72.09% <ø> (ø)
ubuntu_gcc 71.10% <ø> (ø)
ubuntu_gcc_aarch64 74.48% <ø> (ø)
ubuntu_gcc_aarch64_compat_no_opt 72.21% <ø> (ø)
ubuntu_gcc_aarch64_no_acle 73.75% <ø> (ø)
ubuntu_gcc_aarch64_no_neon 74.02% <ø> (ø)
ubuntu_gcc_armhf 74.48% <ø> (ø)
ubuntu_gcc_armhf_compat_no_opt 72.17% <ø> (ø)
ubuntu_gcc_armhf_no_acle 74.73% <ø> (ø)
ubuntu_gcc_armhf_no_neon 74.97% <ø> (ø)
ubuntu_gcc_armsf 74.49% <ø> (ø)
ubuntu_gcc_armsf_compat_no_opt 72.17% <ø> (ø)
ubuntu_gcc_compat_no_opt 73.33% <ø> (ø)
ubuntu_gcc_mingw_i686 0.00% <ø> (ø)
ubuntu_gcc_mingw_x86_64 0.00% <ø> (ø)
ubuntu_gcc_no_avx2 74.61% <ø> (ø)
ubuntu_gcc_no_pclmulqdq 69.46% <ø> (ø)
ubuntu_gcc_no_sse2 72.10% <ø> (ø)
ubuntu_gcc_no_sse4 71.44% <ø> (ø)
ubuntu_gcc_o3 71.90% <ø> (ø)
ubuntu_gcc_osb 74.09% <ø> (ø)
ubuntu_gcc_pigz 31.71% <ø> (ø)
ubuntu_gcc_pigz_aarch64 35.84% <ø> (ø)
ubuntu_gcc_ppc 73.14% <ø> (ø)
ubuntu_gcc_ppc64 75.73% <ø> (ø)
ubuntu_gcc_ppc64le 74.32% <ø> (ø)
ubuntu_gcc_s390x 73.80% <ø> (ø)
ubuntu_gcc_sparc64 76.15% <ø> (ø)
win64_gcc ∅ <ø> (∅)
win64_gcc_compat_no_opt ∅ <ø> (∅)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
zutil_p.h 71.42% <0.00%> (-28.58%) ⬇️
gzlib.c 56.38% <0.00%> (-0.36%) ⬇️
deflate_p.h 100.00% <0.00%> (ø)
trees.c 96.44% <0.00%> (+0.27%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 12a975a...bd9156b. Read the comment docs.

@hlewin
Copy link

hlewin commented Jul 21, 2021

CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 2.8.12 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


-- Android: Targeting API '21' with architecture 'arm', ABI 'armeabi-v7a', and processor 'armv7-a'
-- Android: Selected unified Clang toolchain
-- The C compiler identification is Clang 11.0.5
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Architecture: armv7-a
-- Check size of off64_t
-- Check size of off64_t - done
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for strerror
-- Looking for strerror - found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for stdarg.h
-- Looking for stdarg.h - found
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_HIDDEN
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_HIDDEN - Success
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_INTERNAL
-- Performing Test HAVE_ATTRIBUTE_VISIBILITY_INTERNAL - Success
-- Performing Test HAVE_MMAP
-- Performing Test HAVE_MMAP - Success
-- Performing Test HAVE_BUILTIN_CTZL
-- Performing Test HAVE_BUILTIN_CTZL - Success
-- Performing Test HAVE_SSE2_INTRIN
-- Performing Test HAVE_SSE2_INTRIN - Failed
-- Performing Test HAVE_SSE3_INTRIN
-- Performing Test HAVE_SSE3_INTRIN - Failed
-- Performing Test HAVE_SSSE3_INTRIN
-- Performing Test HAVE_SSSE3_INTRIN - Failed
-- Performing Test HAVE_SSE41_INTRIN
-- Performing Test HAVE_SSE41_INTRIN - Failed
-- Performing Test HAVE_SSE42_INTRIN
-- Performing Test HAVE_SSE42_INTRIN - Failed
-- Performing Test HAVE_PCLMULQDQ_INTRIN
-- Performing Test HAVE_PCLMULQDQ_INTRIN - Failed
-- Performing Test HAVE_XOP_INTRIN
-- Performing Test HAVE_XOP_INTRIN - Failed
-- Performing Test HAVE_AVX_INTRIN
-- Performing Test HAVE_AVX_INTRIN - Failed
-- Performing Test HAVE_AVX2_INTRIN
-- Performing Test HAVE_AVX2_INTRIN - Failed
-- Performing Test HAVE_TUNE_USER
-- Performing Test HAVE_TUNE_USER - Success
-- Architecture-specific source files: arch/arm/fill_window_arm.c
-- The following features have been enabled:

 * CMAKE_BUILD_TYPE, Build type: Release (selected)
 * WITH_OPTIM, Build with optimisation
 * WITH_NEW_STRATEGIES, Use new strategies

-- The following features have been disabled:

 * ZLIB_COMPAT, Provide a zlib-compatible API
 * WITH_GZFILEOP, Compile with support for gzFile-related functions
 * WITH_ASSERTS, Enable asserts
 * WITH_ACLE, Build with ACLE CRC
 * WITH_NEON, Build with NEON intrinsics

-- Configuring done
-- Generating done

Configure log
CMakeOutput.log

 ➭ make VERBOSE=1
/usr/bin/cmake -S/home/hlewin/repos/zlib-ng -B/home/hlewin/repos/zlib-ng/b --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /home/hlewin/repos/zlib-ng/b/CMakeFiles /home/hlewin/repos/zlib-ng/b//CMakeFiles/progress.marks
make  -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/home/hlewin/repos/zlib-ng/b'
make  -f CMakeFiles/zlib.dir/build.make CMakeFiles/zlib.dir/depend
make[2]: Entering directory '/home/hlewin/repos/zlib-ng/b'
cd /home/hlewin/repos/zlib-ng/b && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/hlewin/repos/zlib-ng /home/hlewin/repos/zlib-ng /home/hlewin/repos/zlib-ng/b /home/hlewin/repos/zlib-ng/b /home/hlewin/repos/zlib-ng/b/CMakeFiles/zlib.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/hlewin/repos/zlib-ng/b'
make  -f CMakeFiles/zlib.dir/build.make CMakeFiles/zlib.dir/build
make[2]: Entering directory '/home/hlewin/repos/zlib-ng/b'
[  2%] Building C object CMakeFiles/zlib.dir/adler32.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/adler32.c.o -MF CMakeFiles/zlib.dir/adler32.c.o.d -o CMakeFiles/zlib.dir/adler32.c.o -c /home/hlewin/repos/zlib-ng/adler32.c
[  4%] Building C object CMakeFiles/zlib.dir/compress.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/compress.c.o -MF CMakeFiles/zlib.dir/compress.c.o.d -o CMakeFiles/zlib.dir/compress.c.o -c /home/hlewin/repos/zlib-ng/compress.c
[  7%] Building C object CMakeFiles/zlib.dir/crc32.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/crc32.c.o -MF CMakeFiles/zlib.dir/crc32.c.o.d -o CMakeFiles/zlib.dir/crc32.c.o -c /home/hlewin/repos/zlib-ng/crc32.c
[  9%] Building C object CMakeFiles/zlib.dir/deflate.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/deflate.c.o -MF CMakeFiles/zlib.dir/deflate.c.o.d -o CMakeFiles/zlib.dir/deflate.c.o -c /home/hlewin/repos/zlib-ng/deflate.c
[ 11%] Building C object CMakeFiles/zlib.dir/deflate_fast.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/deflate_fast.c.o -MF CMakeFiles/zlib.dir/deflate_fast.c.o.d -o CMakeFiles/zlib.dir/deflate_fast.c.o -c /home/hlewin/repos/zlib-ng/deflate_fast.c
[ 14%] Building C object CMakeFiles/zlib.dir/deflate_medium.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/deflate_medium.c.o -MF CMakeFiles/zlib.dir/deflate_medium.c.o.d -o CMakeFiles/zlib.dir/deflate_medium.c.o -c /home/hlewin/repos/zlib-ng/deflate_medium.c
[ 16%] Building C object CMakeFiles/zlib.dir/deflate_slow.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/deflate_slow.c.o -MF CMakeFiles/zlib.dir/deflate_slow.c.o.d -o CMakeFiles/zlib.dir/deflate_slow.c.o -c /home/hlewin/repos/zlib-ng/deflate_slow.c
[ 19%] Building C object CMakeFiles/zlib.dir/inflate.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/inflate.c.o -MF CMakeFiles/zlib.dir/inflate.c.o.d -o CMakeFiles/zlib.dir/inflate.c.o -c /home/hlewin/repos/zlib-ng/inflate.c
[ 21%] Building C object CMakeFiles/zlib.dir/infback.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/infback.c.o -MF CMakeFiles/zlib.dir/infback.c.o.d -o CMakeFiles/zlib.dir/infback.c.o -c /home/hlewin/repos/zlib-ng/infback.c
[ 23%] Building C object CMakeFiles/zlib.dir/inftrees.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/inftrees.c.o -MF CMakeFiles/zlib.dir/inftrees.c.o.d -o CMakeFiles/zlib.dir/inftrees.c.o -c /home/hlewin/repos/zlib-ng/inftrees.c
[ 26%] Building C object CMakeFiles/zlib.dir/inffast.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/inffast.c.o -MF CMakeFiles/zlib.dir/inffast.c.o.d -o CMakeFiles/zlib.dir/inffast.c.o -c /home/hlewin/repos/zlib-ng/inffast.c
[ 28%] Building C object CMakeFiles/zlib.dir/match.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/match.c.o -MF CMakeFiles/zlib.dir/match.c.o.d -o CMakeFiles/zlib.dir/match.c.o -c /home/hlewin/repos/zlib-ng/match.c
[ 30%] Building C object CMakeFiles/zlib.dir/trees.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/trees.c.o -MF CMakeFiles/zlib.dir/trees.c.o.d -o CMakeFiles/zlib.dir/trees.c.o -c /home/hlewin/repos/zlib-ng/trees.c
[ 33%] Building C object CMakeFiles/zlib.dir/uncompr.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/uncompr.c.o -MF CMakeFiles/zlib.dir/uncompr.c.o.d -o CMakeFiles/zlib.dir/uncompr.c.o -c /home/hlewin/repos/zlib-ng/uncompr.c
[ 35%] Building C object CMakeFiles/zlib.dir/zutil.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/zutil.c.o -MF CMakeFiles/zlib.dir/zutil.c.o.d -o CMakeFiles/zlib.dir/zutil.c.o -c /home/hlewin/repos/zlib-ng/zutil.c
[ 38%] Building C object CMakeFiles/zlib.dir/arch/arm/fill_window_arm.c.o
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -DHAVE_BUILTIN_CTZL -DHAVE_HIDDEN -DHAVE_INTERNAL -DMEDIUM_STRATEGY -DUNALIGNED_OK -DUNROLL_LESS -DUSE_MMAP -DZLIB_DLL -D_LARGEFILE64_SOURCE=1 -D__USE_LARGEFILE64 -I/home/hlewin/repos/zlib-ng/b -I/home/hlewin/repos/zlib-ng -fexceptions -mtune=generic -O3 -DNDEBUG -fPIC -MD -MT CMakeFiles/zlib.dir/arch/arm/fill_window_arm.c.o -MF CMakeFiles/zlib.dir/arch/arm/fill_window_arm.c.o.d -o CMakeFiles/zlib.dir/arch/arm/fill_window_arm.c.o -c /home/hlewin/repos/zlib-ng/arch/arm/fill_window_arm.c
[ 40%] Linking C shared library libz.so
/usr/bin/cmake -E cmake_link_script CMakeFiles/zlib.dir/link.txt --verbose=1
/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/bin/clang --target=armv7-none-linux-androideabi21 --sysroot=/ldata/Android/Sdk/ndk/23.0.7123448/toolchains/llvm/prebuilt/linux-x86_64/sysroot -fPIC -fexceptions -mtune=generic -O3 -DNDEBUG -Wl,--version-script,"/home/hlewin/repos/zlib-ng/zlib.map" -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libgcc_real.a -Wl,--exclude-libs,libatomic.a -Wl,--build-id=sha1 -Wl,--no-rosegment -Wl,--fatal-warnings -Wl,--exclude-libs,libunwind.a -Qunused-arguments -Wl,--no-undefined -Wl,-rpath-link=/crypt/android-native/root/armeabi-v7a/lib -L/crypt/android-native/root/armeabi-v7a/lib -Wl,--exclude-libs,libgcc.a -Wl,--exclude-libs,libgcc_real.a -Wl,--exclude-libs,libunwind.a -shared -Wl,-soname,libz.so -o libz.so CMakeFiles/zlib.dir/adler32.c.o CMakeFiles/zlib.dir/compress.c.o CMakeFiles/zlib.dir/crc32.c.o CMakeFiles/zlib.dir/deflate.c.o CMakeFiles/zlib.dir/deflate_fast.c.o CMakeFiles/zlib.dir/deflate_medium.c.o CMakeFiles/zlib.dir/deflate_slow.c.o CMakeFiles/zlib.dir/inflate.c.o CMakeFiles/zlib.dir/infback.c.o CMakeFiles/zlib.dir/inftrees.c.o CMakeFiles/zlib.dir/inffast.c.o CMakeFiles/zlib.dir/match.c.o CMakeFiles/zlib.dir/trees.c.o CMakeFiles/zlib.dir/uncompr.c.o CMakeFiles/zlib.dir/zutil.c.o CMakeFiles/zlib.dir/arch/arm/fill_window_arm.c.o   -latomic -lm 
ld: error: undefined symbol: adler32_neon
>>> referenced by adler32.c
>>>               CMakeFiles/zlib.dir/adler32.c.o:(adler32_z)
>>> referenced by adler32.c
>>>               CMakeFiles/zlib.dir/adler32.c.o:(adler32)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [CMakeFiles/zlib.dir/build.make:337: libz.so] Error 1
make[2]: Leaving directory '/home/hlewin/repos/zlib-ng/b'
make[1]: *** [CMakeFiles/Makefile2:223: CMakeFiles/zlib.dir/all] Error 2
make[1]: Leaving directory '/home/hlewin/repos/zlib-ng/b'
make: *** [Makefile:146: all] Error 2

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin It clearly shows WITH_NEON is disabled... the configuration step doesn't show all the expected output so CMakeCache.txt might be stale.

@hlewin
Copy link

hlewin commented Jul 21, 2021

Observations:

adler32.c only checks for __ARM_NEON, which IS set by the compiler

uint32_t ZEXPORT adler32_z(uint32_t adler, const unsigned char *buf, size_t len) {
#if X86_CPUID
    return adler32_x86(adler, buf, len);
#elif (defined(__ARM_NEON__) || defined(__ARM_NEON))
    return adler32_neon(adler, buf, len);
#else
    return adler32_c(adler, buf, len);
#endif
}

The build script (with this patch) deactivates NEON alltogether and hence does not build arch/arm/adler32_neon.c.
I wondered about this as well, but the regular develop branch keeps NEON activated.

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin my machine doesn't even have those lines.

@hlewin
Copy link

hlewin commented Jul 21, 2021

@mtl1979 I am sorry. I was on the wrong branch shame

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin People make mistakes, but in the end we're here to help people even if the error is between computer and the chair.

@hlewin
Copy link

hlewin commented Jul 21, 2021

However, I still get the bus error...

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

armv7 should support unaligned reads that are 16 or 32 bits (2 or 4 bytes)... But your processor might be exception...

@hlewin
Copy link

hlewin commented Jul 21, 2021

I am really not familiar with arms. There are too much variations.

shell@hwbg2:/ $ cat /proc/cpuinfo                                              
Processor	: ARMv7 Processor rev 3 (v7l)
processor	: 0
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 26.00
Features	: half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 3

processor	: 1
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 26.00
Features	: half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 3

processor	: 2
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 26.00
Features	: half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 3

processor	: 3
model name	: ARMv7 Processor rev 3 (v7l)
BogoMIPS	: 26.00
Features	: half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xc07
CPU revision	: 3

Hardware	: MT8127
Revision	: 0000
Serial		: 0000000000000000

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin Me neither... I need to figure out when UNALIGNED_OK also needs to be disabled...

@hlewin
Copy link

hlewin commented Jul 21, 2021

I have tried

void main() {
	volatile char ch[8];
	volatile short x = *(short*)(ch+0);
	volatile short y = *(short*)(ch+1);
	volatile short a = *(short*)(ch+2);
	volatile short b = *(short*)(ch+3);
}

with both shorts and ints and do not get a bus error.

…ier.

* armv7 has partial support for unaligned reads, but compiler might use instructions that do not support unaligned accesses
@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin When you allocate from stack, it's always aligned. When you offset it, it knows that the access is unaligned.

The problem is when the pointer is passed as function parameter. compiler might not know where the variable was allocated from, heap or stack.

@hlewin
Copy link

hlewin commented Jul 21, 2021

Running this with -fsanitize=alignment just gives reads <= 4 bytes

WARNING: linker: ./example: unsupported flags DT_FLAGS_1=0x8000001
zlib-ng version 2.1.0.devel = 0x02010000, compile flags = 0x155
/home/hlewin/repos/zlib-ng/insert_string_tpl.h:67:5: runtime error: load of misaligned address 0xb6aae001 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae001: note: pointer points here
 00 00 00  68 65 6c 6c 6f 2c 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/insert_string_tpl.h:67:5 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:88:18: runtime error: load of misaligned address 0xb6aae009 for type 'bestcmp_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae009: note: pointer points here
 2c 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:88:18 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:157:21: runtime error: load of misaligned address 0xb6aae001 for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6aae001: note: pointer points here
 00 00 00  68 65 6c 6c 6f 2c 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:157:21 in 
/home/hlewin/repos/zlib-ng/compare258.c:126:23: runtime error: load of misaligned address 0xb6aae00a for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae00a: note: pointer points here
 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/compare258.c:126:23 in 
/home/hlewin/repos/zlib-ng/compare258.c:127:23: runtime error: load of misaligned address 0xb6aae003 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae003: note: pointer points here
 00  68 65 6c 6c 6f 2c 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/compare258.c:127:23 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:210:24: runtime error: load of misaligned address 0xb6aae009 for type 'bestcmp_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae009: note: pointer points here
 2c 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:210:24 in 
/home/hlewin/repos/zlib-ng/chunkset.c:54:21: runtime error: load of misaligned address 0xb6a95001 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6a95001: note: pointer points here
 00 00 00  68 65 6c 6c 6f 2c 20 68  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/chunkset.c:54:21 in 
/home/hlewin/repos/zlib-ng/chunkset.c:55:21: runtime error: load of misaligned address 0xb6a95005 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6a95005: note: pointer points here
 65 6c 6c 6f 2c 20 68  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/chunkset.c:55:21 in 
uncompress(): hello, hello!
/home/hlewin/repos/zlib-ng/deflate.h:331:5: runtime error: store to misaligned address 0xb6a02003 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6a02003: note: pointer points here
 00  1f 8b 08 96 c9 d7 51 c8  00 51 8a 0c 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/deflate.h:331:5 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:88:18: runtime error: load of misaligned address 0xb6aae015 for type 'bestcmp_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae015: note: pointer points here
 6c 6f 2c 20 68 65 6c  6c 6f 21 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:88:18 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:157:21: runtime error: load of misaligned address 0xb6aae00d for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6aae00d: note: pointer points here
 6c 6c 6f 21 00 68 65  6c 6c 6f 2c 20 68 65 6c  6c 6f 21 00 00 00 00 00  00 00 00 00 00 00 00 00  00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:157:21 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:92:18: runtime error: load of misaligned address 0xb6aae015 for type 'bestcmp_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae015: note: pointer points here
 6c 6f 2c 20 68 65 6c  6c 6f 21 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:92:18 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:156:21: runtime error: load of misaligned address 0xb6aae00f for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6aae00f: note: pointer points here
 6f 21 00 68 65  6c 6c 6f 2c 20 68 65 6c  6c 6f 21 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:156:21 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:210:24: runtime error: load of misaligned address 0xb6aae017 for type 'bestcmp_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae017: note: pointer points here
 2c 20 68 65 6c  6c 6f 21 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:210:24 in 
/home/hlewin/repos/zlib-ng/chunkset.c:69:5: runtime error: store to misaligned address 0xb69c0015 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb69c0015: note: pointer points here
 6c 6f 2c 20 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/chunkset.c:69:5 in 
/home/hlewin/repos/zlib-ng/chunkset.c:70:5: runtime error: store to misaligned address 0xb69c0019 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb69c0019: note: pointer points here
 68 65 6c  6c 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/chunkset.c:70:5 in 
gzread(): hello, hello!
gzgets() after gzseek:  hello!
gzgets(): hello, hello!
inflate(): hello, hello!
/home/hlewin/repos/zlib-ng/compare258.c:142:9: runtime error: load of misaligned address 0xb6aae001 for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6aae001: note: pointer points here
 00 00 00  68 65 6c 6c 6f 2c 20 68  65 6c 6c 6f 21 00 20 68  65 6c 6c 6f 21 00 00 6c  6c 6f 21 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/compare258.c:142:9 in 
/home/hlewin/repos/zlib-ng/compare258.c:142:30: runtime error: load of misaligned address 0xb6aae01b for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6aae01b: note: pointer points here
 6c  6c 6f 21 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/compare258.c:142:30 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:172:21: runtime error: load of misaligned address 0xb6ab7ddd for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6ab7ddd: note: pointer points here
 65 6c 6c 6f 2c 20 68  65 6c 6c 6f 21 00 20 68  65 6c 6c 6f 21 00 00 6c  6c 6f 21 00 00 00 00 00  00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:172:21 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:173:21: runtime error: load of misaligned address 0xb6ab7b4d for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6ab7b4d: note: pointer points here
 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00
             ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:173:21 in 
large_inflate(): OK
/home/hlewin/repos/zlib-ng/deflate.h:308:5: runtime error: store to misaligned address 0xb6a02001 for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6a02001: note: pointer points here
 00 00 00  00 97 10 00 32 04 32 52  73 72 f2 75 14 c0 94 22  03 8c 66 c8 c9 c9 57 1c  94 ee 1d 75 d4
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/deflate.h:308:5 in 
/home/hlewin/repos/zlib-ng/deflate.h:308:5: runtime error: store to misaligned address 0xb6a02001 for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6a02001: note: pointer points here
 00 00 00  00 00 00 ff ff 04 32 52  73 72 f2 75 14 c0 94 22  03 8c 66 c8 c9 c9 57 1c  94 ee 1d 75 d4
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/deflate.h:308:5 in 
large_inflate(): OK
after inflateSync(): hello, hello!
/home/hlewin/repos/zlib-ng/insert_string_tpl.h:96:9: runtime error: load of misaligned address 0xb6aae001 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6aae001: note: pointer points here
 00 00 00  68 65 6c 6c 6f 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/insert_string_tpl.h:96:9 in 
/home/hlewin/repos/zlib-ng/deflate.h:347:5: runtime error: store to misaligned address 0xb6a02002 for type 'uint32_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6a02002: note: pointer points here
 00 00  78 f9 04 96 c8 48 cd c9  c9 57 64 00 00 09 29 02  ee df 59 88 23 08 16 77  c5 3f 7f ca 59 b5
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/deflate.h:347:5 in 
inflate with dictionary: hello, hello!
deflateBound(): OK
deflate_copy(): OK
deflateGetDictionary(): hello, hello!
deflateSetHeader(): OK
deflateTune(): OK
/home/hlewin/repos/zlib-ng/match_tpl.h:92:18: runtime error: load of misaligned address 0xb6982009 for type 'bestcmp_t' (aka 'unsigned int'), which requires 4 byte alignment
0xb6982009: note: pointer points here
 2c 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:92:18 in 
/home/hlewin/repos/zlib-ng/match_tpl.h:156:21: runtime error: load of misaligned address 0xb6982003 for type 'uint16_t' (aka 'unsigned short'), which requires 2 byte alignment
0xb6982003: note: pointer points here
 00  68 65 6c 6c 6f 2c 20 68  65 6c 6c 6f 21 00 00 00  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00
              ^ 
SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /home/hlewin/repos/zlib-ng/match_tpl.h:156:21 in 
deflatePending(): OK
deflatePrime(): OK

@hlewin
Copy link

hlewin commented Jul 21, 2021

Running this with a debug-build, though, gives a working executable. Maybe the compiler knows more than you think....

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin Debug build disables optimizations, which means compiler doesn't try to combine multiple loads and stores as single wider access...

For some platforms using -O2 instead of -O3 stopped compiler from combining loads and stores.

@hlewin
Copy link

hlewin commented Jul 21, 2021

Process 24927 stopped
* thread #1, name = 'example', stop reason = signal SIGBUS: illegal alignment
    frame #0: 0xb6f39114
->  0xb6f39114: ldm    r1, {r3, r12}
    0xb6f39118: stm    r0, {r3, r12}
    0xb6f3911c: sub    r3, r2, #1
    0xb6f39120: and    r12, r3, #7

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin ldm and stm don't support unaligned accesses on armv7.

@hlewin
Copy link

hlewin commented Jul 21, 2021

Finally, I got the remote debugger to work properly. You are right. Seems like 2 reads were aggregated:

Process 26972 stopped
* thread #1, name = 'example', stop reason = signal SIGBUS: illegal alignment
    frame #0: 0xb6f39114 libz-ng.so`chunkcopy_c [inlined] chunkmemset_8(from="ello, h", chunk=<unavailable>) at chunkset.c:54:21
   51  	    *chunk = *(uint64_t *)from;
   52  	#elif defined(UNALIGNED_OK)
   53  	    uint32_t* p = (uint32_t *)from;
-> 54  	    chunk->u32[0] = p[0];
   55  	    chunk->u32[1] = p[1];
   56  	#else
   57  	    memcpy(chunk, from, sizeof(chunk_t));

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

The latest version should not define UNALIGNED_OK, so that codepath is not used. Delete CMakeCache.txt between builds so that variable is not cached.

@hlewin
Copy link

hlewin commented Jul 21, 2021

Though it likely fixes my problem, it does not seem sufficient to disallow this on armv7. There is a separate UNALIGNED64_OK flag which gets kinda useless if UNALIGNED_OK leaves the possibility for the compiler to do such stuff. This might happen on any platform.

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin Combining memory accesses happens on most platforms, but only few don't support unaligned accesses... Older ARM processors and big-endian PowerPC are the one that I know...

@hlewin
Copy link

hlewin commented Jul 21, 2021

Okay, this is your decision. I just wanted to point out that UNALIGNED_OK and UNALIGNED64_OK become synonymous under these circumstances.
The fixed version seems to work fine so far. Thank you for your quick help!

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin I did read about suggestion to add -mno-multiple command-line option to gcc that disables combining memory loads and stores, but I'm not sure if any gcc version supports that yet...

@hlewin
Copy link

hlewin commented Jul 21, 2021

There are some attributes for variables that might help.
"packed" for gcc: https://gcc.gnu.org/onlinedocs/gcc-3.2/gcc/Variable-Attributes.html
"align_value" for clang: https://clang.llvm.org/docs/AttributeReference.html#align_value

I dunno if these would really work in this case, though.

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin packed attribute has nothing to do with combining two variables with adjacent memory locations, it just changes the expected alignment restriction.

@hlewin
Copy link

hlewin commented Jul 21, 2021

Yes - if the compiler knows that a variable is not on a properly aligned address, it will not generate an illegal load instruction.

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin But it would break the code on other platforms unless the attribute is wrapped inside preprocessor macro that gets expanded only on required platforms... It would make the code quite hard to maintain.

@hlewin
Copy link

hlewin commented Jul 21, 2021

The code already is full of #ifdef UNALIGNED_OK and stuff.
Reviewing the code that caused the signal, the question even is "Why not use memcpy() in all cases?". Any decent compiler will give you the best possible memory transfer (and not really call a function), It is not, that the code is full of constexpr which would prohibit that...

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin We did do speed test with memcpy() and it was actually quite a lot slower.

@hlewin
Copy link

hlewin commented Jul 21, 2021

That again, would be an argument for the attributes if you mind armv7 targets. For my work, setting UNALIGNED_OK and getting the slow memcpy code-path even if the 32-bit path could work, is kinda okay as nobody uses armv7 androids anymore...

PS: But this is all speculation. Again, thank you for your quick support on this one.

@mtl1979
Copy link
Collaborator Author

mtl1979 commented Jul 21, 2021

@hlewin Officially we (zlib-ng) support as old as armv2 (with all optimized code paths disabled) but that doesn't mean the generated code is optimal for anything other than armv8 or newer.

@Dead2 Dead2 merged commit 747ffeb into zlib-ng:develop Aug 11, 2021
Dead2 added a commit that referenced this pull request Dec 13, 2021
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050
@Dead2 Dead2 mentioned this pull request Dec 13, 2021
Dead2 added a commit that referenced this pull request Dec 20, 2021
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1071
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1071
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1071
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1049 #1071
 - Fix incorrect function declaration warning #1080
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1049 #1051 #1056 #1063 #1067
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 20, 2021
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1049 #1071
 - Fix incorrect function declaration warning #1080
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1049 #1051 #1056 #1063 #1067 #1079
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Dead2 added a commit that referenced this pull request Dec 24, 2021
 - Fix hangs on macOS #1031
 - Fix minideflate write buffers being overwritten #1060
 - Fix deflateBound and compressBound returning too small size estimates #1049 #1071
 - Fix incorrect function declaration warning #1080
 - Fix build problems when building outside of source dir #1049
 - Fix build problems on arm2-7 #1030
 - Fixed some compile warnings #1020 #1036 #1037 #1048
 - Improved posix memalign support #888
 - Improvements to testing #637 #1026 #1032 #1035 #1049 #1051 #1056 #1063 #1067 #1079
 - Improvements for integration into other projects #1022 #1042
 - Code style fixes #637 #1040 #1050 #1075
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants