GNUmakefile fails to properly accommodate for AVX512f

[mouse07410]

MacOS 10.14.1 Mojave. Xcode-10.1. Current master.

Problem: no appropriate tests for AVX512 extended instruction set.

Symptoms:

$ CXX=g++ make all test
Using testing flags: -maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdseed -mrdrnd -mavx2 -mavx512f -msha -std=gnu++17 -Os -Ofast -Wa,-q -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1
g++ -maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdseed -mrdrnd -mavx2 -mavx512f -msha -std=gnu++17 -Os -Ofast -Wa,-q -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1 -fPIC -pthread -pipe -mavx2 -msse4.2 -msha -c integer.cpp
<stdin>:11397:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdi,%rax), %zmm0
        ^
<stdin>:11398:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rsi,%rax)
        ^
<stdin>:14420:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rsi,%rax), %zmm0
        ^
<stdin>:14421:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%r10,%rax)
        ^
<stdin>:14616:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rax,%rdx), %zmm1
        ^
<stdin>:14617:2: error: instruction requires: AVX-512 ISA
        vpandq  (%rcx,%rdx), %zmm1, %zmm0
        ^
<stdin>:14618:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rax,%rdx)
        ^
<stdin>:14731:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdi,%rax), %zmm1
        ^
<stdin>:14732:2: error: instruction requires: AVX-512 ISA
        vporq   (%rdx,%rax), %zmm1, %zmm0
        ^
<stdin>:14733:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdi,%rax)
        ^
<stdin>:14816:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rax,%rdx), %zmm2
        ^
<stdin>:14817:2: error: instruction requires: AVX-512 ISA
        vporq   (%rsi,%rdx), %zmm2, %zmm0
        ^
<stdin>:14818:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rax,%rdx)
        ^
<stdin>:15640:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%rax), %zmm0
        ^
<stdin>:15641:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%r12,%rax)
        ^
<stdin>:15826:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdi,%rax), %zmm1
        ^
<stdin>:15827:2: error: instruction requires: AVX-512 ISA
        vpxorq  (%rdx,%rax), %zmm1, %zmm0
        ^
<stdin>:15828:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdi,%rax)
        ^
<stdin>:15915:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rax,%rdx), %zmm2
        ^
<stdin>:15916:2: error: instruction requires: AVX-512 ISA
        vpxorq  (%rsi,%rdx), %zmm2, %zmm0
        ^
<stdin>:15917:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rax,%rdx)
        ^
<stdin>:16302:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%rax), %zmm1
        ^
<stdin>:16303:2: error: instruction requires: AVX-512 ISA
        vpandq  (%rcx,%rax), %zmm1, %zmm0
        ^
<stdin>:16304:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdx,%rax)
        ^
<stdin>:16381:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%rax), %zmm2
        ^
<stdin>:16382:2: error: instruction requires: AVX-512 ISA
        vpandq  (%rcx,%rax), %zmm2, %zmm0
        ^
<stdin>:16383:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdx,%rax)
        ^
<stdin>:16637:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%rax), %zmm1
        ^
<stdin>:16638:2: error: instruction requires: AVX-512 ISA
        vporq   (%rcx,%rax), %zmm1, %zmm0
        ^
<stdin>:16639:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdx,%rax)
        ^
<stdin>:16717:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%rax), %zmm2
        ^
<stdin>:16718:2: error: instruction requires: AVX-512 ISA
        vporq   (%rcx,%rax), %zmm2, %zmm0
        ^
<stdin>:16719:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdx,%rax)
        ^
<stdin>:16972:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%rax), %zmm1
        ^
<stdin>:16973:2: error: instruction requires: AVX-512 ISA
        vpxorq  (%rcx,%rax), %zmm1, %zmm0
        ^
<stdin>:16974:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdx,%rax)
        ^
<stdin>:17052:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%rax), %zmm2
        ^
<stdin>:17053:2: error: instruction requires: AVX-512 ISA
        vpxorq  (%rcx,%rax), %zmm2, %zmm0
        ^
<stdin>:17054:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       %zmm0, (%rdx,%rax)
        ^
make: *** [integer.o] Error 1

Proposed solution: add -mavx512f when appropriate, and for Macports GCC - add -Wa,-mavx512f too (somehow this is not picked by default). Like this:

$ g++ -maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdseed -mrdrnd -mavx2 -mavx512f -msha -std=gnu++17 -Os -Ofast -Wa,-q -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1 -fPIC -pthread -pipe -mavx2 -msse4.2 -msha -Wa,-mavx512f -c integer.cpp
$ 

[mouse07410]

Another alternative would be when you discover OSXPORT_COMPILER, to add -Wa,-march=native to -Wa,-q.

[noloader]

Thanks @mouse07410.

This looks like it could be a Clang issue. I'm still having trouble understanding why you need more than -mavx512f.

Other projects are experiencing it:

• build error on MacOSX: error: instruction requires: AVX-512 ISA deeplearning4j/deeplearning4j#2668
• Assembler checks needed flame/blis#186
• Unable to build libpgmath flang-compiler/flang#434
• Non-fatal 'error' with global_asm "instruction requires: AVX-512 ISA" on target that supports AVX-512 rust-lang/rust#50477

It looks like the other projects switch to GCC and avoid the problem with Clang.

Looking at LLVM bug tracker it does not look like the issue has been reported:

• https://www.google.com/search?q="error%3A+instruction+requires%3A+AVX-512+ISA"+site%3Abugs.llvm.org

Can you reduce the problem to a simple reproducer and report it to the LLVM folks? They may have some recommendations. (The other projects I looked at lack a reproducer, too).

A reproducer would be especially helpful because we can add it to our TestPrograms/ and deterministically determine when we need the additional flags.

I think we have to be careful about -march=native. We used to use it (up to Crypto++ 5.6.x; switch at 6.0) and it got us into hot water with distros and users who distribute their software. It was the reason for the cut-over to BASE+SIMD compilation model.

[mouse07410]

This is not exactly Clang issue because native clang compile‘s fine on that platform. the issue is when Macports needs to involve integrated assembler from native clang

[noloader]

the issue is when Macports needs to involve integrated assembler from native clang

OK, thanks. I believe that makes it a GCC issue.

Related, I wonder why the other projects had the issue when compiling with Clang, but not GCC. In the other bug reports GCC was the fix (and not the problem).

---

What do you see when you examine the *.s file? Something like:

g++ --save-temps -maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdseed -mrdrnd -mavx2 \
  -mavx512f -msha -std=gnu++17 -Os -Ofast -Wa,-q -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1 \
  -fPIC -pthread -pipe -mavx2 -msse4.2 -msha -c integer.cpp

And then:

cat -n integer.s

Finally, check around line 11397. You should see a .cpu directive, if I recall correctly.

[noloader]

@mouse07410

According to the Intel Intrinsics Guide, vmovdqu64 is AVX512VL + AVX512F (and not just AVX512F).

Is it possible you are missing one of your manual flags?

Looking at GCC's x86 Options there is a -mavx512vl, and several other tuning options that include AVX512VL like skylake-avx512 and cannonlake.

[mouse07410]

Is it possible you are missing one of your manual flags?

I think not. Especially since it all compiles (and runs!) nicely under the native Clang toolchain (with the same flags, of course). And contrary to what the Guide might say, it works on my machine without -mavx512vl. Also, botanical cpuid does show avx512f, but not avx512vl.

Related, I wonder why the other projects had the issue when compiling with Clang, but not GCC. In the other bug reports GCC was the fix (and not the problem).

Different platforms, different compilers, different flags -> different causes for an issue that might look somewhat similar. Like in medicine - you wouldn't believe how many possible causes a simple headache can have. ;-)

I think we have to be careful about -march=native. We used to use it (up to Crypto++ 5.6.x; switch at 6.0) and it got us into hot water with distros and users who distribute their software. It was the reason for the cut-over to BASE+SIMD compilation model.

I don't buy this logic. The place I'm proposing to add this is here:

# Tell MacPorts and Homebrew GCC to use Clang integrated assembler
#   http://github.com/weidai11/cryptopp/issues/190
ifeq ($(GCC_COMPILER)$(OSXPORT_COMPILER),11)
  ifeq ($(findstring -Wa,-q,$(CXXFLAGS)),)
    CXXFLAGS += -Wa,-q -Wa,-march=native
  endif
  ifeq ($(findstring -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER,$(CXXFLAGS)),)
    CXXFLAGS += -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1
  endif
endif

It is infeasible to impossible that a distro builder would build on MacOS using Macports-based GCC calling integrated assembler, but would need something different. This is a very localized problem, and a very localized fix.

On the other hand, if you see any fault in my logic - here's the time and place to present it. ;-)

[noloader]

When I go to the Intrinsic Guide it says vmovdqu64 is AVX512VL + AVX512F. I think that means or. If I hover over AVX-512 menu checkbox then it expands like an accordion and I can select AVX512VL by itself. The Guide says vmovdqu64 is available when AVX512VL is selected by itself. The Guide says also says vmovdqu64 is available when AVX512F is selected by itself. So it should be available with either of them.

Now, turning back to your issue, you are using -mavx512f. vmovdqu64 is available per Intel Intrinsics Guide. GCC generates instructions for the AVX512F instruction set, including vmovdqu64. However, the Integrated Assembler rejects the instructions. Hence, this is a Clang bug.

I think we need to get the LLVM folks a reproducer so they can clear this before other folks experience it.

[noloader]

@mouse07410,

A quick test on my old MacBook without AVX2:

$ git diff
diff --git a/GNUmakefile b/GNUmakefile
index 42f13fc1..91059fea 100755
--- a/GNUmakefile
+++ b/GNUmakefile
@@ -104,7 +104,7 @@ endif
 #   http://github.com/weidai11/cryptopp/issues/190
 ifeq ($(GCC_COMPILER)$(OSXPORT_COMPILER),11)
   ifeq ($(findstring -Wa,-q,$(CXXFLAGS)),)
-    CXXFLAGS += -Wa,-q
+    CXXFLAGS += -Wa,-q -Wa,-march=native
   endif
   ifeq ($(findstring -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER,$(CXXFLAGS)),)
     CXXFLAGS += -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1

And then:

$ CXX=/opt/local/bin/g++-mp-6 CXXFLAGS="-maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdseed -mrdrnd -mavx2 -mavx512f -msha -std=gnu++17 -Os -Ofast" make
Using testing flags: -maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdseed -mrdrnd -mavx2 -mavx512f -msha -std=gnu++17 -Os -Ofast -Wa,-q -Wa,-march=native -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1
/opt/local/bin/g++-mp-6 -maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdseed -mrdrnd -mavx2 -mavx512f -msha -std=gnu++17 -Os -Ofast -Wa,-q -Wa,-march=native -DCRYPTOPP_CLANG_INTEGRATED_ASSEMBLER=1 -fPIC -pthread -pipe -c integer.cpp
<stdin>:15229:2: error: instruction requires: AVX-512 ISA
        vmovdqu64       (%rdx,%r9), %zmm0
        ^
<stdin>:15230:2: error: instruction requires: AVX-512 ISA
        vpandq  (%r14,%r9), %zmm0, %zmm0
        ^
...

So this is a good sign. I can duplicate the issue even though I can't test the hardware caps.

[mouse07410]

However, the Integrated Assembler rejects the instructions. Hence, this is a Clang bug.

No, the Integrated Assembler rejects it only when it was not given the -mavx512f flag. Native Clang passes it on to the assembler. Somehow it does not get passed from Macports GCC to it unless I also add -Wa,-mavx512f or -Wa,-march=native. (I'm still searching for the exact meaning of -march-native flag).

I can duplicate the issue even though I can't test the hardware caps.

I don't think you duplicated my issue - because in your case the appropriate flag was passed to the Integrated Assembler.

I wonder what version of Clang/Xcode you have on that old MacBook. When I use Xcode-10.1, then it works correctly on the latest hardware with AVX512f, on the fairly modern hardware with AVX2 and SHANI but no AVX512, and on the older MacBook Air with only AVX2. Here are the machines (all run-ing Mojave or High Sierra, all using Xcode-10.1 and Macports GCC8 (output of botan cpuid):

CPUID flags: sse2 ssse3 sse41 sse42 avx2 avx512f rdtsc bmi1 bmi2 adx aes_ni clmul rdrand rdseed
CPUID flags: sse2 ssse3 sse41 sse42 avx2 rdtsc bmi1 bmi2 adx aes_ni clmul rdrand rdseed
CPUID flags: sse2 ssse3 sse41 sse42 avx2 rdtsc bmi1 bmi2 aes_ni clmul rdrand

I'd like you try the following with GNUmakefile that includes -Wa,-march=native:

$ botan cpuid
$ CXX=/opt/local/bin/g++-mp-6 CXXFLAGS="-maes -mpclmul -msse2 -mssse3 -msse4 -msse4.2 -mrdrnd -std=gnu++17 -Os -Ofast" make distclean all

Because I suspect that provided CXXFLAGS should match the CPU the build is running on (and if it doesn't - that's what GNUmakefile-cross is supposed to be for).

P.S. Note, that with Macports Clang the problem doesn't manifest itself - only with Macports GCC.

[noloader]

I can duplicate the issue even though I can't test the hardware caps.

OK, now open in the LLVM bug tracker: Issue 39875, Error: instruction requires: AVX-512 ISA when using -mavx512f.

It looks like the is a LLVM bug. I guess we need to ping MacPorts next and ask them to pick up the patch.

[noloader]

It looks like the is a LLVM bug...

The issue was cleared in Clang 7.0:

$ sudo port install clang-7.0
...

$ sudo port select clang mp-clang-7.0
Selecting 'mp-clang-7.0' for 'clang' succeeded. 'mp-clang-7.0' is now active.
$ /opt/local/bin/g++-mp-6 -mavx512f -Wa,-q test.cxx -o test.exe
$

Can you update to Clang 7.0 through Macports or use our Clang 7.0 build script?

If not, I think the way to go here is dynamically test for the need to add the extra options. Fortunately we have that now, so there's no need to shell out for Clang version numbers. I'll upload the test program for you.

[mouse07410]

My Macports Clang had been 7.0 for quite a while.

The problem is - Macports Clang has other issues, for example it fails to compile Botan with Boost... So it cannot replace the Xcode Clang as the default compiler for the platform.

[noloader]

A quick status.. I can compile the library with GCC 5/6/7 using MacPorts GCC. However, I need Clang 7.0 when using your CXXFLAGS and the integrated assembler.

---

I wonder what version of Clang/Xcode you have on that old MacBook.

Oh man, my gear is old. The Macbook is a Core2 Duo from 2011 or 2012 running 10.8.5.

$ /usr/bin/xcodebuild -version
Xcode 6.2
Build version 6C131e

$ ./botan cpuid
CPUID flags: sse2 ssse3 sse41 rdts

When I attempt to duplicate your environment and issues I use MacPorts:

$ ls /opt/local/bin/g++*
/opt/local/bin/g++-mp-5 /opt/local/bin/g++-mp-6 /opt/local/bin/g++-mp-7

$ ls /opt/local/bin/clang++*
/opt/local/bin/clang++          /opt/local/bin/clang++-mp-5.0
/opt/local/bin/clang++-mp-3.7   /opt/local/bin/clang++-mp-6.0
/opt/local/bin/clang++-mp-3.9   /opt/local/bin/clang++-mp-7.0
/opt/local/bin/clang++-mp-4.0

---

I think we have to be careful about -march=native. We used to use it (up to Crypto++ 5.6.x; switch at 6.0) and it got us into hot water with distros and users who distribute their software. It was the reason for the cut-over to BASE+SIMD compilation model.

I don't buy this logic...

Well, there's several bug reports from the -march=native days. See for example, Random crashes on different computers because option -march=native. Those folks were ready to tar and feather us because of -march=native.

We could not simply drop -march=native because we lost the higher ISAs. It was actually worse since it broke the compile because the machinery assumed the ISA would be available. To drop -march=native we needed to switch to BASE+SIMD so we could activate an appropriate ISA for a source file.

Finding the appropriate option and activating an ISA hit its limits on ARM, Aarch64 and PowerPC so we recently switched to the dynamic feature detection cut-in last week. In reality it hit the limit a long time ago. And it was also broke for x86_64 on distros like OpenBSD and old distros like Fedora 15 but you probably missed it.

[noloader]

@mouse07410,

Now that we have a reproducer, an upstream bug report, the cause and a fix, I think you should file a bug report in Apple's RADAR. Apple needs to be informed of things that need to be fixed.

If you have the time, open another report at Open RADAR. Open RADAR tries to make Apple bugs transparent. You just copy/paste the details from RADAR to Open RADAR.

[mouse07410]

Oh man, my gear is old. The Macbook is a Core2 Duo from 2011 or 2012 running 10.8.5

I'm pretty sure that you could bring the OS up to probably 10.12, if not 10.13 - and correspondingly, Xcode to at least 9.4.1, if not 10.1.

Unfortunately, your workaround fixes one thing but breaks another - it makes it impossible to compile Botan with Boost:

. . . . .
clang++ -fPIC -fvisibility=hidden -fstack-protector -m64 -pthread -stdlib=libc++ -maes -mpclmul -mrdrnd -msse2 -mssse3 -msse4 -msse4.2 -Os -Ofast -I/opt/local/include -std=c++11 -D_REENTRANT -maes -mpclmul -mrdrnd -mrdseed -msse2 -mssse3 -msse4.1 -msse4.2 -mavx2 -std=gnu++17 -Os -Ofast -Wall -Wextra -Wpedantic -Wshadow -Wstrict-aliasing -Wstrict-overflow=5 -Wcast-align -Wmissing-declarations -Wpointer-arith -Wcast-qual  -Ibuild/include -Ibuild/include/external -c src/lib/utils/socket/socket.cpp -o build/obj/lib/utils_socket.o
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:23:
In file included from /opt/local/include/boost/asio/basic_datagram_socket.hpp:20:
In file included from /opt/local/include/boost/asio/basic_socket.hpp:40:
In file included from /opt/local/include/boost/asio/detail/reactive_socket_service.hpp:22:
In file included from /opt/local/include/boost/asio/buffer.hpp:27:
In file included from /opt/local/include/boost/asio/detail/string_view.hpp:23:
/opt/local/libexec/llvm-7.0/include/c++/v1/experimental/string_view:11:2: error: "<experimental/string_view> has been removed. Use <string_view> instead."
#error "<experimental/string_view> has been removed. Use <string_view> instead."
 ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:23:
In file included from /opt/local/include/boost/asio/basic_datagram_socket.hpp:20:
In file included from /opt/local/include/boost/asio/basic_socket.hpp:40:
In file included from /opt/local/include/boost/asio/detail/reactive_socket_service.hpp:22:
In file included from /opt/local/include/boost/asio/buffer.hpp:27:
/opt/local/include/boost/asio/detail/string_view.hpp:32:12: error: no member named 'experimental' in namespace 'std'
using std::experimental::basic_string_view;
      ~~~~~^
/opt/local/include/boost/asio/detail/string_view.hpp:33:12: error: no member named 'experimental' in namespace 'std'
using std::experimental::string_view;
      ~~~~~^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:23:
In file included from /opt/local/include/boost/asio/basic_datagram_socket.hpp:20:
In file included from /opt/local/include/boost/asio/basic_socket.hpp:40:
In file included from /opt/local/include/boost/asio/detail/reactive_socket_service.hpp:22:
/opt/local/include/boost/asio/buffer.hpp:1489:5: error: no template named 'basic_string_view'; did you mean 'std::basic_string_view'?
    basic_string_view<Elem, Traits> data) BOOST_ASIO_NOEXCEPT
    ^~~~~~~~~~~~~~~~~
    std::basic_string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:194:28: note: 'std::basic_string_view' declared here
class _LIBCPP_TEMPLATE_VIS basic_string_view {
                           ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:23:
In file included from /opt/local/include/boost/asio/basic_datagram_socket.hpp:20:
In file included from /opt/local/include/boost/asio/basic_socket.hpp:40:
In file included from /opt/local/include/boost/asio/detail/reactive_socket_service.hpp:22:
/opt/local/include/boost/asio/buffer.hpp:1510:5: error: no template named 'basic_string_view'; did you mean 'std::basic_string_view'?
    basic_string_view<Elem, Traits> data,
    ^~~~~~~~~~~~~~~~~
    std::basic_string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:194:28: note: 'std::basic_string_view' declared here
class _LIBCPP_TEMPLATE_VIS basic_string_view {
                           ^
clang++ -fPIC -fvisibility=hidden -fstack-protector -m64 -pthread -stdlib=libc++ -maes -mpclmul -mrdrnd -msse2 -mssse3 -msse4 -msse4.2 -Os -Ofast -I/opt/local/include -std=c++11 -D_REENTRANT -maes -mpclmul -mrdrnd -mrdseed -msse2 -mssse3 -msse4.1 -msse4.2 -mavx2 -std=gnu++17 -Os -Ofast -Wall -Wextra -Wpedantic -Wshadow -Wstrict-aliasing -Wstrict-overflow=5 -Wcast-align -Wmissing-declarations -Wpointer-arith -Wcast-qual  -Ibuild/include -Ibuild/include/external -c src/lib/utils/sqlite3/sqlite3.cpp -o build/obj/lib/utils_sqlite3.o
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:24:
/opt/local/include/boost/asio/ip/address_v4.hpp:288:44: error: unknown type name 'string_view'; did you mean 'std::string_view'?
BOOST_ASIO_DECL address_v4 make_address_v4(string_view str);
                                           ^~~~~~~~~~~
                                           std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:24:
/opt/local/include/boost/asio/ip/address_v4.hpp:295:5: error: unknown type name 'string_view'; did you mean 'std::string_view'?
    string_view str, boost::system::error_code& ec);
    ^~~~~~~~~~~
    std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:24:
In file included from /opt/local/include/boost/asio/ip/address_v4.hpp:328:
/opt/local/include/boost/asio/ip/impl/address_v4.ipp:193:28: error: unknown type name 'string_view'; did you mean 'std::string_view'?
address_v4 make_address_v4(string_view str)
                           ^~~~~~~~~~~
                           std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:24:
In file included from /opt/local/include/boost/asio/ip/address_v4.hpp:328:
/opt/local/include/boost/asio/ip/impl/address_v4.ipp:198:28: error: unknown type name 'string_view'; did you mean 'std::string_view'?
address_v4 make_address_v4(string_view str,
                           ^~~~~~~~~~~
                           std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:25:
/opt/local/include/boost/asio/ip/address_v6.hpp:277:44: error: unknown type name 'string_view'; did you mean 'std::string_view'?
BOOST_ASIO_DECL address_v6 make_address_v6(string_view str);
                                           ^~~~~~~~~~~
                                           std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:25:
/opt/local/include/boost/asio/ip/address_v6.hpp:284:5: error: unknown type name 'string_view'; did you mean 'std::string_view'?
    string_view str, boost::system::error_code& ec);
    ^~~~~~~~~~~
    std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:25:
In file included from /opt/local/include/boost/asio/ip/address_v6.hpp:335:
/opt/local/include/boost/asio/ip/impl/address_v6.ipp:309:28: error: unknown type name 'string_view'; did you mean 'std::string_view'?
address_v6 make_address_v6(string_view str)
                           ^~~~~~~~~~~
                           std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:25:
In file included from /opt/local/include/boost/asio/ip/address_v6.hpp:335:
/opt/local/include/boost/asio/ip/impl/address_v6.ipp:314:28: error: unknown type name 'string_view'; did you mean 'std::string_view'?
address_v6 make_address_v6(string_view str,
                           ^~~~~~~~~~~
                           std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
/opt/local/include/boost/asio/ip/address.hpp:218:38: error: unknown type name 'string_view'; did you mean 'std::string_view'?
BOOST_ASIO_DECL address make_address(string_view str);
                                     ^~~~~~~~~~~
                                     std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
/opt/local/include/boost/asio/ip/address.hpp:226:5: error: unknown type name 'string_view'; did you mean 'std::string_view'?
    string_view str, boost::system::error_code& ec);
    ^~~~~~~~~~~
    std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:259:
/opt/local/include/boost/asio/ip/impl/address.ipp:140:22: error: unknown type name 'string_view'; did you mean 'std::string_view'?
address make_address(string_view str)
                     ^~~~~~~~~~~
                     std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:71:
In file included from /opt/local/include/boost/asio/ip/address.hpp:259:
/opt/local/include/boost/asio/ip/impl/address.ipp:145:22: error: unknown type name 'string_view'; did you mean 'std::string_view'?
address make_address(string_view str,
                     ^~~~~~~~~~~
                     std::string_view
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/sqlite3/sqlite3.cpp:11:
/opt/local/include/sqlite3.h:246:9: warning: struct 'sqlite3' was previously declared as a class [-Wmismatched-tags]
typedef struct sqlite3 sqlite3;
        ^
build/include/botan/sqlite3.h:13:7: note: previous use is here
class sqlite3;
      ^
In file included from src/lib/utils/sqlite3/sqlite3.cpp:11:
/opt/local/include/sqlite3.h:3498:9: warning: struct 'sqlite3_stmt' was previously declared as a class [-Wmismatched-tags]
typedef struct sqlite3_stmt sqlite3_stmt;
        ^
build/include/botan/sqlite3.h:14:7: note: previous use is here
class sqlite3_stmt;
      ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:80:
In file included from /opt/local/include/boost/asio/ip/basic_resolver.hpp:27:
In file included from /opt/local/include/boost/asio/ip/basic_resolver_iterator.hpp:27:
/opt/local/include/boost/asio/ip/basic_resolver_entry.hpp:54:7: error: no type named 'string_view' in namespace 'boost::asio'; did you mean 'std::string_view'?
      BOOST_ASIO_STRING_VIEW_PARAM host, BOOST_ASIO_STRING_VIEW_PARAM service)
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
      std::string_view
/opt/local/include/boost/asio/detail/string_view.hpp:42:39: note: expanded from macro 'BOOST_ASIO_STRING_VIEW_PARAM'
# define BOOST_ASIO_STRING_VIEW_PARAM boost::asio::string_view
                                      ^~~~~~~~~~~~~
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
In file included from src/lib/utils/socket/socket.cpp:20:
In file included from /opt/local/include/boost/asio.hpp:80:
In file included from /opt/local/include/boost/asio/ip/basic_resolver.hpp:27:
In file included from /opt/local/include/boost/asio/ip/basic_resolver_iterator.hpp:27:
/opt/local/include/boost/asio/ip/basic_resolver_entry.hpp:54:42: error: no type named 'string_view' in namespace 'boost::asio'; did you mean 'std::string_view'?
      BOOST_ASIO_STRING_VIEW_PARAM host, BOOST_ASIO_STRING_VIEW_PARAM service)
                                         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                         std::string_view
/opt/local/include/boost/asio/detail/string_view.hpp:42:39: note: expanded from macro 'BOOST_ASIO_STRING_VIEW_PARAM'
# define BOOST_ASIO_STRING_VIEW_PARAM boost::asio::string_view
                                      ^~~~~~~~~~~~~
/opt/local/libexec/llvm-7.0/include/c++/v1/string_view:770:37: note: 'std::string_view' declared here
typedef basic_string_view<char>     string_view;
                                    ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.

We could not simply drop -march=native because we lost the higher ISAs.

So, on one hand, some features on the native computer are not detected/treated properly. On the other hand, some people want to build Crypto++ on one computer, and then deploy it to other that may have different CPU features (I fail to see how improving detection on the build machine is going to help them).

IMHO, the solution is - to allow the builder (not the makefile) to explicitly set what CPU features exactly to compile in. I should be able to compile for AVX2 on an older box, if I choose so. Similarly, I should be able to compile without AVX2 on the latest box, if my target environment is old.

[noloader]

Unfortunately, your workaround fixes one thing but breaks another - it makes it impossible to compile Botan with Boost...

Well, I'm not sure what is happening with Botan or Boost. Botan seems to build OK with the following and pass its self tests:

$ ./configure.py --cc-bin=/opt/local/bin/g++-mp-6 --cc-abi-flags="-DNDEBUG -g2 -O3 -Wa,-q"

The problems you seem to be having are with Boost. You should file a bug report with Boost to get it fixed.

I don't mind trying to play well with other libraries, but keep in mind that neither library is our responsibility. But I also fail to see how we are not playing well with others. The best I can tell we are in our own sandbox and there's no one to play with.

---

We could not simply drop -march=native because we lost the higher ISAs.

So, on one hand, some features on the native computer are not detected/treated properly.

Yeah, that was 5.6.x and earlier. At 6.0 we changed to BASE+SIMD and -march=native was no longer necessary. It fixed the distro use case. But you can still add it to CXXFLAGS if you want.

In fact I recommend using -march=native if it is available (especially on AVX machines where those fat memcpy's are available). I like -march=native so much it is a makefile target. You can issue make native and it adds the flags if available.

skylake:cryptopp$ make native
g++ -DNDEBUG -g2 -O3 -fPIC -pthread -pipe -march=native -c cryptlib.cpp
g++ -DNDEBUG -g2 -O3 -fPIC -pthread -pipe -march=native -c cpu.cpp
g++ -DNDEBUG -g2 -O3 -fPIC -pthread -pipe -march=native -c integer.cpp

---

IMHO, the solution is - to allow the builder (not the makefile) to explicitly set what CPU features exactly to compile in. I should be able to compile for AVX2 on an older box, if I choose so. Similarly, I should be able to compile without AVX2 on the latest box, if my target environment is old.

Yes, absolutely. You have complete control over CXXFLAGS. We honor whatever the user puts on the command line.

If you want AVX2 on a machine with a compiler that supports it, then you don't have to do anything. Or, you can manually add -mavx2 if it makes you feel better. If you don't want AVX2 when available then you can add -mno-avx2. These are standard GCC options. There's no mischief going on.

You don't seem to have any problems exercising CXXFLAGS. Where are you having trouble?

[mouse07410]

You don't seem to have any problems exercising CXXFLAGS. Where are you having trouble?

I'm having trouble getting Macports GCC (which is the only usable GCC on MacOS, because Xcode GCC isn't worth much) to pass the architecture/CPU flags to the native (Xcode) Integrated Assembler.

My solution is adding architecture-specific flags to Macports GCC invocation if detected. I think that since it's properly guarded (aka done only for Apple MacOS Macports GCC), it won't affect those who complained about, e.g., -march=native in the past.

I would also prefer to revert to my prior way of dealing with the flags, when for each discovered feature, the appropriate -mXXX flag was passed to the GCC, together with `-Wa,-mXXX if it was Macports.

In fact I recommend using -march=native if it is available...

The problem is, as I said above, that between Macports GCC and Integrated Assembler, something gets lost, so it has to be accompanied by -Wa,-march=native only for that combination.

[noloader]

I would also prefer to revert to my prior way of dealing with the flags, when for each discovered feature, the appropriate -mXXX flag was passed to the GCC, together with `-Wa,-mXXX if it was Macports.

Yeah, I have never quite understood this. Maybe its an Apple Clang thing.

I don't know how to write a for-each in GNUmake to convert each -mXXX into a -Wa,-mXXX. There is a for-each function at 8.5 The foreach Function. There is also 8 Functions for Transforming Text.

---

The problem is, as I said above, that between Macports GCC and Integrated Assembler, something gets lost, so it has to be accompanied by -Wa,-march=native only for that combination.

This is what I don't really understand. There's a buggy toolchain out there that only you use. You literally add 17 compiler options but you are moaning about adding number 18.

I reduced it to a minimal test case for you, I filed the LLVM bug report for you, I checked-in the test program for you. If you want it then you need to finish it. You need to write the feature test and add the option if necessary.

Otherwise, file a bug report with Apple and wait for them to push a fix to their devs.

[mouse07410]

There's a buggy toolchain out there that only you use.

I wouldn't call Macports GCC or Xcode Clang something "that only I use". Though I admit that it's rare when one needs to compile something with the modern GCC (therefore Macports instead of Xcode), but that compiled package contains assembly code that requires using Integrated Assembler from Xcode (because GAS hasn't been maintained for more than a decade).

Yes, there's something wrong with how the two toolchains interact.

However, it is easy for each of them to point finger at the other one (probably harder for Macports GCC, but still, unless I can demonstrate that it's GCC that doesn't pass something, rather than Clang not receiving something passed to it, I've no evidence to fix the blame to one of the two).

...moaning about adding number 18...

The problem is - my "17 compiler options" work on all of my compilers, from Xcode and Macports. To address this issue by adding 18th option, I'd break all the other compilers to save Macports GCC build of Crypto++ (because Macports GCC is the only compiler that needs the extra option -Wa,-march=native, and in my case only for compiling Crypto++).

You need to write the feature test and add the option if necessary

That's not a bad idea.

P.S. Assembly does get tricky - FYI: newhopecrypto/newhope#1

[pcordes]

When I go to the Intrinsic Guide it says vmovdqu64 is AVX512VL + AVX512F. I think that means or.

vmovdqu64 (%rdi), %zmm0 requires only AVX512F, and thus is supported on KNL for example.

vmovdqu64 (%rdi), %xmm0 or ymm0 requires AVX512F + AVX512VL.

---

AVX512-anything implies AVX512F. F = foundation, and includes support for the EVEX encoding that all AVX512 instructions use.

VL = Vector Length - 128 and 256-bit versions of any other AVX512 instruction (using an EVEX encoding, not AVX1/2 VEX, for instructions like vmovups where exactly the same instruction is encodeable with either).

Enabling AVX512VL implies enabling AVX512F.

[noloader]

We'll revisit if a testing machine becomes available.