Enable AVX2 by default

[dktapps]

Currently our prebuilt *nix binaries use the vanilla x86_64 architecture, which does not permit using stuff like AVX2 (however, SSE2 and other extensions are explicitly enabled on Linux).

The Windows binaries are built using the default cl.exe flags, which uses SSE2 as a baseline, also disabling AVX2.

AVX2 has been on every Intel processor since Haswell, and every Ryzen processor. This means that we can probably get away with distributing prebuilts with AVX2 enabled, which would offer performance advantages for stuff like libdeflate, and possibly stuff like xxhash and libmorton too.

(Binaries built using compile.sh without the -t flag will automatically use -march=native, so anyone building their own binaries automatically gets the best performance.)

[SOF3]

I am concerned that running on a wrong CPU would cause very unpredictable undefined behavior silently. Or are we confident that machines that do not support AVX2 will generate garbage output before it actually runs something in production to cause some damage?

What about we provide AVX2 binaries separately first and detect that we can use these binaries in install.sh?

[dktapps]

Attempted usage of AVX2 instructions will cause an immediate crash with SIGILL since the host processor won't recognize the instruction.

In any case, as it turns out, some stuff such as libdeflate already compiles AVX2 code anyway, and uses cpuid() to switch the used function at runtime depending on what features the processor supports, so this isn't as big of a deal as I expected.

[SOF3]

Attempted usage of AVX2 instructions will cause an immediate crash with SIGILL since the host processor won't recognize the instruction.

This is not exactly true. Other CPU models may interpret the same instruction code as something else and treat it differently.

[dktapps]

I hadn't considered that, but since we're dealing with only two x86_64 vendors with known histories (AMD and Intel), I think that's probably not a concern.

[SOF3]

I am not very familiar with various SIMD instruction sets, but it appears that AVX2 is already common enough. It is probably still a good idea for users to compile their own binaries to get the best performance though.

[dktapps]

I think currently most people aren't aware that they are sacrificing performance by using prebuilt binaries instead of compiling their own, especially since installing a prebuilt is recommended by the docs, and is also done automatically by installer.sh.

[BlackCatDevel0per]

(Binaries built using compile.sh without the -t flag will automatically use -march=native, so anyone building their own binaries automatically gets the best performance.)

Recently I've added this flag to my fork) I think I'll upload it this week..

[dktapps]

-march=native makes binaries non-portable. There's a reason we don't use it for the prebuilts.

[kazan417]

I think if someone could add detecting support of avx2 in runtime of already builded binary than enabling avx2 in prebuils will be good idea.