Influxdb 3 Core/Enterprise 'illegal instruction' on x86_64 CPU with CPUs older than Intel Haswell

[jdstrand]

https://github.com/influxdata/influxdb/blob/main/.cargo/config.toml#L21 lists:

[target.x86_64-unknown-linux-gnu]
rustflags = [
    ...
    # Enable all features supported by CPUs more recent than haswell (2013)
    "-C", "target-cpu=haswell",

I have an old test machine (Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz) with up to date Ubuntu 18.04 LTS, but despite having proper glibc compatibility, it fails to run:

$ ./influxdb3 -V
Illegal instruction (core dumped)

$ gdb ./influxdb3
...
Program received signal SIGILL, Illegal instruction.
(gdb) run -V

(gdb) x/i $pc
=> 0x555558233124 <_ZN12clap_builder5error14Error$LT$F$GT$5print17hcabd755f43c3aba1E+8484>:	shrx   %r12d,%eax,%r12d

Googling tells me that the shrx instruction was introduced with the Intel BMI2 (Bit Manipulation Instruction Set 2) extension, which was first supported in Intel's Haswell microarchitecture, released in 2013.

It appears that this was introduced because we wanted the avx2 instruction, first introduced in haswell: https://github.com/influxdata/influxdb_iox/pull/2119 states "I think that the current setting for ROARING_ARCH of x86-64 may not be specific enough to ensure that things are built with the avx2 instructions. Ivy Bridge Haswell has those so I think that this should work better." https://github.com/influxdata/influxdb_iox/pull/2140 has additional information.

It's apparently quite intentional that we don't support CPUs older than haswell. We probably need a documentation note for this. I expect this issue to be closed but wanted to file it for posterity since others will likely run into this.

cc @pauldix, @peterbarnett03, @alamb (since you wrote https://github.com/influxdata/influxdb_iox/pull/2140), @sanderson and @jstirnaman

References:

• https://github.com/influxdata/influxdb_iox/pull/2119
• https://github.com/influxdata/influxdb_iox/pull/2140

[pauldix]

I don't think there's a need so support CPUs older than that. However, if this is due to some dependency on croaring, I don't think we use that anymore. So I'm not sure why we'd still have this limitation. It would take some investigation or someone that knows more about this part than me.

[jdstrand]

From my reading, it seems that we were wanting to support avx2 specifically which is only available in Haswell and later (and croaring just happened to be part of those changes, but not having to do with the desire for avx2 and needing Haswell).

[alamb]

From my reading, it seems that we were wanting to support avx2 specifically which is only available in Haswell and later (and croaring just happened to be part of those changes, but not having to do with the desire for avx2 and needing Haswell).

Yes I think the idea is to let the rust compiler / auto vectorizer take advantage of avx2 SIMD instructions which are significantly faster for many of the lower level arrow kernels

If we want to support older hardware without slowing down for more modern hardware, the standard options I know of are

1. Make a special build targeting older instructions sets (easy for us, less easy for customers)
2. Compile code for multiple instructions sets and then pick the appropriate code version at runtime based on the available hardware

I suggest we go with 1 at first unless there is significant demand for something eaiser

[pauldix]

Ok, I don't think we need to even bother with it at this point. We'll wait until someone comes around asking for the special build before we produce it. Shockingly, anything before 2013 is now long ago, lost to the sands of time. Onwards and upwards into our SIMD accelerated future utopia!

[jdstrand]

Shockingly, anything before 2013 is now long ago, lost to the sands of time

/me gazes at his old test machine... is that a mirage? ;)

(just riffing on the comment; not suggesting we do anything different for my test machine!)

[alamb]

Very meta