Support VEX and EVEX encodings

[abrown]

• What is the feature or code improvement you would like to do in Cranelift?

Add VEX and EVEX encoding mechanisms for x86 code generation.

• What is the value of adding this in Cranelift?

Certain Wasm SIMD operations (e.g. convert, trunc) only can be lowered with VEX and EVEX encodings. Many VEX and EVEX encodings are only available in later SIMD feature sets (e.g. AVX-512); adding the ability to use VEX and EVEX encodings does not preclude legalizations for x86 CPUs without later SIMD feature sets, but that is a separate issue.

• Do you have an implementation plan, and/or ideas for data structures or algorithms to use?

I think the relevant thing here is that it may impact how we go about resolving #1141.

• Have you considered alternative implementations? If so, how are they better or worse than your proposal?

Not having the ability to use VEX and EVEX encodings would force cranelift to use multi-instruction legalizations instead of optimal instructions during instruction selection.

[bnjbvr]

We'll definitely support VEX and EVEX at some point.

For what it's worth, we added support for those in Spidermonkey a while ago but this resulted in hard to track down slowdowns. I seem to recall it was related to the fact that the VEX chip was slow to start up, such that the cost of starting it up would show up in profiles and sometimes regress performance, if the VEX instructions weren't used a lot. @sunfishcode might have more insights.

In the meanwhile, it seems fine to have multi instruction legalizations for the wasm opcodes without VEX, since we'd ideally would like to support SSE2 eventually...

[sunfishcode]

The Bugzilla bug is https://bugzilla.mozilla.org/show_bug.cgi?id=1124751. Some OS's cause code using VEX prefixes to run significantly slower, so we'll need to determine which OS's and versions are ok, and test for them at runtime.

[bjorn3]

Using this will be behind a has_avx flag, right? If so disabling it in Spidermonkey should prevent the hard to track down slowdowns, while still giving other users the ability to use this.

[abrown]

@sunfishcode, we talked about this in IRC a while back and I haven't tried to replicate the issue yet. In fact, I think it would be good to implement the VEX/EVEX encodings so that we can try to find the OS's that cause slowdowns. The way we would use this initially (in truncate/convert) would be along the lines with what you observed on Linux so I would hope we would not see any regressions:

On Linux [... ] it's 0 [AVX-to-SSE transitions] regardless of whether we use the optimization that opportunistically mixes legacy SSE encodings with AVX-128 instructions

Might be good to expand on what is benchmarked with cargo bench; I just ran it on cranelift:

running 2 tests
test bugpoint::tests::test_reduce ... ignored
test run::test::nop ... ignored

test result: ok. 0 passed; 0 failed; 2 ignored; 0 measured; 0 filtered out

[bjorn3]

I believe this is handled with the new framework backend.

[abrown]

It should be handled but I don't think it is yet... at least not that I know of. I haven't ported the VEX/EVEX encoding stuff over from the old backend.

[cfallin]

This is all covered now in the current x64 backend, and we have VEX encodings for many vector instructions -- closing now!