Multiple disassembler errors on vector instructions

[UnlimitedChild]

DisasmBugs.zip

[mappzor]

This is actually more related to Zydis rather than x64dbg directly nevertheless it's an interesting case. By looking at syntax used in your example code I assume you were using MASM64 to compile that. The problem is that MASM is quite permissive and allows you to specify things like zeroing-mask in places where you should not be allowed to do so, generating invalid instructions as a result.

If you look up VMOVAPS in Intel's manual you can find that following encoding is allowed:
EVEX.512.0F.W0 29 /r VMOVAPS zmm2/m512 {k1}{z}, zmm1
However above line actually describes two different cases: "register-copy" form when first operand is zmm register and "store-form" which is what you are using (first operand is memory). If you look at pseudocode provided in the manual you can notice that zeroing-mask is actually allowed ONLY for register-copy form but not for store-form.

Therefore according to official documentation those instructions

VMOVAPS zmmword ptr [4 * r14 + r14 + 123456h]{k1}{z} , zmm31
VMOVAPS zmmword ptr [4 * r14 + r14 + 123456h]{k2}{z} , zmm30
VMOVAPS zmmword ptr [4 * r14 + r14 + 123456h]{k3}{z} , zmm29
VMOVAPS zmmword ptr [4 * r14 + r14 + 123456h]{k4}{z} , zmm28
VMOVAPS zmmword ptr [4 * r14 + r14 + 123456h]{k5}{z} , zmm27

are all INVALID, so Zydis handles them correctly by failing to recognize them.

Were you able to test how those instructions behave on real hardware? Unfortunately I don't have any CPU with AVX-512 support to check this. I would expect that they either cause #UD or silently ignore EVEX.z. In latter case it would mean that there are valid yet undocumented alternative encodings not only for VMOVAPS but for several other AVX-512 instructions with similar semantics.