Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AVX-512 handling breaks DynamoRIO execution on Windows with full AVX-512 support #3949

Closed
fmoessbauer opened this issue Nov 19, 2019 · 7 comments · Fixed by #3954
Closed

Comments

@fmoessbauer
Copy link
Contributor

Describe the bug
Execution of any application under DynamoRIO is impossible in the following scenario:

Note: This behavior appears with and w/o client and is deterministic.

To Reproduce
Steps to reproduce the behavior:

  1. Take Windows machine with AVX-512 (full) support (No VM), check flag avx512_enabled = true in proc.c.
  2. Execute any Program like notepad.exe
    3.1 In release builds, DR immediately exits
    3.2 In debug builds, assertion fires: rip-relative reference out of 32-bit reach

Access Violation, as reported by WinDBG

(3aa8.3a44): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
dynamorio!encode_cti+0x3b [inlined in dynamorio!instr_encode_arch+0x102]:
00000000`7107e042 8803            mov     byte ptr [rbx],al ds:00000000`00000000=??
0:000> kb
 # RetAddr           : Args to Child                                                           : Call Site
00 (Inline Function) : --------`-------- --------`-------- --------`-------- --------`-------- : dynamorio!encode_cti+0x3b [c:\users\felix\source\repos\dynamorio\core\arch\x86\encode.c @ 2598] 
01 00000000`7107b13a : 00000000`00000000 00000000`71070c00 00000237`69b0e780 00000000`00000000 : dynamorio!instr_encode_arch+0x102 [c:\users\felix\source\repos\dynamorio\core\arch\x86\encode.c @ 2785] 
02 (Inline Function) : --------`-------- --------`-------- --------`-------- --------`-------- : dynamorio!instr_encode_to_copy+0x16 [c:\users\felix\source\repos\dynamorio\core\arch\encode_shared.c @ 132] 
03 00000000`710b0a51 : 00000000`00000000 00000237`69b0e780 00000000`00000000 00000000`00000002 : dynamorio!instr_encode+0x1a [c:\users\felix\source\repos\dynamorio\core\arch\encode_shared.c @ 140] 
04 00000000`710b1eaa : 00000000`00000007 00000237`69b35b30 00000000`710b32e0 00000000`00000000 : dynamorio!emit_intercept_code+0x1eb1 [c:\users\felix\source\repos\dynamorio\core\win32\callback.c @ 1328] 
05 00000000`710adaf9 : 00000000`7118427f 00000237`69ac1a01 00000000`710b32e0 00000000`00000000 : dynamorio!intercept_call+0x26a [c:\users\felix\source\repos\dynamorio\core\win32\callback.c @ 1743] 
06 00000000`7100594a : 00000000`7118c4b8 00000000`00000218 00000000`00000000 00000000`00000000 : dynamorio!callback_interception_init_finish+0x59 [c:\users\felix\source\repos\dynamorio\core\win32\callback.c @ 7747] 
07 00000000`7109b40c : 000000a1`0510f700 00000000`00000000 00000000`00000000 00000237`67ea0031 : dynamorio!dynamorio_app_init+0x23a [c:\users\felix\source\repos\dynamorio\core\dynamo.c @ 692] 
08 00000000`710d28e0 : 00007ff9`1c1d9770 00000000`710d28d0 00000000`00000000 00000000`00000000 : dynamorio!auto_setup+0x1c [c:\users\felix\source\repos\dynamorio\core\arch\x86_code.c @ 169] 
09 00007ff9`1c1d9770 : 00000000`710d28d0 00000000`00000000 00000000`00000000 00000000`00000000 : dynamorio!dynamo_auto_start+0x10 [C:\Users\felix\CMakeBuilds\1ac36d74-80db-f93a-9dba-3c80b8e1e38c\build\x64-Release\core\x86.asm_core.s @ 1599] 
0a 00000000`710d28d0 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!GetProcAddressStub
0b 00000000`00000000 : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : dynamorio!dynamo_auto_start [C:\Users\felix\CMakeBuilds\1ac36d74-80db-f93a-9dba-3c80b8e1e38c\build\x64-Release\core\x86.asm_core.s @ 1591] 

Debug Output with loglevel 3

log.0.2376.html.txt

Versions

  • All versions newer than (including) cronbuild-7.91.18109
  • Prior Versions are not affected
  • When manually disabling AVX512 support by setting avx512_enabled = false (in proc.c). Dynamorio is executed correctly

cc @TomSie, @jones718

@derekbruening
Copy link
Contributor

May have been hit in #3923 also.

@derekbruening
Copy link
Contributor

derekbruening commented Nov 19, 2019

Unfortunately we do not have much Windows automated testing capacity. Help is wanted augmenting that.

@TomSie
Copy link
Contributor

TomSie commented Nov 19, 2019

What exactly would you need?

@derekbruening
Copy link
Contributor

Cycles somewhere on Windows with AVX-512 hooked up to Jenkins to run the test suite on pull requests would be ideal. Today we have Appveyor for CI but at least its default does not have AVX-512.

hgreving2304 pushed a commit that referenced this issue Nov 19, 2019
…IX and 32-bit.

As pointed out in cf1ec32, AVX-512 context management by DynamoRIO is not
fully supported or untested on non-UNIX as well as on 32-bit builds. Yet processor
detection and certain AVX-512 functionality was enabled on those builds when running on
machines with AVX-512 support enabled by the OS. This was causing problems (xref #3949).
This patch completely disables all AVX-512 features on unsupported and untested builds.

Fixes #3949
Issue: #1312
@hgreving2304
Copy link

hgreving2304 commented Nov 19, 2019

Please read

All non-UNIX 64-bit builds are essentially either not tested or known to be unsupported yet. So far parts of DynamoRIO's code for AVX-512 were active even for those unsupported builds. We are disabling this temporarily in order to get a stable Windows build on AVX-512 machines as pointed out in this issue.

Contributions to get full support in Windows would be greatly appreciated!

hgreving2304 pushed a commit that referenced this issue Nov 19, 2019
…IX and 32-bit. (#3954)

As pointed out in cf1ec32, AVX-512 context management by DynamoRIO is not
fully supported or is untested on non-UNIX as well as on 32-bit builds. Yet processor
detection and certain AVX-512 functionality was enabled on those builds when running on
machines with AVX-512 support enabled by the OS. This was causing problems (xref #3949).
For now, this patch completely disables all AVX-512 features on unsupported and untested
builds.

Fixes #3949
Issue: #1312
@derekbruening
Copy link
Contributor

Thank you for reporting the bug. It should be fixed in the cronbuild at https://github.com/DynamoRIO/dynamorio/releases/tag/cronbuild-7.91.18219

@fmoessbauer
Copy link
Contributor Author

Thanks for the quick fix. I can confirm that the latest cronbuild works on my AVX512 system (with and without client).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants