Implement breakpoint disassembly support for Intel APX #114120

BruceForstall · 2025-04-01T20:34:05Z

Teach the amd64 breakpoint disassembler about APX, specifically the REX2 and extended EVEX encodings.

Update the tools to work with newer versions of gcc/gdb, such as handling new gdb output format in the parsing regular expressions.

Due to these newer versions, there are differences in the non-APX tables, apparently due to gcc/gdb bug fixes and improvements (e.g., supporting instructions previously unsupported).

Note that the APX code is untested due to lack of APX hardware. Also, the Windows SDK CONTEXT record does not define APX extended GPR (eGPR) registers yet, so accessing those registers is disabled.

The tables were generated using the following versions of gcc/gdb on Ubuntu 24.04.2 LTS, in WSL2:

gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git

Details:

Change the "createOpcodes.cpp" tool to generate more varieties of possible instructions, to include encoding forms for REX2 and extended EVEX.
Change createOpcodes to always generate 16 bytes of codes. Previously, the parser looked for a "58" followed by "59 pop" to indicate the end of an instruction sequence. This failed in various cases. Note that the longest legal x86 instruction sequence is 15 bytes, as defined by the architecture.
Update the parser and table generation tool (Amd64InstructionTableGenerator.cs) to be able to parse REX2 and extended EVEX instructions, and generate a new EVEX table for EVEX map 4.
The parser was updated to handle new gdb disassembly formats, such as different whitespace usage (spaces versus tabs), and using the "BCST" tag to indicate EVEX embedded broadcast.
The native walker was updated to understand the new tables, including when to use them (thus, it needs to recognize REX2 and extended EVEX formats).
Fixed bugs in existing AVX-512 (EVEX) handling of b, L'L, and pp bits: they were being read from the wrong prefix byte.
There seem to be a couple existing bugs in NativeWalker::Decode which I annotated but did not feel confident fixing: a. the loop to read and process instruction prefixes only reads a single prefix. Thus, a case like 0x66 (operand size) followed by 0x40 (REX) improperly assumes the REX byte is the instruction opcode. b. if the instruction opcode (after the prefix) is 0xcc, DebuggerController::GetPatchedOpcode() is called to read the actual opcode, but it uses the wrong address to do so.

Contributes to #112588

dotnet-policy-service · 2025-04-01T20:34:36Z

Tagging subscribers to this area: @tommcdon
See info in area-owners.md if you want to be subscribed.

Copilot

Pull Request Overview

This PR implements support for APX breakpoint disassembly by extending the decoder to recognize new instruction encoding forms such as REX2 and extended EVEX (map Evex_4). It also updates regular expressions for modern gcc/gdb disassembly output and revises instruction sample parsing and opcode table generation.

Added new encoding flags and map (Evex_4) for extended EVEX.
Updated regex patterns and sample parsing logic to accommodate different disassembly formats.
Modified debug logging conditions and updated opcode handling for REX2 and EVEX instructions.

Files not reviewed (2)

src/coreclr/debug/ee/amd64/gen_amd64InstrDecode/createOpcodes.cpp: Language not supported
src/coreclr/debug/ee/amd64/walker.cpp: Language not supported

Copilot · 2025-04-01T20:34:40Z

src/coreclr/debug/ee/amd64/gen_amd64InstrDecode/Amd64InstructionTableGenerator.cs

@@ -1035,7 +1116,7 @@ private void AddOpCode(Map map, int opCodeExt, bool reg, int modrmReg, string ru
            else
            {
                string oldstring = null;
-                if (Debug.debug)
+                if (true) // Debug.debug


The unconditional debug logging in AddOpCode may lead to excessive log output in production; consider using a conditional check (e.g., Debug.debug) or removing debug statements for production builds.

Suggested change

if (true) // Debug.debug

if (Debug.debug)

BruceForstall · 2025-04-01T20:34:50Z

@tommcdon PTAL
cc @anthonycanino @DeepakRajendrakumaran @Ruihan-Yin

BruceForstall · 2025-04-01T20:37:12Z

Note on testing: I have done no testing on this. I don't expect to be able to do any testing on the APX impact of this change for quite some time, until we have more available APX hardware. I think we should do whatever testing is appropriate to ensure this doesn't regress (and in fact, improves) existing x64 scenarios. Note that there are some bug fixes here for the existing AVX-512 support.

tommcdon · 2025-04-03T16:46:40Z

@BruceForstall Thanks for the work on debugger support for Intel APX! Please feel free to reach out to me offline if any assistance is needed with validation.

tommcdon

LGTM. It's a big change and so we should do some validation.

tommcdon · 2025-04-03T16:34:26Z

src/coreclr/debug/ee/amd64/walker.cpp


    BYTE prefix = *ip;
    if (prefix == 0xcc)
    {
-        prefix = (BYTE)DebuggerController::GetPatchedOpcode(m_ip);
+        prefix = (BYTE)DebuggerController::GetPatchedOpcode(m_ip); // REVIEW: change `m_ip` to `ip`?


I don't think it is critical, but in general since we have a const BYTE *ip = m_ip defined, it makes sense to switch to ip.

tommcdon · 2025-04-03T16:42:51Z

src/coreclr/debug/ee/amd64/walker.cpp

            ip++;
+            // REVIEW: it looks like a bug that we don't loop here looking for additional


Very good observation! Please feel free to make a second PR changing to while(true) - it looks non-intentional that we would break out of the loop here.

anthonycanino · 2025-04-03T19:55:36Z

Note on testing: I have done no testing on this. I don't expect to be able to do any testing on the APX impact of this change for quite some time, until we have more available APX hardware. I think we should do whatever testing is appropriate to ensure this doesn't regress (and in fact, improves) existing x64 scenarios. Note that there are some bug fixes here for the existing AVX-512 support.

Thanks for this work Bruce.

Regarding validation, what is typically done for the disassembler? Perhaps we can help evaluate.

Teach the amd64 breakpoint disassembler about APX, specifically the REX2 and extended EVEX encodings. Update the tools to work with newer versions of gcc/gdb, such as handling new gdb output format in the parsing regular expressions. Due to these newer versions, there are differences in the non-APX tables, apparently due to gcc/gdb bug fixes and improvements (e.g., supporting instructions previously unsupported). Note that the APX code is untested due to lack of APX hardware. Also, the Windows SDK CONTEXT record does not define APX extended GPR (eGPR) registers yet, so accessing those registers is disabled. The tables were generated using the following versions of gcc/gdb on Ubuntu 24.04.2 LTS, in WSL2: ``` gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0 GNU gdb (Ubuntu 15.0.50.20240403-0ubuntu1) 15.0.50.20240403-git ``` Details: 1. Change the "createOpcodes.cpp" tool to generate more varieties of possible instructions, to include encoding forms for REX2 and extended EVEX. 2. Change createOpcodes to always generate 16 bytes of codes. Previously, the parser looked for a "58" followed by "59 pop" to indicate the end of an instruction sequence. This failed in various cases. Note that the longest legal x86 instruction sequence is 15 bytes, as defined by the architecture. 3. Update the parser and table generation tool (Amd64InstructionTableGenerator.cs) to be able to parse REX2 and extended EVEX instructions, and generate a new EVEX table for EVEX map 4. 4. The parser was updated to handle new gdb disassembly formats, such as different whitespace usage (spaces versus tabs), and using the "BCST" tag to indicate EVEX embedded broadcast. 5. The native walker was updated to understand the new tables, including when to use them (thus, it needs to recognize REX2 and extended EVEX formats). 6. Fixed bugs in existing AVX-512 (EVEX) handling of `b`, `L'L`, and `pp` bits: they were being read from the wrong prefix byte. 7. There seem to be a couple existing bugs in `NativeWalker::Decode` which I annotated but did not feel confident fixing: a. the loop to read and process instruction prefixes only reads a single prefix. Thus, a case like 0x66 (operand size) followed by 0x40 (REX) improperly assumes the REX byte is the instruction opcode. b. if the instruction opcode (after the prefix) is 0xcc, `DebuggerController::GetPatchedOpcode()` is called to read the actual opcode, but it uses the wrong address to do so.

BruceForstall · 2025-04-28T20:40:24Z

/ba-g unrelated failures

BruceForstall added area-Diagnostics-coreclr apx labels Apr 1, 2025

BruceForstall requested review from Copilot and tommcdon April 1, 2025 20:34

dotnet-policy-service bot assigned BruceForstall Apr 1, 2025

Copilot AI reviewed Apr 1, 2025

View reviewed changes

This was referenced Apr 1, 2025

System.Net.Quic tests timeout #107761

Open

System.TimeoutException : The operation has timed out. dotnet/dnceng#5279

Open

System.Net.Requests test timeout #113883

Closed

tommcdon requested a review from a team April 3, 2025 16:44

tommcdon approved these changes Apr 3, 2025

View reviewed changes

BruceForstall force-pushed the APXBreakpoints branch from 8116809 to 9761ad2 Compare April 18, 2025 05:34

build-analysis bot mentioned this pull request Apr 18, 2025

The Operation will be canceled. The next steps may not contain expected logs. dotnet/dnceng#3008

Open

3 tasks

BruceForstall mentioned this pull request Apr 18, 2025

Intel architecture improvements for .NET 10 #108869

Open

46 tasks

BruceForstall merged commit 29e3af6 into dotnet:main Apr 28, 2025
88 of 93 checks passed

BruceForstall deleted the APXBreakpoints branch April 28, 2025 20:42

github-actions bot locked and limited conversation to collaborators May 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement breakpoint disassembly support for Intel APX #114120

Implement breakpoint disassembly support for Intel APX #114120

Uh oh!

BruceForstall commented Apr 1, 2025

Uh oh!

dotnet-policy-service bot commented Apr 1, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 1, 2025

Uh oh!

BruceForstall commented Apr 1, 2025

Uh oh!

BruceForstall commented Apr 1, 2025

Uh oh!

tommcdon commented Apr 3, 2025

Uh oh!

tommcdon left a comment

Uh oh!

tommcdon Apr 3, 2025

Uh oh!

tommcdon Apr 3, 2025

Uh oh!

anthonycanino commented Apr 3, 2025

Uh oh!

BruceForstall commented Apr 28, 2025

Uh oh!

Uh oh!

Uh oh!

		ip++;
		// REVIEW: it looks like a bug that we don't loop here looking for additional

Implement breakpoint disassembly support for Intel APX #114120

Implement breakpoint disassembly support for Intel APX #114120

Uh oh!

Conversation

BruceForstall commented Apr 1, 2025

Uh oh!

dotnet-policy-service bot commented Apr 1, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Apr 1, 2025

Choose a reason for hiding this comment

Uh oh!

BruceForstall commented Apr 1, 2025

Uh oh!

BruceForstall commented Apr 1, 2025

Uh oh!

tommcdon commented Apr 3, 2025

Uh oh!

tommcdon left a comment

Choose a reason for hiding this comment

Uh oh!

tommcdon Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

tommcdon Apr 3, 2025

Choose a reason for hiding this comment

Uh oh!

anthonycanino commented Apr 3, 2025

Uh oh!

BruceForstall commented Apr 28, 2025

Uh oh!

Uh oh!

Uh oh!