Skip to content

stage1_protector_fault

Jump to bottom
runner edited this page Oct 5, 2026 · 14 revisions

Stage 1: The EndfieldBase.dll Protector Fault

Status: ✅ Solved (2026-07-14) — kept as the investigation record. The fault is fixed by the 0F 1F NOP-skip in patches/stage1-macos/; the outcome and final patch set are in 13-working-solution.md. Read on for how the root cause was found.

The Problem

Before the fix, Arknights: Endfield would fail to launch on macOS under CrossOver with a characteristic failure signature: the game would enter an infinite exception-handling loop inside EndfieldBase.dll (the VMProtect/TenProtect "tpshell" protector), eventually causing a stack overflow and process termination. This occurred before the ACE anti-cheat even loaded.

Root Cause

The root cause was a Rosetta 2 bug in how it handles certain x86_64 instructions:

  1. The instruction: 0F 1F C1 — a multi-byte NOP (nop ecx)
  2. The fault: Rosetta 2 raises an EXCEPTION_ILLEGAL_INSTRUCTION (c000001d) on this plain NOP
  3. The loop: Wine's SEH handler processes it, but during unwind, control jumps to an unmapped address 0x6CD268, causing a collided-unwind loop that exhausts the stack (~273 iterations)

The Exception Sequence

EndfieldBase.dll (tpshell) runs anti-debug probes
  → triggers EXCEPTION_ILLEGAL_INSTRUCTION on 0F 1F NOP
  → SEH handler catches it, tries to unwind
  → jumps to constant address 0x6CD268 (unmapped on macOS)
  → c0000005 EXECUTE fault
  → handler re-enters itself (collided unwind)
  → repeats ~273 times until stack overflow
  → thread dies, game relaunches (anti-tamper watchdog)
```javascript

## The Fix

The fix was added to `dlls/ntdll/unix/signal_x86_64.c` in the `handle_cet_nop` function:

```c
case 0x1F:
    // Decode multi-byte NOP length (mod/rm, SIB, displacement)
    // and advance RIP past it. These NOPs have zero side effects.
    ...
```python

**Why this works:** VMProtect emits `0F 1F` NOPs pervasively (23,227+ per launch). By decoding the instruction length and skipping it, we avoid the illegal-instruction fault entirely. The NOP has no side effects, so skipping it is semantically exact.

## Prior Art

This exact failure class was documented but **never solved publicly**:

- **WineHQ Bug 45083** — A VMProtect-3.x 64-bit app (MetaTrader 5) that runs fine on Linux Wine but faults on macOS Wine with the identical exception family: recursive `c0000005` + `RtlUnwindEx` + "setup_exception stack overflow." Status: NEW, unfixed.
- **Bug 34254** — Same terminal signature, different root cause.
- **vinegar #382** — Identical pattern for Roblox/Hyperion protector.

## Investigation & Experiments

### Capturing the Failure Signature

To reproduce and verify:

```bash

# Use the capture script (handles CrossOver's Perl wrapper correctly)

./scripts/01

Clone this wiki locally