Add kernel quirk 'FixAppleVTD'

[CaseySJ]

In macOS 13.3, WiFi and most Ethernet devices stop working when AppleVTD is enabled and system has more than 16GB memory. This affects motherboards that have one or more Reserved Memory Regions in their DMA Remap table (DMAR). The problem exists even if Reserved Memory Regions are removed and a substitute SSDT-DMAR.aml is injected.

This quirk side-steps the problem, allowing WiFi and Ethernet devices to function once again. It restores the behavior that existed in earlier versions of Ventura.

This quirk patches IOPCIFamily.kext.

---

I'm submitting this pull request a little early to allow more time for review and update. Some comments and questions:

• Does it make sense to introduce this as a kernel quirk in OpenCore?
• I've tested the quirk myself, but others are in the process of testing
• Configuration.tex has been updated, but not sure if other documentation files also need to be updated; does Differences.pdf get updated automatically?
• Both Sample.plist and SampleCustom.plist have been updated

---

A description of the patch is provided here.

[vit9696]

What is the issue with DisableIoMapper?

[CaseySJ]

What is the issue with DisableIoMapper?

Various Ethernet and Thunderbolt devices do not establish a connection in Big Sur and later when DisableIoMapper is turned on.

For example, Aquantia AQC107 / AQC113, Antelope Audio Thunderbolt audio interfaces, and Apple Thunderbolt to Gigabit Ethernet adapter, among others. These devices require AppleVTD, so we enable VT-d in BIOS and turn off DisableIoMapper. This enables AppleVTD and causes all devices to work properly, but a change introduced in macOS 13.3 caused these devices to fail (with and without AppleVTD).

These threads contain a number of posts about this problem:

https://www.insanelymac.com/forum/topic/356135-release-macos-133/?do=findComment&comment=2802842

https://www.tonymacx86.com/threads/gigabyte-z490-vision-d-thunderbolt-3-i5-10400-amd-rx-580.298642/post-2363929

https://www.tonymacx86.com/threads/success-gigabyte-designare-z390-thunderbolt-3-i7-9700k-amd-rx-580.316533/post-2365183

[vit9696]

Hmm, what in particular does this patch do? To me it feels like correcting the DMA table is the most efficient way to resolve this as patching the driver may break VT-d even more.

[CaseySJ]

Normally we would remove Reserved Memory Regions and create a modified SSDT-DMAR.aml. This worked perfectly until macOS 13.3, when Apple introduced a new AppleVTD::addMemoryRange function and called it from one place:IOPCIBridge::addBridgeMemoryRange.

In the screenshot below we see early 13.3 on left without call to AppleVTD::addMemoryRange and late 13.3 on the right. The patch skips the call to AppleVTD::addMemoryRange, which restores behavior of earlier 13.3 builds.

[PMheart]

Are you sure that this patch is only required on 13.3+? I recall some old discussions when adding the Aquantia patch: 2441455#commitcomment-70530145

[CaseySJ]

Are you sure that this patch is only required on 13.3+? I recall some old discussions when adding the Aquantia patch: 2441455#commitcomment-70530145

The discussion in that link started with user beefon who has an AMD system. Because VT-d is an Intel technology and AppleEthernetAquantiaAqtion.kext in Monterey and later requires AppleVTD, it means AQC107 and AQC113 do not work on AMD Ryzen platforms.

That conversation occurred almost exactly a year ago. At the time there was no way to enable AQC107 and AQC113 on AMD platforms, but that changed in December 2022 when I posted the following kext patches:
https://github.com/CaseySJ/Aquantia-macOS-Patches

With these patches we bypass AppleVTD in Monterey and Ventura, which allows AQC107 and AQC113 to function on AMD platforms, effectively reverting to Big Sur behavior. These patches also work on Intel platforms when AppleVTD is disabled.

However, AppleVTD affects more than just the Aquantia driver. It also affects Thunderbolt devices such as Antelope Audio interfaces and Apple's Thunderbolt-to-Gigabit Ethernet adapter (which is still in use because it supports Ethernet AVB [audio video bridging protocol]).

All of these devices were working properly on Intel platforms in macOS 13.3 beta builds 1 and 2 with AppleVTD enabled and no driver-specific patches applied. But in macOS 13.3 beta build 3 or 4, Apple made some small changes to IOPCIFamily. One of those changes broke all kexts that rely on AppleVTD -- but only if the motherboard's native DMAR table has one or more Reserved Memory Regions. Even if we remove those regions by using a substitute SSDT-DMAR the problem remains.

Motherboards such as Asus Z690 and Z790 do not have Reserved Memory Regions and are not affected by the change. But many Z370, Z390, Z490 (and maybe Z590) boards are affected.

In effect, a new problem was introduced in macOS 13.3 late beta, which this quirk addresses. The target audience for this quirk are users with:

• Z370, Z390, Z490 and possibly Z590 (and their Bxxx chipset counterparts)
• AppleVTD enabled and more than 16GB memory installed
• Aquantia 10GbE cards, various Thunderbolt devices
• Intel i225-V and i226-V Ethernet ports

Side Note: In Ventura, Apple removed AppleIntelI210Ethernet, which means Intel i225-V and i226-V devices are driven by DriverKit-AppleEthernetE1000, which requires AppleVTD. The alternative is disabling AppleVTD, adding boot argument e1000=0 and injecting AppleIntelI210Ethernet from Monterey.

[vit9696]

Sorry, but I repeat my question. What do these patches do? I am not keen into loading IDA and investigating it myself. Could you explain based on the IOPCIFamily source code?

[CaseySJ]

Sorry, but I repeat my question. What do these patches do? I am not keen into loading IDA and investigating it myself. Could you explain based on the IOPCIFamily source code?

Source code for 13.3 has not yet been posted, so the explanation below is based partly on 13.2 source code and partly on disassembled 13.3 x86 binary.

In 13.2 the function IOPCIBridge::addBridgeMemoryRange looks like this (from source code):

bool IOPCIBridge::addBridgeMemoryRange( IOPhysicalAddress start,
                                        IOPhysicalLength length, bool host )
{
    bool ok;

    // fix - ACPIPCI makes this up for hosts with zero space
    if ((0x80000000 == start) && (0x7f000000 == length))
        return (false);

    ok = IOPCIRangeListAddRange(&reserved->rangeLists[kIOPCIResourceTypeMemory],
                                kIOPCIResourceTypeMemory, 
                                start, length);
    return (ok);

}

But in 13.3 it looks like this (from x86 disassembly):

undefined8 IOPCIBridge::addBridgeMemoryRange(ulonglong param_1,ulonglong param_2,bool param_3)

{
  undefined8 uVar1;
  undefined7 in_register_00000011;
  ulonglong uVar2;
  
  uVar2 = CONCAT71(in_register_00000011,param_3);
  if ((param_2 == 0x80000000) && (uVar2 == 0x7f000000)) {
    return 0;
  }
  AppleVTD::addMemoryRange(param_2,uVar2);
  uVar1 = IOPCIRangeListAddRange(*(IOPCIRange ***)(param_1 + 0x98),0,param_2,uVar2,1);
  return uVar1;
}

We can see that the only noteworthy change is a newly added call to AppleVTD::addMemoryRange(..). This function does not exist in early 13.3 builds. It was first introduced in a late 13.3 beta build.

The code for addMemoryRange looks like this (from x86 disassembly):

// AppleVTD::x(unsigned long long, unsigned long long)

void AppleVTD::addMemoryRange(ulonglong param_1,ulonglong param_2)
{
  uint uVar1;
  long lVar2;
  int iVar3;
  long lVar4;
  char *pcVar5;
  
  pcVar5 = *(char **)PTR_gSystem_00031058;
  lVar4 = OSMetaClassBase::safeMetaCast((OSMetaClassBase *)pcVar5,(OSMetaClass *)&gMetaClass);
  if (lVar4 == 0) {
    pcVar5 = "[%s()] AppleVTD is not yet installed as gSystem\n";
  }
  else {
    if (*(long *)(lVar4 + 0x1d0) != 0x40) {
      *(ulonglong *)(lVar4 + 0x1d8 + *(long *)(lVar4 + 0x1d0) * 8) = param_1;
      *(ulonglong *)(lVar4 + 0x3d8 + *(long *)(lVar4 + 0x1d0) * 8) = param_2;
      if (*(long *)(lVar4 + 0x1a8) != 0) {
        lVar2 = *(long *)(lVar4 + 0x1d0);
        uVar1 = *(uint *)(lVar4 + 0x3d8 + lVar2 * 8);
        if ((_gIOPCIFlags & 4) != 0) {
          iVar3 = _ml_at_interrupt_context();
          if (iVar3 == 0) {
            pcVar5 = "[%s(%p)] Freeing 0x%llx->0x%llx\n";
            _IOLog("[%s(%p)] Freeing 0x%llx->0x%llx\n","addMemoryRange",
                   *(undefined8 *)(lVar4 + 0x1a8));
          }
        }
        if ((_gIOPCIFlags & 4) != 0) {
          pcVar5 = "[%s(%p)] Freeing 0x%llx->0x%llx\n";
          _kprintf("[%s(%p)] Freeing 0x%llx->0x%llx\n","addMemoryRange",
                   *(undefined8 *)(lVar4 + 0x1a8));
        }
        space_alloc_fixed((AppleVTD *)pcVar5,*(vtd_space **)(lVar4 + 0x1a8),
                          (uint)(*(ulong *)(lVar4 + 0x1d8 + lVar2 * 8) >> 0xc),uVar1 >> 0xc,false);
      }
      *(long *)(lVar4 + 0x1d0) = *(long *)(lVar4 + 0x1d0) + 1;
      return;
    }
    pcVar5 = "[%s()] Mem regions limit reached\n";
  }
  _IOLog(pcVar5,"addMemoryRange");
  return;
}

Although not a substitute for actual source code, this function first checks if AppleVTD is enabled, otherwise it exits. If AppleVTD is enabled, we can see there is a call to space_alloc_fixed(..). This function's source code exists in 13.2 and looks like this:

void 
AppleVTD::space_alloc_fixed(vtd_space_t * bf, vtd_baddr_t addr, vtd_baddr_t size)
{
	vtd_balloc_fixed(bf, addr, size);
	vtd_rballoc_fixed(bf, addr, size);
	vtd_space_fault(bf, addr, size);
}

We can see that it's part of the AppleVTD:: class, and it calls 3 sub-functions that seem to allocate fixed size memory chunks presumably for DMA remapping purposes.

Note that there is no freeMemoryRange counterpart to addMemoryRange. Memory chunks used for DMA remapping are presumably set up at boot time and remain in effect for as long as the system runs.

In early beta builds of 13.3 the addMemoryRange function did not exist. In fact, this function has never existed until now, so it does not seem to be critical. But this function causes Ethernet, WiFi and some Thunderbolt devices to fail when AppleVTD is enabled on motherboards whose native DMAR table contains at least one Reserved Memory Region, which many Z370, Z390, Z490, and presumably Z590 boards do.

By looking for differences between early and late beta builds of 13.3, we spotted this function and applied a patch to skip over it. In other words, we created a patch that specifically blocks out the call to addMemoryRange, resulting in this:

undefined8 IOPCIBridge::addBridgeMemoryRange(ulonglong param_1,ulonglong param_2,bool param_3)
{
  undefined8 uVar1;
  undefined7 in_register_00000011;
  ulonglong uVar2;
  
  uVar2 = CONCAT71(in_register_00000011,param_3);
  if ((param_2 == 0x80000000) && (uVar2 == 0x7f000000)) {
    return 0;
  }
  // skip over AppleVTD::addMemoryRange
  // AppleVTD::addMemoryRange(param_2,uVar2);
  uVar1 = IOPCIRangeListAddRange(*(IOPCIRange ***)(param_1 + 0x98),0,param_2,uVar2,1);
  return uVar1;
}

After applying the patch we found that all of our WiFi, Ethernet and Thunderbolt devices started to work properly once again. Skipping addMemoryRange simply restores the behavior that existed in earlier 13.3 beta builds, and we have not seen any adverse effect from the patch (which users may enable or disable at any time).

[vit9696]

Got it, thank you!

(1) Suggest to replace the patch with:

Base: __ZN8AppleVTD14addMemoryRangeEyy
Replace: <C3>

It will simply disable the function, which should be more reliable than finding the call to it.

(2) Suggest to rename the quirk to DisableIoMapperMapping, as it is literally a counterpart to DisableIoMapper, which instead of disabling it entirely just makes it do nothing (IoMapper is Apple generalised named to Intel VT-d and any other IOMMU, like DART on Apple Silicon).

(3) Rewrite the description to explain that this quirk disables mapping PCI bridge device memory in IOMMU (VT-d). Then you may add that this happens to resolve compatibility issues with Wi-Fi and such due to not trying to restrict access to them via VT-d. OpenCore Configuration Manual is not a guide after all.

[CaseySJ]

These changes have been made and tested on my Gigabyte Z490 Vision D and Gigabyte Z390 Designare.

[PMheart]

Thank you very much! Looks good to me now.

[vit9696]

Thanks! Merged)

[wuyanghua123]

华硕z390还是不行，

[CaseySJ]

华硕z390还是不行，

You might have a different problem. Please create a new thread in the link below and describe the problem in detail.

https://www.tonymacx86.com/forums/general-help.10/

[wuyanghua123]

华硕z390还是不行，>> 你可能有不同的问题。请在下面的链接中创建一个新线程，并详细描述问题。>> https://www.tonymacx86.com/forums/general-help.10/谢谢你的回复。它已提交。https://www.tonymacx86.com/threads/asus-z390-open-the-vt_d-wireless-card-and-it-wont-open.325424/

[JetpropelledSnake]

Nice patch!
My motherboard is GigaByte Z590 AROUS Master Version1.0 with a NIC AQC-107.
DisableIoMapperMapping is ON and disableIoMapper is OFF. VT-D is enabled in BIOS.

After reboot, blueteeth works fine but WIFI can't be enabled.
If any further info needed, pls. let me know.

[CaseySJ]

Nice patch!

My motherboard is GigaByte Z590 AROUS Master Version1.0 with a NIC AQC-107.

DisableIoMapperMapping is ON and disableIoMapper is OFF. VT-D is enabled in BIOS.

After reboot, blueteeth works fine but WIFI can't be enabled.

If any further info needed, pls. let me know.

Please see the discussion here:

acidanthera/bugtracker#2278 (comment)

We can follow up with you there.