Upstream Linux patch for _DMA constraints, to support booting with 4GB RAM

[andreiw]

The USB controller on Pi 4 can only do DMA to 3GB. ACPI has a way of describing this, but Linux's support for inheriting _DMA descriptor constraints is limited to PCI devices only.

There's a simple patch for 5.5 - https://gist.github.com/andreiw/f2d27fdf5ed18f4af0db95a380dc36af

NetBSD already supports 4GB.

[Tungsten842]

Does this patch work with raspberry 4.19 kernel? In that case can you send a pull request to: https://github.com/raspberrypi/linux

[pbatard]

This is not intended for the custom 4.19 Linux kernel that the Raspberry Foundation uses for Raspbian, which is what you are pointing to. IIRC, they are using bounce buffers there to work around the same issue.

Instead, we need something that can apply to official Linux kernels.

Be very mindful that the Linux tree you see at https://github.com/raspberrypi/linux is very custom and intended only for use with Raspbian only. Therefore, because it's not official, that kernel will never be used by Debian, Ubuntu, Fedora and so on, which are the distros we are targeting for use with the UEFI firmware. Therefore, sending patches to the Raspberry Foundation's version of the kernel will achieve nothing...

[andreiw]

Yeah to rephrase somewhat, looking for someone (ideally working today upstream with AArch64 Linux day to day) to upstream this functionality, either as this direct patch or equivalent behavior.

[Tungsten842]

@lategoodbye

[lategoodbye]

I think @vianpl and Jeremy Linton are the more appropriate persons for this discussion. Furthermore Linux development happends on mailing lists and not on Github.

linux-acpi@vger.kernel.org
linux-arm-kernel@lists.infradead.org (subscription required)
linux-rpi-kernel@lists.infradead.org (subscription required)

[andreiw]

That’s a fair comment, yet this Github issue exists specifically so folks know this is a TBD item for proper ACPI support on Pi or similar platforms. Not to coordinate actual development. The expectation is that development would be done in the appropriate manner (engaging the devs and maintainers on appropriate lists).

[andreiw]

Again, it’s not really a discussion, just a way to broadcast that this is code that needs to be upstreamed, as none of the current Pi UEFI developers (including Jeremy) have cycles to dedicate to Linux support.

As per https://rpi4-uefi.dev, we have a channel on the Arm Discord server where we are open 24/7 to a discussion.

[ardbiesheuvel]

https://lore.kernel.org/linux-acpi/20200420092753.9819-1-ardb@kernel.org/

I was wondering if we should replace the workaround that limits memory to 3 GB with an IORT that limits the DMA range of the XHCI to 30 bits. This works with Linux even if it ignores _DMA, without losing ~1 GB of memory on 4 GB units.

[andreiw]

I think we added that w/a only for Linux, so if the special IORT accomplishes something similar, sounds good to me.

[ardbiesheuvel]

Actually, that would still only work for Linux versions that implement ZONE_DMA, which was added in v5.5, and is less likely to be backported than the fix for _DMA handling, which itself depends on ZONE_DMA as well. (ZONE_DMA is a 1 GB region at the start of memory to serve heap allocations that will be used for DMA, and without that, Linux only distinguishes between <4 GB and >4 GB, which is not sufficient for us)

[andreiw]

There's another aspect here, too... with this particular fix, I was able to get mass storage to work, yet other devices (HID, network, etc) did not - failing to perform control transfers, likely because they were still using addresses that did not come from the DMA zone. Taking a wild guess that the storage subsystem can deal with some bounce buffering, but the others - not so much.

[ehem]

I'm unsure whether it is of interest, but this is also a challenge for Linux running as Xen Dom0. This set has been proposed for that situation:
https://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git/log/?h=fix-rpi4-v3

[ehem]

https://lore.kernel.org/linux-acpi/20200420092753.9819-1-ardb@kernel.org/

I was wondering if we should replace the workaround that limits memory to 3 GB with an IORT that limits the DMA range of the XHCI to 30 bits. This works with Linux even if it ignores _DMA, without losing ~1 GB of memory on 4 GB units.

With this patch, plus @sstabellini's fix-rpi4-v3 (git://git.kernel.org/pub/scm/linux/kernel/git/sstabellini/xen.git) patches on top of Debian's 5.6 kernel source I was able to change the "Limit RAM to 3GB" setting to "disabled" and the kernel successfully booted with near 4GB of memory available.

Behavior of the kernel is troublesome in this setting, but the kernel did boot and was responsive to network packets. A USB keyboard which was connect does NOT work (wasn't even visible to lsusb), but I was able to confirm operation via SSH (network chip works now! huzzah!).

[samerhaj]

The patch is now in 5.8 kernel:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/acpi/arm64/iort.c?h=v5.8&id=bcf876870b95592b52519ed4aafcf9d95999bc9c#n1125

[jlinton]

And enough of the dma pool/cma/etc fixes are also in 5.8 that I think we can safely close this. There are some further cleanups coming down the pipe, and there is still a problem with zero copy DMA on the XHCI controller, but none of these are directly related to the core _DMA()/etc support.