-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Set DMA coherent pool size to 2MB #3040
Conversation
This is needed for UAS attached storage (mkfs -V -t ext4 -b 4096 -m 0 -E lazy_itable_init=0,lazy_journal_init=0) and also for some DVB dongles. Details: https://forum.armbian.com/topic/4811-uas-mainline-kernel-coherent-pool-memory-size/ (in case you'll ever release a 64-bit kernel the same change is needed at arch/arm64/mm/dma-mapping.c)
See: #2924 The parameter can be altered by a cmdline.txt setting. Increasing the reservation as a #define reduces the available memory for other users, so the preference is to increase it on a case-by-case basis. |
The downstream dts files currently set coherent_pool to 1m for all Pis except Pi 4, which is an anomaly. |
The coherent_pool requirement was a function of plugging many devices into dwc_otg - but if UAS on Pi4 can hit this limit as well then I don't see why Pi4 shouldn't follow suit. |
It will. Increasing to 1M is already sufficient for this use case (UAS). BTW: In case you didn't notice. The new VL805 firmware trashes IOPS at small block sizes (not an issue with HDD but users with databases on SSD should be aware). |
We did notice, and are currently trying to identify the triggers and failure mechanism. Note that on the drive I've managed to reproduce the issue on (it doesn't affect all of them), the throughput reduction is caused by an occasional bus reset rather than a general slowing down. |
@pelwell thank you for the additional info, I might repeat my USB storage torture test then (simulating power losses while writing to two UAS connected SSDs). As a reference
Tons of these last messages following. Closing in the hope of an adoption of the RPi 4 DT. |
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
That's an interesting datapoint. With a 1M coherent_pool I still see the bounce buffer warnings (N.B. the XHCI driver uses bounce buffers internally - these are not the bounce buffers used by the PCIe driver to manage the fourth GB of a 4GB Pi). Setting it to 2M didn't make any obvious improvement, so I've pushed the 1M change. |
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: raspberrypi/linux#3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: raspberrypi/linux#3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org> Signed-off-by: Michael Scott <mike@foundries.io>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
Downstream Raspberry Pi dts files add "coherent_pool=1M" to the kernel command line to aid the dwc_otg driver, but this excluded Pi 4 which uses a new XCHI interface instead. UAS also benefits from a larger coherent_pool value, so replicate the addition in bcm2711-rpi-4-b.dts. See: #3040 Signed-off-by: Phil Elwell <phil@raspberrypi.org>
This is needed for UAS attached storage (mkfs -V -t ext4 -b 4096 -m 0 -E lazy_itable_init=0,lazy_journal_init=0) and also for some DVB dongles.
Details: https://forum.armbian.com/topic/4811-uas-mainline-kernel-coherent-pool-memory-size/ (in case you'll ever release a 64-bit kernel the same change is needed at arch/arm64/mm/dma-mapping.c)