Ported pcie-brcmstb bounce buffer implementation to ARM64.

[yaroslavros]

This enables full 4G RAM usage on Raspberry Pi in 64-bit mode.
This addresses #3093

[pelwell]

Thank you. Do you have a "Signed-off-by: name " I can add for you?

[yaroslavros]

Added a commit d1e0a4c with "Signed-off-by"

Sorry about missing this

[yaroslavros]

I left a diagnostic trace message in arch/arm64/mm/dma-mapping.c
This is rectified by 6e7a2b8

[pelwell]

Thanks - that should all squash down when it is time to merge.

[agherzan]

@yaroslavros all we need now is a squash to one commit so we can merge it. Do you think you can do that soon?

[pelwell]

No need - I can squash and merge if you've had a chance to test it.

[agherzan]

I see. I'll give it a spin tomorrow. Will update as soon as.

[sakaki-]

Very nice - seems to work well (in the very basic testing I have done anyhow)!

I have added this PR to my bcm2711-kernel-bis weekly autobuild pro tem, until it is merged upstream. A prebuilt release tarball may be downloaded here.

Screenshot (64-bit Gentoo userland, also has #3063 applied for V3D):

Thanks, sakaki

[pY4x3g]

I tested the current kernel with the bounce buffer implementation on arm64. After the ram went over 1GB most of the programs are not usable anymore, network broke down and I was not able to shutdown the system at all. More or less the same as before the merge. I see errors in dmesg like in issue #3120 is this to be expected with a 32bit userspace? Sorry for asking, I dont know much about kernels.

Edit: after using the kernel from sakaki the problem is not present anymore. Maybe I compiled the kernel wrong... (used the introduction from https://andrei.gherzan.ro/linux/raspbian-rpi-64/)

[agherzan]

@pY4x3g How have you tested this? I just ran some stress tools and I was able to go above 3G without any obvious issues.

root@raspberrypi4-64:~# free -m
              total        used        free      shared  buff/cache   available
Mem:        3780536     3269616      447608       10784       63312      468548
Swap:             0           0           0

This is on a full 64bit distro (kernel + userspace).

[pelwell]

It's all looking good here, with just arm_64bit=1 in config.txt.

[pY4x3g]

@agherzan I started a self compiled go program (siacoin - arm64), after 1 GB of memory was used the system was not usable anymore. I actually do not know what I did wrong, but with the compiled kernel as mentioned above all is working fine now with usage at full 4GB (-300M for GPU) memory and swapping of 2 GB. System is running stable since yesterday. Using the official 32bit raspbian userspace.

[sakaki-]

@agherzan, see @pY4x3g's edit to the original message:

Edit: after using the kernel from sakaki the problem is not present anymore. Maybe I compiled the kernel wrong... (used the introduction from https://andrei.gherzan.ro/linux/raspbian-rpi-64/)

So I guess just a localized build issue originally?

[sakaki-]

Ooops - our posts just crossed @pY4x3g, apologies ^-^

[agherzan]

Gotcha. Thanks for clarifying. Looks good!