Support for High DDR Memory?

[yu-zou]

Hi,

ZynqMP has two memory regions, low memory and high memory. E.g. on ZCU106, two memory regions exist:

It seems like udmabuf always allocate dma buffer in low memory. But that size is limited compared to the high memory.
Is there any support to allocate a dma buffer in high DDR memory?

Thank you.

[ikwzm]

Thank you for the issue

I haven't confirmed it directly, but I've received reports that that issue #45 has resolved a similar issue. please refer.

[nejc-rosenstein-isystem]

Hello @ikwzm , we are using high DDR memory (0x80000000 and up) on our project (Zyng US+, Petalinux kernel ver. 6.1.30). Reading the memory from user space works fine (using mmap), but we are having trouble with manual cache management - we consistently run into kernel panic whenever we write '1' into either sync_for_cpu or sync_for_device, regardless of size/offset parameters values.

We checked conformance with documentation (e.g. dma-coherent is 0, dma-mask is <64>, quirk_mmap is enabled ...).
Is manual cache management supposed to work in such cases? Thanks.

[ikwzm]

Thank you for the issue.

Please wait a moment for us to investigate.

[ikwzm]

I verified it here with kv260 using the following device tree, and it did not cause a kernel panic.

        reserved-memory {
                #address-cells = <2>;
                #size-cells = <2>;
                ranges;
                image_buf0: image_buf@0 {
                        compatible = "shared-dma-pool";
                        reusable;
                        reg = <0x8 0x00000000 0x0 0x00400000>;
                        label = "image_buf0";
                };
        };
        amba_pl: amba_pl@0 {
                #address-cells = <2>;
                #size-cells = <2>;
                compatible = "simple-bus";
                ranges ;
                udmabuf@0 {
                        compatible  = "ikwzm,u-dma-buf";
                        device-name = "udmabuf0";
                        size = <0x0 0x00400000>;
                        dma-mask = <64>;
                        memory-region = <&image_buf0>;
                };
        };

The log (excerpt) at boot time is as follows

[    0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd034]
[    0.000000] Linux version 6.1.70-zynqmp-fpga-trial (ichiro@Jabberwock) (aarch64-linux-gnu-gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, GNU ld (GNU Binutils for Ubuntu) 2.34) #3 SMP Wed Feb  7 15:30:51 JST 2024
[    0.000000] Machine model: ZynqMP SMK-K26 Rev1/B/A
[    0.000000] efi: UEFI not found.
[    0.000000] Reserved memory: created CMA memory pool at 0x0000000800000000, size 4 MiB
[    0.000000] OF: reserved mem: initialized node image_buf@0, compatible id shared-dma-pool
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000000000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000087fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x000000007fefffff]
[    0.000000]   node   0: [mem 0x0000000800000000-0x000000087fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x000000087fffffff]
[    0.000000] On node 0, zone Normal: 256 pages in unavailable ranges
[    0.000000] cma: Reserved 1000 MiB at 0x0000000041600000
    :
    :
[    7.465992] u_dma_buf: loading out-of-tree module taints kernel.
[    7.467165] u-dma-buf amba_pl@0:udmabuf@0: assigned reserved memory node image_buf@0
[    7.468707] u-dma-buf udmabuf0: driver version = 4.5.2
[    7.468722] u-dma-buf udmabuf0: major number   = 239
[    7.468728] u-dma-buf udmabuf0: minor number   = 0
[    7.468733] u-dma-buf udmabuf0: phys address   = 0x0000000800000000
[    7.468740] u-dma-buf udmabuf0: buffer size    = 4194304
[    7.468746] u-dma-buf amba_pl@0:udmabuf@0: driver installed.
    :

shell# cat /sys/class/u-dma-buf/udmabuf0/sync_offset
0x0
shell# cat /sys/class/u-dma-buf/udmabuf0/sync_size
4194304
shell# cat /sys/class/u-dma-buf/udmabuf0/sync_direction
0
shell# echo 1 > /sys/class/u-dma-buf/udmabuf0/sync_for_cpu

This verification was performed in the following environment (The target is KV260).

https://github.com/ikwzm/ZynqMP-FPGA-Ubuntu22.04-Desktop/tree/v3.0.0