Buffer size limit?

[njsys]

Hi
When I allocate a udma buffer upto 4MB, I have no issues.

But when I allocated a udma buffer of 8MB...

insmod udmabuf.ko udmabuf0=8388608

and then ran

ls -la /dev/udmabuf0

I get a message : ls: cannot access /dev/udmabuf0: No Such file or directory

How can I fix this?
Thanks

dmesg output:

[ 5050.855332] CPU: 1 PID: 5502 Comm: insmod Tainted: G W OE ------------ 3.10.0-693.el7.x86_64 #1
[ 5050.855333] Hardware name: innotek GmbH VirtualBox/VirtualBox, BIOS VirtualBox 12/01/2006
[ 5050.855334] ffff8800a057b990 000000002964dae8 ffff8800a057b940 ffffffff816a3d91
[ 5050.855337] ffff8800a057b980 ffffffff810879c8 000002782964dae8 ffffffffc06b8340
[ 5050.855339] ffff8801175c5400 ffff8800a0796410 ffff8801175c5400 ffff8800a0570780
[ 5050.855342] Call Trace:
[ 5050.855346] [] dump_stack+0x19/0x1b
[ 5050.855349] [] __warn+0xd8/0x100
[ 5050.855351] [] warn_slowpath_fmt+0x5f/0x80
[ 5050.855354] [] device_create_file+0x8d/0xa0
[ 5050.855356] [] device_add+0x5fb/0x7c0
[ 5050.855359] [] device_create_groups_vargs+0xe0/0x110
[ 5050.855361] [] device_create+0x51/0x70
[ 5050.855364] [] ? udmabuf_platform_driver_probe+0x1b5/0x7e0 [udmabuf]
[ 5050.855367] [] udmabuf_platform_driver_probe+0x5dd/0x7e0 [udmabuf]
[ 5050.855370] [] platform_drv_probe+0x42/0x110
[ 5050.855372] [] driver_probe_device+0xc2/0x3e0
[ 5050.855375] [] ? driver_probe_device+0x3e0/0x3e0
[ 5050.855377] [] __device_attach+0x3b/0x40
[ 5050.855379] [] bus_for_each_drv+0x6b/0xb0
[ 5050.855382] [] device_attach+0x90/0xb0
[ 5050.855384] [] bus_probe_device+0x98/0xc0
[ 5050.855386] [] device_add+0x4ff/0x7c0
[ 5050.855388] [] platform_device_add+0xd1/0x2d0
[ 5050.855391] [] udmabuf_static_device_create+0xbc/0xe9 [udmabuf]
[ 5050.855394] [] ? 0xffffffffc06bafff
[ 5050.855397] [] udmabuf_module_init+0x272/0x1000 [udmabuf]
[ 5050.855400] [] do_one_initcall+0xb8/0x230
[ 5050.855403] [] load_module+0x1f64/0x29e0
[ 5050.855406] [] ? ddebug_proc_write+0xf0/0xf0
[ 5050.855409] [] ? copy_module_from_fd.isra.42+0x53/0x150
[ 5050.855411] [] SyS_finit_module+0xa6/0xd0
[ 5050.855414] [] system_call_fastpath+0x16/0x1b
[ 5050.855416] ---[ end trace 8e98ba62cbd19414 ]---
[ 5050.855523] dma_alloc_coherent() failed. return(0)
[ 5050.855526] udmabuf udmabuf.0: driver setup failed. return=-12
[ 5050.855554] udmabuf udmabuf.0: driver installed.
[ 5050.855557] udmabuf: probe of udmabuf.0 failed with error -12

[ikwzm]

Thank you for the issue.

Please tell me how the cma area is secured. When you boot Linux Kernel, you will see the following message first.

Starting kernel ...

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.0-xlnx-v2018.2-zynqmp-fpga (ichiro@sphinx-vm-ubuntu) (gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9)) #1 SMP Fri Aug 10 11:05:02 JST 2018
[    0.000000] Boot CPU: AArch64 Processor [410fd034]
[    0.000000] Machine model: ZynqMP Ultra96
[    0.000000] efi: Getting EFI parameters from FDT:
[    0.000000] efi: UEFI not found.
[    0.000000] cma: Reserved 256 MiB at 0x0000000070000000

udmabuf uses dma_alloc_coherent () to allocate memory.
dma_alloc_coherent () allocates memory from kernel space when there is no CMA.

[ikwzm]

Please tell me in addition.

In what kind of environment are you running udmabuf?
What is the version of udmabuf?

Looking at dmsg, it looks like Linux Kernel 3.10, could you build udmabuf?

[njsys]

Hi
Yes. I tried building and using udmabuf on 2 computers.
Both had kernel version 3.10.
udmabuf version is 1.4.3

I do NOT see this message on both computers:
[ 0.000000] cma: Reserved 256 MiB at 0x0000000070000000

So there is no cma on both. In this situation, how do I use udmabuf for 16MB?
What are my options?

Thanks

[ikwzm]

Thank you for your reply.

Is the architecture ARM? Is it ARM64? Is it x86?

Since udmabuf is made on the assumption of ARM/ARM64 + FPGA + Linux 4.x, it may be difficult to run otherwise.

I don't know in your environment, but if you specify cma=128M for Linux bootargs, 128MB can be secured in the CMA area.

[njsys]

Architecture is x86_64

3.10.0-693.el7.x86_64

[ikwzm]

Oh, I'm sorry.
I don't have x86 + Linux 3.10 anymore.
Older versions of udmabuf may work.

[njsys]

Will it work for 16MB buffer if we upgrade Linux kernel to 4.1?
Thanks

[ikwzm]

Will it work for 16MB buffer if we upgrade Linux kernel to 4.1?

Regardless of the Linux Kernel version, I do not know how to secure the CMA area in your environment.
I was able to help you if I could prepare the same environment as you, but I've thrown it away.

[njsys]

Thanks. Closing Issue.

[njsys]

I did upgrade to version 4.4 and dmesg shows i have cma reserved at 256mb.

But when i try to insmod udmabuf for 8mb, it fails.

Dmesg shows dma_alloc_coherant failed returned 0. Driver setup failed. Return -12.

Any idead how i can make it work?

[njsys]

I had to modify your source code for dma_mask_bit at a few places to 64 from 32. Then i was able to create udma buf more than 4mb

[ikwzm]

Thank you for the issue.

I also confirmed that dma_alloc_coherent fails. The dmesg here is as follows.

[35208596.491643] udmabuf udmabuf.0: swiotlb buffer is full (sz: 8388608 bytes)
[35208596.491653] swiotlb: coherent allocation failed for device udmabuf.0 size=8388608
[35208596.491669] CPU: 0 PID: 26184 Comm: insmod Tainted: G           O    4.9.0-7-amd64 #1 Debian 4.9.110-1
[35208596.491674] Hardware name:  /NUC5CPYB, BIOS PYBSWCEL.86A.0057.2016.1012.1134 10/12/2016
[35208596.491684]  0000000000000000 ffffffffb3530644 ffffffffffffffff 0000000000000000
[35208596.491705]  ffffffffb355adb1 0000000bb68fabc0 ffff98f0b68fabc0 ffff98f0b68fab00
[35208596.491722]  ffff98f25fb75410 ffffffffb3e24460 ffff98f25fb75400 ffff98f0b68fabc0
[35208596.491740] Call Trace:
[35208596.491766]  [<ffffffffb3530644>] ? dump_stack+0x5c/0x78
[35208596.491784]  [<ffffffffb355adb1>] ? swiotlb_alloc_coherent+0x131/0x150
[35208596.491811]  [<ffffffffc07e5400>] ? udmabuf_platform_driver_probe+0x450/0x800 [udmabuf]
[35208596.491827]  [<ffffffffb3681b98>] ? platform_drv_probe+0x38/0x90
[35208596.491842]  [<ffffffffb367be56>] ? devices_kset_move_last+0x46/0x90
[35208596.491853]  [<ffffffffb367fb73>] ? driver_probe_device+0x223/0x430
[35208596.491864]  [<ffffffffb367fe60>] ? __driver_attach+0xe0/0xe0
[35208596.491874]  [<ffffffffb367d774>] ? bus_for_each_drv+0x64/0xb0
[35208596.491884]  [<ffffffffb367f829>] ? __device_attach+0xd9/0x150
[35208596.491894]  [<ffffffffb367eaba>] ? bus_probe_device+0x8a/0xa0
[35208596.491906]  [<ffffffffb367c721>] ? device_add+0x3c1/0x610
[35208596.491919]  [<ffffffffb36818ee>] ? platform_device_add+0x10e/0x260
[35208596.491938]  [<ffffffffc07e5bea>] ? udmabuf_static_device_create+0x11a/0x165 [udmabuf]
[35208596.491958]  [<ffffffffc07ea27f>] ? udmabuf_module_init+0x27f/0x1000 [udmabuf]
[35208596.491967]  [<ffffffffc07ea000>] ? 0xffffffffc07ea000
[35208596.491980]  [<ffffffffb320218e>] ? do_one_initcall+0x4e/0x180
[35208596.491995]  [<ffffffffb33c915d>] ? __vunmap+0x6d/0xc0
[35208596.492010]  [<ffffffffb337f291>] ? do_init_module+0x5b/0x1f6
[35208596.492022]  [<ffffffffb3302c1a>] ? load_module+0x25ba/0x2ac0
[35208596.492033]  [<ffffffffb32ff340>] ? __symbol_put+0x60/0x60
[35208596.492049]  [<ffffffffb3303366>] ? SYSC_finit_module+0xc6/0xf0
[35208596.492062]  [<ffffffffb3203b7d>] ? do_syscall_64+0x8d/0xf0
[35208596.492075]  [<ffffffffb3814bce>] ? entry_SYSCALL_64_after_swapgs+0x58/0xc6
[35208596.492083] dma_alloc_coherent() failed. return(0)
[35208596.492345] udmabuf udmabuf.0: driver setup failed. return=-12
[35208596.492920] udmabuf udmabuf.0: driver installed.
[35208596.492941] udmabuf: probe of udmabuf.0 failed with error -12

Is the content of your dmesg the same?

[njsys]

Yes

[ikwzm]

I had to modify your source code for dma_mask_bit at a few places to 64 from 32. Then i was able to create udma buf more than 4mb

Please tell me where you changed.

Thanks

[jamescamdenmartin]

I ran into the same issue on an x86_64 and it was fixed by changing the dma_mask_bit module parameter like so:
insmod udmabuf.ko dma_mask_bit=64 udmabuf0=8388608

[bipingamiyv]

I ran into the same issue on an x86_64 and it was fixed by changing the dma_mask_bit module parameter like so: insmod udmabuf.ko dma_mask_bit=64 udmabuf0=8388608

I tried the above command, but it fails to create size more than 4MiB. Can you suggest any other alternative?