Driver do not work in 64bit system with DMA limited to 32bits address #90
Description
Hi!
I had two custom platforms, one is based on ARM32(kernel 4.14) and the other with ARM64(kernel 4.19), i already tested the ATCWILC3000 in the ARM32 platform with success.
But when i try ATCWILC3000 with the ARM64, i got memory problems, when it generate the scatterlist during the CMD53, it do not take in a count the DMA restrictions of my platform. It try to do DMA with address bigger than 32bits, my hardware do not support DMA addressing bigger than 32bits.
I tried to create a buffer in "wilc_sdio_cmd53" function, where i can control the address to it with devm_kzalloc, by this way i can so far that the device works for almost a hour, but i steel get some problem after some time working, i get this;
Mar 30 11:52:01 localhost kernel: [ 2715.464842] [READ]temp_data = 4dcf0080 | cmd->buffer = 5a5b1da4 | *temp_data = 5a | *cmd->buffer = 5a Mar 30 11:52:01 localhost kernel: [ 2715.722595] ksdioirqd/mmc1: page allocation failure: order:5, mode:0x40dc0(GFP_KERNEL|__GFP_COMP|__GFP_ZERO), nodemask=(null) Mar 30 11:52:01 localhost kernel: [ 2715.722613] CPU: 3 PID: 815 Comm: ksdioirqd/mmc1 Not tainted 5.4.2 #3 Mar 30 11:52:01 localhost kernel: [ 2715.722615] Hardware name: Caninos Labrador 7 (DT) Mar 30 11:52:01 localhost kernel: [ 2715.722619] Call trace: Mar 30 11:52:01 localhost kernel: [ 2715.722632] dump_backtrace+0x0/0x140 Mar 30 11:52:01 localhost kernel: [ 2715.722637] show_stack+0x14/0x20 Mar 30 11:52:01 localhost kernel: [ 2715.722644] dump_stack+0xb0/0xd8 Mar 30 11:52:01 localhost kernel: [ 2715.722651] warn_alloc+0xf4/0x160 Mar 30 11:52:01 localhost kernel: [ 2715.722656] __alloc_pages_slowpath+0x8e0/0xad0 Mar 30 11:52:01 localhost kernel: [ 2715.722661] __alloc_pages_nodemask+0x228/0x270 Mar 30 11:52:01 localhost kernel: [ 2715.722667] kmalloc_order+0x24/0x70 Mar 30 11:52:01 localhost kernel: [ 2715.722671] kmalloc_order_trace+0x28/0xd0 Mar 30 11:52:01 localhost kernel: [ 2715.722676] __kmalloc_track_caller+0x208/0x290 Mar 30 11:52:01 localhost kernel: [ 2715.722683] devm_kmalloc+0x2c/0x98 Mar 30 11:52:01 localhost kernel: [ 2715.722712] wilc_sdio_cmd53.isra.6+0x48/0x1c0 [wilc_sdio] Mar 30 11:52:01 localhost kernel: [ 2715.722729] wilc_sdio_read+0x24c/0x2b0 [wilc_sdio] Mar 30 11:52:01 localhost kernel: [ 2715.722745] wilc_handle_isr+0x124/0x520 [wilc_sdio] Mar 30 11:52:01 localhost kernel: [ 2715.722760] wilc_sdio_interrupt+0x3c/0x70 [wilc_sdio] Mar 30 11:52:01 localhost kernel: [ 2715.722767] process_sdio_pending_irqs+0x5c/0x1a8 Mar 30 11:52:01 localhost kernel: [ 2715.722771] sdio_irq_thread+0x9c/0x198 Mar 30 11:52:01 localhost kernel: [ 2715.722775] kthread+0x124/0x128 Mar 30 11:52:01 localhost kernel: [ 2715.722781] ret_from_fork+0x10/0x1c Mar 30 11:52:01 localhost kernel: [ 2715.722785] Mem-Info: Mar 30 11:52:01 localhost kernel: [ 2715.722801] active_anon:147729 inactive_anon:27584 isolated_anon:27 Mar 30 11:52:01 localhost kernel: [ 2715.722801] active_file:3972 inactive_file:3908 isolated_file:0 Mar 30 11:52:01 localhost kernel: [ 2715.722801] unevictable:8 dirty:30 writeback:0 unstable:0 Mar 30 11:52:01 localhost kernel: [ 2715.722801] slab_reclaimable:5362 slab_unreclaimable:237903 Mar 30 11:52:01 localhost kernel: [ 2715.722801] mapped:23214 shmem:28262 pagetables:2149 bounce:0 Mar 30 11:52:01 localhost kernel: [ 2715.722801] free:60273 free_pcp:131 free_cma:10652 Mar 30 11:52:01 localhost kernel: [ 2715.722812] Node 0 active_anon:590916kB inactive_anon:110336kB active_file:15888kB inactive_file:15632kB unevictable:32kB isolated(anon):108kB isolated(file):0kB mapped:92856kB dirty:120kB writeback:0kB shmem:113048kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 186368kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no Mar 30 11:52:01 localhost kernel: [ 2715.722825] DMA32 free:241092kB min:22528kB low:28160kB high:33792kB active_anon:590536kB inactive_anon:110336kB active_file:16232kB inactive_file:15756kB unevictable:32kB writepending:120kB present:2031616kB managed:1976072kB mlocked:32kB kernel_stack:6080kB pagetables:8596kB bounce:0kB free_pcp:524kB local_pcp:524kB free_cma:42608kB Mar 30 11:52:01 localhost kernel: [ 2715.722826] lowmem_reserve[]: 0 0 0 Mar 30 11:52:01 localhost kernel: [ 2715.722832] DMA32: 13168*4kB (UME) 6437*8kB (UME) 3181*16kB (UMEC) 979*32kB (UMEHC) 191*64kB (UMHC) 0*128kB 0*256kB 1*512kB (C) 1*1024kB (C) 0*2048kB 10*4096kB (C) = 241112kB Mar 30 11:52:01 localhost kernel: [ 2715.722861] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=1048576kB Mar 30 11:52:01 localhost kernel: [ 2715.722864] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=32768kB Mar 30 11:52:01 localhost kernel: [ 2715.722868] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB Mar 30 11:52:01 localhost kernel: [ 2715.722872] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=64kB Mar 30 11:52:01 localhost kernel: [ 2715.722874] 36191 total pagecache pages Mar 30 11:52:01 localhost kernel: [ 2715.722877] 0 pages in swap cache Mar 30 11:52:01 localhost kernel: [ 2715.722880] Swap cache stats: add 0, delete 0, find 0/0 Mar 30 11:52:01 localhost kernel: [ 2715.722883] Free swap = 0kB Mar 30 11:52:01 localhost kernel: [ 2715.722884] Total swap = 0kB Mar 30 11:52:01 localhost kernel: [ 2715.722886] 507904 pages RAM Mar 30 11:52:01 localhost kernel: [ 2715.722888] 0 pages HighMem/MovableOnly Mar 30 11:52:01 localhost kernel: [ 2715.722890] 13886 pages reserved Mar 30 11:52:01 localhost kernel: [ 2715.722892] 16384 pages cma reserved Mar 30 11:52:01 localhost kernel: [ 2715.722894] 0 pages hwpoisoned Mar 30 11:52:01 localhost kernel: [ 2715.722899] fail to alloc temp data Mar 30 11:52:01 localhost kernel: [ 2715.722925] caninos-dma e0230000.dma-controller: Cannot do DMA to address 0x0000008000000000 Mar 30 11:52:01 localhost kernel: [ 2715.722932] caninos-dma e0230000.dma-controller: overflow 0x0000008000000000+180 of DMA mask ffffffff bus mask 0
my changes in the wilc_sdio_cmd53 are the following, i had add the dma restriction in probe function with;
dma_set_coherent_mask(wilc->dev, DMA_BIT_MASK(32));
`static int wilc_sdio_cmd53(struct wilc *wilc, struct sdio_cmd53 *cmd)
{
struct sdio_func *func = container_of(wilc->dev, struct sdio_func, dev);
int size, ret;
sdio_claim_host(func);
u8 *temp_data;
temp_data = devm_kzalloc(wilc->dev, cmd->count * cmd->block_size, GFP_KERNEL);
if(!temp_data){
pr_err("fail to alloc temp data");
}
func->num = cmd->function;
func->cur_blksize = cmd->block_size;
if (cmd->block_mode)
size = cmd->count * cmd->block_size;
else
size = cmd->count;
if (cmd->read_write) { /* write */
if (cmd->block_mode)
memcpy(temp_data, cmd->buffer, cmd->count * cmd->block_size);
else
memcpy(temp_data, cmd->buffer, cmd->count);
pr_info("[WRITE]temp_data = %x | cmd->buffer = %x | *temp_data = %x | *cmd->buffer = %x", temp_data,cmd->buffer, *temp_data,*cmd->buffer);
ret = sdio_memcpy_toio(func, cmd->address,
(void *)temp_data, size);
} else { /* read */
ret = sdio_memcpy_fromio(func, (void *)temp_data,
cmd->address, size);
if (cmd->block_mode)
memcpy(cmd->buffer, temp_data, cmd->count * cmd->block_size);
else
memcpy(cmd->buffer, temp_data, cmd->count);
pr_info("[READ]temp_data = %x | cmd->buffer = %x | *temp_data = %x | *cmd->buffer = %x", temp_data,cmd->buffer, *temp_data,*cmd->buffer);
}
sdio_release_host(func);
if (ret)
dev_err(&func->dev, "%s..failed, err(%d)\n", __func__, ret);
return ret;
}`
There are any better way to pass the DMA restriction for the ATCWIL3000's driver?
thanks!
best regards
Igor Ruschi