ILI9341 hangs after a few seconds

[Adesin-fr]

Hi

I'm using fbtft_ili9340 (for adafruit22a-like device) on an Orange Pi Pc, armbian last release (3.4.112-sun8i ).

I can get a display. This is OK.

I've tried some things :

• Play a movie thru mplayer : hangs after a few frames.
• cat /dev/urandom >/dev/fb8 : works a few times, then no more display updates. dmesg doesn't show anymore the "fbtft debugging" info.
• made a custom application using SDL. Works and hangs after a few seconds.

I did tried another power supply, same result.
The board itself doesn't hang (still able to ssh into it), but when it freeze, I cannot rmmod module.

How can I help debugging this ?

[notro]

Try fbcon and just let the cursor blink to see if it lasts longer (small transfer length): con2fbmap 2 8.
You can also try and lower txbuflen if your spi hw doesn't support 16 bits per word. If the spi hw can only do 8 bit, then a buffer is used to swap the bytes (little endian).

[Adesin-fr]

Hi

Thanks for your answer !

Tried con2fbmap, worked for more than 20 minutes without hanging.
I've tried different things with no success :
txbuflen values 2048, 1024, 512, 256, 128, -1 . Same result each time, when I try to refresh whole screen, it hangs after a few times.
tried a lower spi clock (32Mhz, 16Mhz), no more success.

regards

[notro]

I was about to suggest disabling DMA, but that is already done in the orangepi source: https://github.com/orangepi-xunlong/linux-sunxi/blob/orangepi-3.4/drivers/video/fbtft/fbtft-core.c

Verify this by ensuring that DMA is gone from the driver load message:
dmesg | grep graphics

Further debugging requires building a kernel and is beyond what I can assist you with here.

[Adesin-fr]

Strange, I'm sure that DMA is enabled !
I'm going to make a test without it.

[Adesin-fr]

I've made a test with dma=0 (checked in dmesg).
Same thing, hangs after about 20 full updates :(

Made another test : I refresh the whole screen (cat /dev/urandom > /dev/fb8), wait 1 second, and repeat...
It works, but I see 2 refreshs for the same command (spaced by 150ms ) :

[ 379.550473] fb_ili9340 spi0.0: Display update: 992 kB/s (150.410 ms), fps=0 (1069.629 ms)
[ 379.701292] fb_ili9340 spi0.0: Display update: 990 kB/s (150.766 ms), fps=6 (150.454 ms)

It did more than 150 copies (and so 300 refresh) and did not hanged !

My SDL software is making screen refresh every 30 ms (approx.). Could there be a "thread" problem ? fbtft read data as the software updates it ?

Update : that is totally random ...
I could update at a high refresh rate (3/4 fps) for more thant 20 seconds, stopped then launch the same test again, hanged after 2-3 seconds.
The "1 second delay" test did worked for more than 100 seconds a few minutes ago, and then just hanged now...

[notro]

There are 2 different fbdev code paths that lead to a display update:

• Using fb_ops functions like fbcon and writing to /dev/fb* does.
• mmap /dev/fb* like SDL xorg etc. does. This triggers page faults on write that is handled by http://lxr.free-electrons.com/source/drivers/video/fbdev/core/fb_defio.c

Both code paths uses the same worker (info->deferred_work) which results in fbtft_deferred_io() being called after a delay.
http://lxr.free-electrons.com/source/drivers/staging/fbtft/fbtft-core.c#L435

If I were to troubleshoot this, I would build a kernel and just let fbtft_write_spi() return 0 right away to see if the problem is in the spi driver.
http://lxr.free-electrons.com/source/drivers/staging/fbtft/fbtft-io.c

[Adesin-fr]

Thanks for those details.

I don't have enought time at the moment to build a custom kernel, but I'll give a try as soon as possible.
Perhaps another linux distr with another kernel would do the trick !

[Adesin-fr]

Got some time to try with another distro :
Instead of raspbian, tried dietpi (I know, maybe a wrong test, dietpi is made on raspbian !!)

Seemed to work a few minutes, then I had to reboot, and didn't worked long, but some information in dmesg :
Not sure if it will help ?

(Code is "long", made a pastebin : http://pastebin.com/PeV0mHkJ)

[notro]

These are the two important ones:

sunxi_spi_transfer() queue up work that calls into sunxi_spi_work() which calls into sunxi_spi_xfer() and probably gets stuck waiting for an interrupt or some other problem (just guessing). sunxi_spi_xfer() is probably inlined thats why it doesn't show up here.

[  240.370117] INFO: task kworker/u:0:5 blocked for more than 120 seconds.
[  240.370145] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  240.370169] kworker/u:0     D c068a4a4     0     5      2 0x00000000
[  240.370246] [<c068a4a4>] (__schedule+0x574/0x7fc) from [<c068a7bc>] (schedule+0x90/0x94)
[  240.370292] [<c068a7bc>] (schedule+0x90/0x94) from [<c06886fc>] (schedule_timeout+0x2c/0x2bc)
[  240.370341] [<c06886fc>] (schedule_timeout+0x2c/0x2bc) from [<c0689c98>] (wait_for_common+0x120/0x168)
[  240.370391] [<c0689c98>] (wait_for_common+0x120/0x168) from [<c0689d00>] (wait_for_completion+0x20/0x24)
[  240.370443] [<c0689d00>] (wait_for_completion+0x20/0x24) from [<c045eaa8>] (sunxi_spi_work+0x748/0x888)
[  240.370496] [<c045eaa8>] (sunxi_spi_work+0x748/0x888) from [<c0044508>] (process_one_work+0x248/0x404)
[  240.370543] [<c0044508>] (process_one_work+0x248/0x404) from [<c0045940>] (worker_thread+0x1b4/0x2d0)
[  240.370590] [<c0045940>] (worker_thread+0x1b4/0x2d0) from [<c0049c58>] (kthread+0x94/0xa0)
[  240.370639] [<c0049c58>] (kthread+0x94/0xa0) from [<c000f1b0>] (kernel_thread_exit+0x0/0x8)

This looks like a good source candidate for the spi master driver: https://github.com/loboris/OrangePI-Kernel/blob/master/linux-3.4/drivers/spi/spi-sunxi.c

This is the one that initiated the transfer and waits for the other thread to signal completion:

[  240.371171] INFO: task kworker/3:1:27 blocked for more than 120 seconds.
[  240.371191] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  240.371212] kworker/3:1     D c068a4a4     0    27      2 0x00000000
[  240.371264] [<c068a4a4>] (__schedule+0x574/0x7fc) from [<c068a7bc>] (schedule+0x90/0x94)
[  240.371307] [<c068a7bc>] (schedule+0x90/0x94) from [<c06886fc>] (schedule_timeout+0x2c/0x2bc)
[  240.371353] [<c06886fc>] (schedule_timeout+0x2c/0x2bc) from [<c0689c98>] (wait_for_common+0x120/0x168)
[  240.371401] [<c0689c98>] (wait_for_common+0x120/0x168) from [<c0689d00>] (wait_for_completion+0x20/0x24)
[  240.371449] [<c0689d00>] (wait_for_completion+0x20/0x24) from [<c045b920>] (spi_sync+0x84/0xa0)
[  240.371522] [<c045b920>] (spi_sync+0x84/0xa0) from [<bf15234c>] (fbtft_write_spi+0xf0/0x104 [fbtft])
[  240.371611] [<bf15234c>] (fbtft_write_spi+0xf0/0x104 [fbtft]) from [<bf152208>] (fbtft_write_vmem16_bus8+0x11c/0x138 [fbtft])
[  240.371691] [<bf152208>] (fbtft_write_vmem16_bus8+0x11c/0x138 [fbtft]) from [<bf14f918>] (fbtft_update_display+0x150/0x2fc [fbtft])
[  240.371767] [<bf14f918>] (fbtft_update_display+0x150/0x2fc [fbtft]) from [<bf14f2a8>] (fbtft_deferred_io+0x110/0x11c [fbtft])
[  240.371829] [<bf14f2a8>] (fbtft_deferred_io+0x110/0x11c [fbtft]) from [<c03fdc60>] (fb_deferred_io_work+0x84/0xcc)
[  240.371877] [<c03fdc60>] (fb_deferred_io_work+0x84/0xcc) from [<c0044508>] (process_one_work+0x248/0x404)
[  240.371922] [<c0044508>] (process_one_work+0x248/0x404) from [<c0045940>] (worker_thread+0x1b4/0x2d0)
[  240.371968] [<c0045940>] (worker_thread+0x1b4/0x2d0) from [<c0049c58>] (kthread+0x94/0xa0)
[  240.372013] [<c0049c58>] (kthread+0x94/0xa0) from [<c000f1b0>] (kernel_thread_exit+0x0/0x8)

Ref: http://lxr.free-electrons.com/ident?v=3.4;i=spi_sync

So it seems to be a spi master driver problem. You have to build a kernel with debuging enabled in the driver to be sure.

[Adesin-fr]

Ok, thanks for pointing out the problem.
I think this will go a bit beyond my competences, so I'm giving up this time !

I'm switching on a raspberry pi 3, since it seems that this but doesn't exist there !

Thanks again for your time !

[Adesin-fr]

Well...
Tried a raspberry pi 3
Works well...
It seems clear that there is a bug in the spi driver from orange pi kernel...

[st4ck]

News about this bug? I've the same problem with ILI9486. When refresh rate became high and for the entire screen, the tft hangs. No matter what I do, I'm not unable to update it again until I reboot the system

[Adesin-fr]

Hi,

I haven't checked back on a Orange pi PC, but I didn't had the issue on a raspberry pi 3.

It seems to be a bug related to spi/dma driver on allwinner H3 kernel.

[st4ck]

It's sad no one tried to solve this problem and update the kernel. I think I will migrate to a Raspberry. Thanks for the reply

[Adesin-fr]

The problem is highly due to allwinner spi "driver", hence the job will certainly be done via sunxi team ?
Try to contact them, but I'm not sure they will be aware of our problem :(

[st4ck]

Yes, the problem must be resolved by their team. I've tried everything and the problem is less often when the fps are higher (don't ask me why). With 100FPS for example the problem appear after some minutes and not after 10 seconds
I've done a version of fbcp (https://github.com/st4ck/OrangePi_FBCP/) with entire rw from/to memory and the problem seems to be less frequent but the bug remain and there's not a solution except to recompile fw & kernel
I'll try to contact the team hoping they'll respond. Thanks anyway

[krachlatte]

Hi any news on this topic, having the same problem, what do you need in order to fix this i could do some test provide logs .... just let me know thanks in advance

[notro]

Closing issue since there has been no activity for more than 2 months. Reopen if needed.

[kotc]

commenting to save others from redoing whole search. on Allwinner H3 and similar max transfer spi is limited to 64 bytes, otherwise controller could get confused sometimes. in legacy kernel (3.4.x) the workaround is to use txbuflen=64 with fbtft_device, in mainline kernels it's already fixed. cheers!