Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

sending frame with a while time and system get crash on websokcet (IDFGH-1973) #4169

Closed
huazhenjiang opened this issue Oct 7, 2019 · 5 comments
Closed

sending frame with a while time and system get crash on websokcet (IDFGH-1973) #4169

huazhenjiang opened this issue Oct 7, 2019 · 5 comments
Assignees
@david-cermak

Comments

@huazhenjiang
Copy link

Environment

Development Kit: ESP32-DevKitC
Kit version (for WroverKit/PicoKit/DevKitC): v2
Module or chip used: ESP32-WROOM-32
IDF version (run git describe --tags to find it): v4.0-released
Operating System: Windows 10
Power Supply: USB

###Problem Description
I try the websoket example to connect wss server and send a binary data about 50 bytes per 50ms. Also I modify the mask bit for each sending frame. Then the system get crash and auto reboot about 10 minutes later. The following log show it get a 'esp-tls:write error' first then get a 'esp-tls: read error', disconnect finally. It seems some memory address can not be writable but it occured.

###Expected Behavior
sending packets continuously
###Actual Behavior
system crash and auto reboot
###Steps to repropduce

###Code to reproduce this issue
none.

###Debug Logs
E (775041) esp-tls: write error :-27648:
E (775041) TRANS_SSL: esp_tls_conn_write error, errno=Success

E (775661) esp-tls: read error :-30592:
E (775661) TRANS_SSL: esp_tls_conn_read error, errno=Success
E (775661) TRANSPORT_WS: Error read data
E (775671) WEBSOCKET_CLIENT: Error read data
E (775671) esp-tls: write error :-27648:
I (775671) WEBSOCKET_CLIENT: Reconnect after 10000 ms
E (775681) TRANS_SSL: esp_tls_conn_write error, errno=Success
Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC : 0x4000c344 PS : 0x00060730 A0 : 0x801186e1 A1 : 0x3ffcde00
A2 : 0x3ffcbde4 A3 : 0xfefefefc A4 : 0x0000000c A5 : 0x3ffcbde4
A6 : 0x00000000 A7 : 0x00000000 A8 : 0x80000000 A9 : 0x3ffcdaa0
A10 : 0x00000049 A11 : 0x00000002 A12 : 0x00000001 A13 : 0x3ffcdcb0
A14 : 0x00000049 A15 : 0x3ffcdcb0 SAR : 0x00000010 EXCCAUSE: 0x0000001c
EXCVADDR: 0xfefefefc LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xfffffffd

ELF file SHA256: 95686d089b74a9f15266d91d78e6e37e0539990f031642a2da9aa14dbe45ae1b

Backtrace: 0x4000c341:0x3ffcde00 |<-CORRUPTED

Rebooting...
ets Jun 8 2016 00:22:57
@github-actions github-actions bot changed the title sending frame with a while time and system get crash on websokcet sending frame with a while time and system get crash on websokcet (IDFGH-1973) Oct 7, 2019
@david-cermak
Copy link
Collaborator

Hi @huazhenjiang

Thanks for reporting this issue! Seems to be a race condition as the client first checks if connected and locks afterwards. You can test if the issue goes away moving locking before connection check.
Will fix this issue!

@huazhenjiang
Copy link
Author

huazhenjiang commented Oct 8, 2019 •
edited
Loading

Hi @david-cermak,

Do you mean moving the semaphoreTake like the following graph?
image

I try this on my project and also work for a while, then it occurs the following situation.

log

E (1643110) esp-tls: write error :-27648:
E (1643110) TRANS_SSL: esp_tls_conn_write error, errno=Success
E (1643110) TRANSPORT_WS: Error write header

E (1643570) esp-tls: read error :-30592:
E (1643570) TRANS_SSL: esp_tls_conn_read error, errno=Success
E (1643570) TRANSPORT_WS: Error read data
E (1643580) WEBSOCKET_CLIENT: Error read data
I (1643590) WEBSOCKET_CLIENT: Reconnect after 10000 ms
assertion "sock->fd_used != 0" failed: file "/home/huazhen/Work/mmwave-esp32/esp-idf/components/lwip/lwip/src/api/sockets.c", liI (1643590) WEBSOCKET: WEBSOCKET_EVENT_DISCONNECTED

ne 364, function: sock_inc_used_locked
abort() was called at PC 0x4016bbab on core 1
0x4016bbab: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)

ELF file SHA256: 2ae57db2bd4a6bbb80018a9c062fd700149eeefe8bdbca01813eda5033e36f57

Backtrace: 0x4008c55d:0x3ffcd3f0 0x4008c8d5:0x3ffcd410 0x4016bbab:0x3ffcd430 0x400fbb2a:0x3ffcd460 0x400fbb67:0x3ffcd480 0x400fbce1:0x3ffcd4a0 0x400fbd2a:0x3ffcd4d0 0x400fe0d3:0x3ffcd4f0 0x400d5e02:0x3ffcd590 0x4011ca5d:0x3ffcd5f0 0x401186ef:0x3ffcd610 0x40182411:0x3ffcd640 0x40118788:0x3ffcd660 0x401823d1:0x3ffcd690 0x40118e02:0x3ffcd6b0 0x40118e98:0x3ffcd6f0 0x401823d1:0x3ffcd710 0x40112ee7:0x3ffcd730 0x400dacad:0x3ffcd750 0x4008eaad:0x3ffcd8f0
0x4008c55d: invoke_abort at /home/huazhen/Work/mmwave-esp32/esp-idf/components/esp32/panic.c:155

0x4008c8d5: abort at /home/huazhen/Work/mmwave-esp32/esp-idf/components/esp32/panic.c:172

0x4016bbab: __assert_func at /builds/idf/crosstool-NG/.build/xtensa-esp32-elf/src/newlib/newlib/libc/stdlib/assert.c:62 (discriminator 8)

0x400fbb2a: sock_inc_used_locked at /home/huazhen/Work/mmwave-esp32/esp-idf/components/lwip/lwip/src/api/sockets.c:364 (discriminator 1)

0x400fbb67: tryget_socket_unconn_locked at /home/huazhen/Work/mmwave-esp32/esp-idf/components/lwip/lwip/src/api/sockets.c:448

0x400fbce1: lwip_select_inc_sockets_used_set at /home/huazhen/Work/mmwave-esp32/esp-idf/components/lwip/lwip/src/api/sockets.c:1918

0x400fbd2a: lwip_select_inc_sockets_used at /home/huazhen/Work/mmwave-esp32/esp-idf/components/lwip/lwip/src/api/sockets.c:1939

0x400fe0d3: lwip_select at /home/huazhen/Work/mmwave-esp32/esp-idf/components/lwip/lwip/src/api/sockets.c:1992

0x400d5e02: esp_vfs_select at /home/huazhen/Work/mmwave-esp32/esp-idf/components/vfs/vfs.c:999

0x4011ca5d: select at /home/huazhen/Work/mmwave-esp32/esp-idf/components/newlib/select.c:62

0x401186ef: ssl_poll_write at /home/huazhen/Work/mmwave-esp32/esp-idf/components/tcp_transport/transport_ssl.c:104

0x40182411: esp_transport_poll_write at /home/huazhen/Work/mmwave-esp32/esp-idf/components/tcp_transport/transport.c:206

0x40118788: ssl_write at /home/huazhen/Work/mmwave-esp32/esp-idf/components/tcp_transport/transport_ssl.c:112

0x401823d1: esp_transport_write at /home/huazhen/Work/mmwave-esp32/esp-idf/components/tcp_transport/transport.c:190

0x40118e02: _ws_write at /home/huazhen/Work/mmwave-esp32/esp-idf/components/tcp_transport/transport_ws.c:204

0x40118e98: ws_write at /home/huazhen/Work/mmwave-esp32/esp-idf/components/tcp_transport/transport_ws.c:231

0x401823d1: esp_transport_write at /home/huazhen/Work/mmwave-esp32/esp-idf/components/tcp_transport/transport.c:190

0x40112ee7: esp_websocket_client_send at /home/huazhen/Work/mmwave-esp32/esp-idf/components/esp_websocket_client/esp_websocket_client.c:565

0x4008eaad: vPortTaskWrapper at /home/huazhen/Work/mmwave-esp32/esp-idf/components/freertos/port.c:143

@david-cermak
Copy link
Collaborator

@huazhenjiang Thanks for reporting this bug and sorry for not getting back to you sooner.
The fix seems a bit more complex that what was suggested here above. It will be delivered to the IDF as soon as formally approved and tested.

Meanwhile, posting a patch agains IDF-4.0 that should address these issues.
0001-websocket_client-fix-locking-mechanism-in-ws-client-.patch.txt

@huazhenjiang
Copy link
Author

@david-cermak thank you, this patch looks like worked.

@Alvin1Zhang
Copy link
Collaborator

@huazhenjiang Thanks for the testing and confirmation. Will reopen the issue and wait until our fix is merged, thanks.

@Alvin1Zhang Alvin1Zhang reopened this Oct 25, 2019
espressif-bot pushed a commit that referenced this issue Feb 27, 2020
@david-cermak @espressif-bot
websocket_client: fix locking mechanism in ws-client task and when se…
daf9580 
…nding data

closes #4169
gabsuren pushed a commit to gabsuren/esp-protocols-1 that referenced this issue Apr 8, 2022
@david-cermak @suren-gabrielyan-espressif
websocket_client: fix locking mechanism in ws-client task and when se…
419123b 
…nding data

closes espressif/esp-idf#4169


* Original commit: espressif/esp-idf@7c5011f
euripedesrocha pushed a commit to euripedesrocha/esp-protocols that referenced this issue Oct 17, 2022
@david-cermak @euripedesrocha
websocket_client: fix locking mechanism in ws-client task and when se…
bc8d988 
…nding data

closes espressif/esp-idf#4169


* Original commit: espressif/esp-idf@7c5011f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@david-cermak david-cermak

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@huazhenjiang @david-cermak @Alvin1Zhang