-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ESP32 restart if using websockets. #325
Comments
This code also works 100%. Here I am using this lib https://github.com/bbx10/WebServer_tng So something is not well on the ESP32 using this lib.
|
I am also caught with a similar error. I also like this code, so I would like to use it if possible. Error 1:
Error 2:
Error 3:
Error 4:
Error 5:
|
I am experiencing the same errors also. I'm trying to get my JTAG debugger setup but it is very touchy especially when in the core guts of the memory allocation functions. All you have to do to recreate the problem is just open up a web socket and start sending text. I was waiting until there was others that had the problem before posting. Thinking it was something in my code, I am confirming Error2 in the previous post. Now I did clear up the some errors, by making the onWS callback function simple and placing any memory it modifies in a critical section. I believed for awhile that the issue was sending large amounts of text with WebSocket->textAll function. I think that the callback function of the web socket is invoked from another thread. For those young players this means that you have to take thread synchronization into account. I have done that by placing my callback function of the received data so that the data it modifies it does so in a critical section. This helped with other errors but this one still remains. This is on a textAll function call, however it is in the task that handles the lwip functions. Below is the stack backtrace decoded. 0x40088274: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/./panic.c line 578 On the client side I usually get this error in the web browser. When that happens the processor throws assertion "head != NULL" failed: file "/Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/heap/./multi_heap_poisoning.c", line 199, function: multi_heap_free Any help would be greatly appreciated. This might not be in this library also as I have seen a lot of issues with heap corruption detection code screwing up other items. It might be issues with one task allocates memory and another frees it, causing issues. |
My esp32 is having similar issues with random crashes, sometime within a few hours or maybe runs a day. |
I am also caught with a similar error. Connecting to WiFi... ws[/ws][1] connect Backtrace: 0x400882ec:0x3ffd4210 0x400883eb:0x3ffd4230 0x400dd277:0x3ffd4250 0x40088015:0x3ffd4280 0x4008413a:0x3ffd42a0 0x400846c5:0x3ffd42c0 0x4000bec7:0x3ffd42e0 0x400f8f39:0x3ffd4300 0x400f8ffb:0x3ffd4320 0x4011b9d5:0x3ffd4340 0x400f6135:0x3ffd4360 Rebooting... rst:0xc (SW_CPU_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT) |
I have not had any issues using this lib https://github.com/bbx10/WebServer_tng . The ESP32 has been running for 3 weeks without any crash and I serve large html5 bootstrap files using web-sockets. Here are the screens I am using and all is working fine using the other lib. Pity this one is so unstable. |
Thanks @JacoFourie for sharing this... yes I know about the random crashes and I am looking into it... could not figure out why it happens yet but I have some TCP redesigns in mind that should help. @JacoFourie I don't know if you realize this, but the server you liked is Sync server ;) |
@me-no-dev sorry I don't understand what you mean by Sync server ? |
@JacoFourie if you have to call something like server.handleClient() in your loop() then its sync server (and supports probably only one connection @ time). Another disadvantage of sync server without native ws plugin is that you need two ports - one for www server and one for ws server. ESPAsyncWebServer IMHO handles more than one connection simultaneously and uses only one listening port for both www and ws part. |
@Pablo2048 Aaa OK I get that. But the code I have can handle up to 20 connections at once. I tested it. I only need one connection as it will be a user interface to set the program values and view the telemetry. I would love to use this lib. But it crashes all the time. So even if the design is better, it does not work for my use with websockets. The other one does work 100% for my use, even if I do have to use another lib for the web sockets and define another port. I am willing to test this and port my code to this lib if they can sort out the crashes and reboots. |
I monitored this bug while monitoring the heap. I do not know how to solve it, but it is good to use it. |
I gradually understood the structure, but I am pretty tough. |
I cited a video here. |
I believe I've got the same problem, I put this on gitter last weekend and @me-no-dev confirmed it was reproducible: When the ESP crashes this is always preceded by an error 1002 which has as data 'Unexpected continuation....' where the remainder of the string will have some combination of pieces from the JSON string and rubbish data. In some cases this error is shown but it's not followed by a crash. The crash will only occur if the websocket client also sends a message, if you just leave it listening it keeps running for 10+ minutes without problems I've uploaded the minimalist code which demonstrates the crash here.
|
Any news on the subject? This library is very good, but on ESP32 it can not be used. |
@valerivp I use websockets on an esp32 without issues. From the exceptions on the initial report it looks like a heap exhaustion issue which may not be related to this library. |
@atanisoft, Are you using this library or something? It is important for me to use HTTP and Websocket on the same port |
I have not tested using another websockets lib with this web server lib. I have been using the other lib as stated and it is very stable. Been running for months now without any crashes. even if it is a sync server. |
@JacoFourie , which
? |
And I use this websocket server with the web server. |
@JacoFourie , They work together on the same port (80)? |
No. And that is why I would like to use this lib. But last time I check it was not stable at all. If the lib can be made stable on the ESP32 than I would move my code to this lib. |
The issues with ESPAsyncWebServer remain. Even with the most easy example of using this lib like (https://techtutorialsx.com/2017/12/01/esp32-arduino-asynchronous-http-webserver/) if you enter on page generated by esp32 <ip_adr>/hello from multiple computers and refresh the web-page it will result in the same errors and backtraces. Tested this last week multiple times. Do not know if this is a espressif/esp-idf problem or if this results from the espressif/arduino_esp32 |
If this is happening only on ESP32 (dual core) then this could well be a race issue. I had very similar problems using AsyncTCP in my code, because often I had (mistakenly) written the code assuming that events would never interrupt each other, which is fine on the ESP8266, but can lead to memory corruption on the ESP32. |
@BlackEdder can you not have a look as it seems you have experience with this issue? It would be great if it can get fixed. |
@valerivp yes, I use this library and have extensive usage of websockets from an http page served by this library which makes callbacks to the esp32 via websockets. It is possible there is a race condition in the handlers in your code and you can add some guards for that using semaphores and delays etc. In all of my handlers I write them so they are simple and do a single task quickly and return control to the library/OS to avoid issues. I have plenty of other issues to handle (hardware interrupt tasks every 50-70ms as example). |
My project work correctly on ESP8266, but on ESP32 я have problem with webcocket |
There are major architecture differences between esp32 and esp8266. More than likely it is not a bug in this library but instead a race condition or other single threaded design pattern in your code or another library you are using that is at fault. |
I get errors with the sample. So then how should one use it ? |
As far as making this work I was reading about many similar issues when using web sockets with AsyncTCP It is my intuition that it is not an issue with this library but something deeper as this mostly just builds upon AsyncTCP. I read about a workaround that adds a handler into the Async loop this would prevent any issues of both your code and the loop trying to send at the same time, but I can't remember where I saw that. |
I see now I am using the Arduino Core web server. I installed my IDE using the new method and the IDE picked that lib. So all is working using the core webserver also. |
Here is the system I am working on. Now porting it to the M5Stack, |
Can you all with this issue (that you can reproduce) test it with different browsers? I get conflicting reports that maybe Chrome/Webkit is causing this to happen. The more I know, the easier it will be to pinpoint the problem. |
My default browser is Fire Fox. I will test with some other browsers also. |
I can reproduce in Firefox and Chromium browsers. |
I can reproduce continuation error with this simple project:
|
Just tried commit 4c621f3 and it works for me. 👍 |
Hello again - updated comment, I also tried the commit "4c621f3" and I see that the problem persists unfortunately... The code that I tested on is presented in the ESPAsyncWebServer in the examples, the simple_server.ino (https://github.com/me-no-dev/ESPAsyncWebServer/blob/master/examples/simple_server/simple_server.ino). In order to reproduce the issue:
Case 1:
========================================================= Case 2:
.Most of the time I get just the first error case, with the "CORRUPT HEAP: Bad head at 0x3ffbdba4. Expected 0xabba1234 got 0x3ffbe034". The second error case happens more rare. |
please decode any backtrace exception you get. The message on it's own does not mean much |
Also it's important to separate WebSocket issues from other issues. They might or might not be related. In all cases! Please post minimal example sketches and instructions on how to reproduce the issues you have. If I can reproduce an issue, chances are that I will fix it :) Also! Don't be shy and come to gitter :) I get hundreds of emails daily and going through them is not easy |
I have just pushed an update to the AsyncTCP lib. please give it a go (in combination with the latest code for the server) |
I updated the AsyncTCP and ESPAsyncWebServer to the commit heads from last Thursday and now to the latest commits. I tested again with the code from: the simple_server.ino (https://github.com/me-no-dev/ESPAsyncWebServer/blob/master/examples/simple_server/simple_server.ino). Tests were conducted on: Google Chrome/Mozila Firefox/internet Explorer Edge. What I tested was refreshing the same page ex: 192.168.1.195/ on 2 different pc's in the same web browser and testing 2 different pages, ex: pc 1:192.168.1.195/ and pc 2: 192.168.1.195/get in the same web browser as explained bellow. .=> results using ESPAsyncWebServer commit: 4c621f3 and AsyncTCP commit: 34a0320 (the txt files have also the results from Exception decoder plugin)
Small facts: what I saw was that the Esp32 cracks with the above traces usually after multiple msgs like .=> results using ESPAsyncWebServer commit: bed4146 and AsyncTCP commit: 5453ec2 (the txt files have also the results from Exception decoder plugin):
Small facts: what I saw was that the Esp32 cracks with the above traces usually after multiple msgs like .I hope that the above documents can shine some light over the problem. |
Hi! |
So this lib is still not working using websockets with the webserver ? |
This is still an open issue. There was a huge progress on it. With the latest Async TCP lib is very stable for me. However when i open my webpages from SOFTAP with an android phone, the restart and the heap poisoning is always coming back.
Decoded:
I'am not doing anything else then opening a page on android phone in the softap. In that page there are 1-2 websocket calls when the server loaded. But the crash happens before the websocket calls. |
I have submitted 3 Pull Requests, which seem together fix all of the instability I have had with websockets. The 3 problems I identified are:
With these three changes in place I have not seen the ESP32 reboot, although I am not doing any kind of serious testing at the moment. It would be great if anyone who still has instability issues around the websockets to give the changes a go. |
@matt123p Thank you, it runs much more stable now (no crashes especially on Chrome with normal site refreshes). But I managed to crash it nonetheless with Chrome and Edge (extreme refreshes):
Crashes are:
But I don't know if this is websocket related or not... |
Unfortunately I blew up my last ESP32, so I can't test at the moment, but if I had to guess, I would say you are hitting some kind of resource limit. If you want to debug the problem further, then you should check the number of open sockets at any one time and make sure it isn't spiralling upwards. You can look at calling count() on the websockets object or if that isn't the problem then try checking the open of open AsyncTCP sockets (there is no functionality to do this - so you would have to add the code to do the counting). I will be able to have a better look when I get a new ESP32. |
Hi. Where do I find the latest version of this that you state is stable now? I would like to test it and I will give you feedback. |
Sorry forgot to mention that - you can download a copy from my fork: matt123p/ESPAsyncWebServer There is a small code change required, you need to call "cleanupClients()" periodically from your main loop - in my test I do this around once per second. |
[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. |
[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions. |
This issue seems to be resolved in the latest close this one out. Thanks me-no-dev for your library and time. |
I faced the similar issues with AsyncWebSocket when sending huge data continuously . Check the size of the messages you are sending and set this parameter. |
My post may be relevant. |
Awesome . I will note it down . Thanks 👍🏻
Thanks And Regards,
Ujwal Nandanwar
…On Thu, 18 Apr 2024 at 6:48 PM, VikingVoltage ***@***.***> wrote:
My post
<s00500/ESPUI#75 (comment)> may
be relevant.
—
Reply to this email directly, view it on GitHub
<#325 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ASDSL567JOF3QSKNU5JO75TY57BYPAVCNFSM4EQE7CC2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMBWGM4DIOBVGYZA>
.
You are receiving this because you commented.Message ID:
***@***.***>
|
Hi all. I get different types of errors and restarts at random. I use the webserver to send out a html page. In this page there is a connection to a websocket and then the ESP32 will send out the temperature to all clients. It will work fine for a while and then it will crash at random with diffrent errors.
here are the errors I see.
error 1
error 2
error3
If I replace the websever with the Arduino-Core one and use another websocket lib all works fine. But I would love to use this one as it handles a lot for me. But it is not stable at this point.
This code works 100%
and this code will crash at random.
The text was updated successfully, but these errors were encountered: