Support for websocket? #49
Comments
|
@squonk11 I'm actually currently thinking about how to best implement minimal, but functional, websocket support. If you have suggestions on how, feel free to contribute. PRs are accepted. |
|
Unfortunately I am not a professional SW engineer but a privately interested enthusiast. I implemented already the websocket protocol into this repository. But since this is Arduino I am asking myself if it is easier to port this to esp-idf or implement websocket into "smooth". |
Not really, mainly template and polymorphism. I'm sure you can understand it given time and a bit of help :) The main classes of interest are likely
Keep-alive is already property of the I'll be thinking about this a bit more while doing some gardening. Btw, I'm usually on IRC in #ESP32 @ freenode, under nick "permal" if you want to chat. |
|
So I gave this some thought. I think the proper way is to rework the |
|
@squonk11 As you can see, I've begun an implementation of websockets, currently working on the connection upgrade. Do you happen to have a good debugging tool for it, Firefox doesn't give any info on why it doesn't accept the websocket connection from /websocket.html, generated here? |
|
Sorry for not answering yesterday - Saturdays and/or Sundays I have to take care for my father. |
|
No worries, @squonk11. IRL is important :) |
|
Sorry, but I did not get much time to check your sourcecode. What I have seen so far is that you already provided the skeleton for websockets and the detection of the upgrade. But reading the RegularHTTPProtocol.cpp (in order to make a similar WebsocketProtocol.cpp) is still a bit difficult for me as I am not so familiar with templates. I think I need to read first a little bit about using templates. |
|
@squonk11 Its OK. I wanted this functionality myself so don't feel you owe me anything. I'm currently working out how to decode the incoming frames and how to re-use/transform the If you have any gotchas from when you implemented the protocol feel free to share them. FYI: I'm not going to support data frames with sizes given by 64 bits, it just doesn't make sense for an embedded device. |
|
When I implemented the websocket server I implemented a WebsocketHandler class with virtual functions for onMessage, onClose and onError which the end-user has to override. For reading the data I was using a std::streambuf.
Your approach not to implement frame sizes with 64bit size probably makes sense - although in my application I have to transmit a file with a size of approx. 170kB. Here 64k (16bit) is too small. In order to handle that amount of data I will be using a Wroover module with 8MB PSRAM. But I think I can also cut the file into pieces of <64kB. |
|
I realized that handling 65k or larger is no additional work compared to handling <65k so I implemented it. |
|
And it is thankfully reproducible on Linux. |
|
Good to know that it also happens on Linux. I will try to make some investigation this evening (when I am back home) using debug outputs on the ESP32. Maybe the root cause can be isolated by this. |
|
I know now that something goes wrong when receiving data in WebSocketProtocol.cpp. Sometimes, the data in the buffer looks like garbage. I'm wondering if the de-masking is to blame? |
|
I now detected that sometimes the data sent from ESP to the client gets partially corrupted. I can see in the wireshark traces that some of the "a" are damaged. Everytime when this happens the client (browser) closes the TCP connection. |
option is trickier to get right re. indexing.
|
Yep - it was the de-masking of the data that were wrong, so when the data was echoed back, it was just garbage. I tried to do it on the fly, but the indexing got wrong. Now it is done on a per-fragment basis instead. I'm still not sure if this covers all cases with different chunk sizes, but at at least I can't force the error any longer I did have a look at this earlier today, but didn't see the issue then... oh well, got to keep oneself busy, right? :) |
|
Ah, found a way to force the issue - set a chink size of 53 and send data in blocks of 8092. At least I now know what the problem is. |
|
Argh, dammit. Its still possible to break it. |
|
Hm, this is annoying. All of a sudden, a single message can't be properly demasked, but those before and after can. Is the masking key somehow corrupted? |
|
Observation: It's always the first fragment of the message. The second part is OK. |
|
Has to be a logic error in |
|
Made the code as simple as I can and while it is much more stable now, sometimes the de-masked data is still garbage in the first fragment. The de-mask key is intact between the different fragments so that's not the problem. It only seems to happen when FF or Chrome sends lots of data in a short amount of time. For example, I can send 10M multiple times, then all of a sudden there is demasked garbage. |
|
I could imagine that the problem is related to concurrency of incoming packets. E.g. I see that the demasking is done at the very end of |
|
The mask key is held for all fragments so it is still valid (and the code for demaksing is running) |
|
I think I've found the actual root cause. I've forgotten that there is no guarantee that data_received receives the amount requested in get_wanted_amount, only that it never receives more than the requested amount. As such the the protocol gets out of sync with the data and things stop working. I'm going to push a work-in-progress commit that logs some data that shows this is the case. To fix it, I'll probably rewrite the data_received, get_wanted_amount and get_write_pos. as can be seen in the coming commit, it just gets messy when trying to patch it. |
|
When I implemented websocket communication in the previous project I used |
|
With the last changes I'm now able to push gigabytes through a single websock connection without errors on a Linux system. However, when run on the ESP and really stressing the system, the connection breaks occationally, but I cannot see that it has anything to do with the implementation. As such, I consider the implementation correct until someone proves otherwise, and it is repeatable on a Linux system. |
|
@squonk11 I've merge websocket support into master. Closing this issue. Please open a new one if needed. |
|
I just wanted to inform you about the following observation: One other topic: I will need the ESP32 wifi to be working in access point mode. I will make a class "wifiAP" for this (similar to your wifi class). Are you interested in a pull request? |
|
@squonk11 That's a good observation, and one I was hoping for. Memory issue are a pain; I'm glad I can develop on Linux before running things on the ESP, it allows me to be more sure of my own code. Yes, I will accept PRs; expect constructive criticism. I've long thought of adding AP-support to the Wifi class, but never gotten around to it. Give some thought on if you can add it to the existing one, possibly renaming Please open a new issue for further discussions. |
|
@PerMalmberg Yes, I will open a new issue for the wifi access point topic. But be aware that my progress will be quite slow because I am working on this during my free time which is very rare at the moment and in SW I am a hobbyist only (my profession is more related to HW). |
Hi, congratulations for these nice classes for the ESP32. I am currently searching for a HTTP(S) webserver software with the following attributes:
I see that 1. - 3. is already provided by your classes. The only thing missing for me is the support for websocket communication. Is there a plan to implement the websocket protocol also?
Best regards
Lothar
The text was updated successfully, but these errors were encountered: