Using client.send() outside the loop (FireFox client incompatibility)

[xgarb]

Hi. This might be more of a programming question again but...

Something like this works:

void delete_all_faces() {
  delete_face_all_in_flash_with_name(&st_face_list);
}
void handle_message(WebsocketsMessage msg) {
  if (msg.data() == "delete_all") {
    delete_all_faces();
    g_state = DELETE_ALL;
  }
}
void loop() {
   auto client = socket_server.accept();
   client.onMessage(handle_message);
   while (client.available()) {
      client.poll();
      switch (g_state) {
         case DELETE_ALL:
         client.send("All Faces Deleted");
         break;
         case ...
      }
   }
}

But I would prefer to send the message from the delete_all_faces function something like this...

void delete_all_faces() {
  delete_face_all_in_flash_with_name(&st_face_list);
  client.send("All Faces Deleted");
}
void handle_message(WebsocketsMessage msg) {
  if (msg.data() == "delete_all") {
    delete_all_faces();
  }
}
void loop() {
  auto client = socket_server.accept();
  client.onMessage(handle_message);
  while (client.available()) {
    ...
  }
}

Is it possible to make 'client' accessible outside the loop?

[gilmaimon]

Yes it is possible, and even very easy. Message callbacks come in 2 forms, you use the shorter one but if you wish to also accept the client as parameter to handle_message, you can. Look here

After you get the client as parameter in handle_message, passing it over to delete_all_faces() is simple (pass as reference)

[xgarb]

Like this? (I've cut non relevant code) It works, but I'm not sure I've done it correctly.

static esp_err_t send_face_list(WebsocketsClient &client)
{
  face_id_node *head = st_face_list.head;
  char face[64];
  for (int i = 0; i < st_face_list.count; i++)
  {
    sprintf(face, "facelist:%s", head->id_name);
    client.send(face);
    head = head->next;
  }
}

static esp_err_t delete_all_faces(WebsocketsClient &client)
{
  delete_face_all_in_flash_with_name(&st_face_list);
  client.send("update_faces");
}

void handle_message(WebsocketsClient &client, WebsocketsMessage msg)
{
  if (msg.data() == "detect") {
    client.send("DETECTING");
  }
  if (msg.data() == "delete_all") {
    delete_all_faces(client);
  }

}

void loop() {
  auto client = socket_server.accept();
  client.onMessage(handle_message);
  while (client.available()) {
    client.poll();
	if (left_sample_face == 0)
	{
		send_face_list(client);
	}
	client.send("NO FACE DETECTED");
	}
}

[gilmaimon]

Looks good to me 👍

[xgarb]

This is the project - https://robotzero.one/esp-who-recognition-with-names/ I need to make some improvements to the tutorial

I noticed that it doesn't work in Firefox. I get a cannot connect to ws//... error. If you're interested in testing your library with one of the ESP32 cameras I'll buy one for you.

[gilmaimon]

There is no need to buy me anything, thank you.

Can you elaborate on what's not working? In what manners do things fail? How can I re-produce? Are you able to re-produce without a camera (just by sending very long messages)?
What software do you use as server (what do you mean by Firefox, a Firefox ws client?)
Have you turned on board logs to see what fails? Do you use the latest library version?

[xgarb]

I've tried with 0.4.0. and 0.4.2 and this code in a web page:

document.addEventListener("DOMContentLoaded", function(event) {
  var baseHost = document.location.origin;
  var streamUrl = baseHost + ":81";
  const WS_URL = "ws://" + window.location.host + ":82";
  const ws = new WebSocket(WS_URL);
  ....
}

gives a Firefox can't establish a connection to the server at ws://192.168.1.108:82/. error

Cut down ESP32 code:

#include <ArduinoWebsockets.h>
#include "esp_http_server.h"
#include <WiFi.h>
#include "camera_index.h"

const char* ssid = "NSA ";
const char* password = "";

using namespace websockets;
WebsocketsServer socket_server;
httpd_handle_t camera_httpd = NULL;

void setup() {
  Serial.begin(115200);
 
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.println("WiFi connected");

  app_httpserver_init();
  socket_server.listen(82);

  Serial.print("Camera Ready! Use 'http://");
  Serial.print(WiFi.localIP());
  Serial.println("' to connect");
}

static esp_err_t index_handler(httpd_req_t *req) {
  httpd_resp_set_type(req, "text/html");
  httpd_resp_set_hdr(req, "Content-Encoding", "gzip");
  return httpd_resp_send(req, (const char *)index_ov2640_html_gz, index_ov2640_html_gz_len);
}



httpd_uri_t index_uri = {
  .uri       = "/",
  .method    = HTTP_GET,
  .handler   = index_handler,
  .user_ctx  = NULL
};

void app_httpserver_init ()
{
  httpd_config_t config = HTTPD_DEFAULT_CONFIG();
  if (httpd_start(&camera_httpd, &config) == ESP_OK)
    Serial.println("httpd_start");
  {
    httpd_register_uri_handler(camera_httpd, &index_uri);
  }
}

void handle_message(WebsocketsClient &client, WebsocketsMessage msg)
{
}

void loop() {
  auto client = socket_server.accept();
  client.onMessage(handle_message);
}

Works great in Chrome. All the communication after the page loads is done with Websockets: https://vimeo.com/338402026

[gilmaimon]

Thank you for the details! This definitely looks like a big with my implementation of websockets that FireFox does not want to accept (maybe chrome doesn't care).
I will try to re-produce this weekend, if I will be able to I will probably open a new issue about it with a more specific name.
Can you write down your FireFox version, just in case?

Thanks

[xgarb]

Firefox is 67 (64 bit) on Win 10.
Same with the Developer Edition 68.0b6
Thanks.

[gilmaimon]

I found and fixed the issue. Patch 0.4.3 is now released. Thank you for the error reporting and details.
Please make sure this works for you, so I can consider this issue resolved.

Gil.

[xgarb]

New problem with .4.3 and .4.4 with Chrome and Firefox. The send binary part of the script (that sends video frames) is crashing:

[D][WiFiClient.cpp:482] connected(): Disconnected: RES: 0, ERR: 128
[E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 11, "No more processes"
[E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 11, "No more processes"
[E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 11, "No more processes"
[E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 11, "No more processes"
[E][WiFiClient.cpp:365] write(): fail on fd 63, errno: 11, "No more processes"
abort() was called at PC 0x40136eeb on core 1

Backtrace: 0x40091408:0x3ffb1d20 0x40091639:0x3ffb1d40 0x40136eeb:0x3ffb1d60 0x40136f32:0x3ffb1d80 0x401362bf:0x3ffb1da0 0x401363ae:0x3ffb1dc0 0x40136365:0x3ffb1de0 0x400d575b:0x3ffb1e00 0x400d57b6:0x3ffb1e30 0x400d4b8a:0x3ffb1e60 0x400d3339:0x3ffb1e80 0x400d9be9:0x3ffb1fb0 0x4008d3a5:0x3ffb1fd0

Rebooting...

Decoding stack results
0x40091408: invoke_abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/panic.c line 155
0x40091639: abort at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/esp32/panic.c line 170
0x40136eeb: __cxxabiv1::__terminate(void ()()) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc line 47
0x40136f32: std::terminate() at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc line 57
0x401362bf: __cxxabiv1::__cxa_throw(void, std::type_info*, void ()(void)) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc line 87
0x401363ae: operator new(unsigned int) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/new_op.cc line 54
0x40136365: operator new[](unsigned int) at /builds/idf/crosstool-NG/.build/src/gcc-5.2.0/libstdc++-v3/libsupc++/new_opv.cc line 32
0x400d575b: websockets::internals::WebsocketsEndpoint::send(char const*, unsigned int, unsigned char, bool, bool, char const*) at C:\Users\Dude\Documents\Arduino\libraries\ArduinoWebsockets\src\websockets_endpoint.cpp line 367
0x400d57b6: websockets::internals::WebsocketsEndpoint::send(char const*, unsigned int, unsigned char, bool) at C:\Users\Dude\Documents\Arduino\libraries\ArduinoWebsockets\src\websockets_endpoint.cpp line 321
0x400d4b8a: websockets::WebsocketsClient::sendBinary(char const*, unsigned int) at C:\Users\Dude\Documents\Arduino\libraries\ArduinoWebsockets\src\websockets_client.cpp line 396
0x400d3339: loop() at C:\Users\Dude\Documents\Arduino\CameraWebServerWeChatWebSockets/CameraWebServerWeChatWebSockets.ino line 358
0x400d9be9: loopTask(void*) at C:\Users\Dude\Documents\ArduinoData\packages\esp32\hardware\esp32\1.0.3-rc1\cores\esp32\main.cpp line 19
0x4008d3a5: vPortTaskWrapper at /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/freertos/port.c line 143

[gilmaimon]

Can you try reverting to version 0.4.2 to see if it is an error in the library or in something external (WiFiClient for example)?

[xgarb]

0.4.0 works great. After that they crash.

[gilmaimon]

Interesting.. I will look into it, thank you.

Just making sure, you mean even 0.4.1 crashes, right?

[xgarb]

Yep but with less detail in the serial monitor. This is after refreshing the browser 3 times to test:

[E][WiFiClient.cpp:365] write(): fail on fd 60, errno: 104, "Connection reset by peer"
[E][WiFiClient.cpp:365] write(): fail on fd 62, errno: 104, "Connection reset by peer"
[E][WiFiClient.cpp:365] write(): fail on fd 59, errno: 104, "Connection reset by peer"

[gilmaimon]

Can you open a new issue with this new bug?

[gilmaimon]

Iv'e started working on it. The error is raised here

It's an error inside esp32's library code of WiFiClient. This might be because the connection is not available and you try to write to it. I can't see any reason why it should not work, all tests I've tried pass with no issue. This might be a server side error..

Otherwise, please share more info:
What is exactly happening? What server and clients are running, and anything else that can be beneficial.
Also, Please open a new issue ❤️

Thank you.