Streaming using code built with PlatformIO fails rapidly with: Stream crash - ESP_ERR_HTTPD_RESP_SEND

[TheWolf95]

Hello!

I'm having issues having a reliable stream, since in short times (random, but between 2 to 30 seconds) the stream crashes, ending itself due to ESP_ERR_HTTPD_RESP_SEND. Which, looking to the ESP-IDF wiki, is due to sending an invalid chunk packet, confirmed by Chrome JS console returning ERR_INVALID_CHUNK_ENCODING 200 (OK).
I've been searching around for a solution, but I couldn't make it far. I also tried forcing ESP ignore the invalid packet error changing res into ESP_OK, but I guess once it happens Chrome just closes that stream connection, returning the error from the first if too.

I do not know if this is of any help, but it happens always on the third if in the stream handler method.

Thank you for your amazing work so far and in advance for your assistance!

[easytarget]

Thankyou for the report;

Each frame of the stream is returned in response to a request sent by the browser (that's how MJPEG streams work)
ESP_ERR_HTTPD_RESP_SEND indicates that sending the response failed; that's all the cam knows. Since it has seen a failure it will close the stream, stopping it from doing that will fail anyway because:
The ERR_INVALID_CHUNKED_ENCODING (I assume you mis-typed above) is chrome saying 'i got a invalid response', it will then stop requesting new frames and the stream ends anyway.

It could be a esp bug that causes an invalid frame/chunk to be sent, but nothing showed up specifically on google; and I do not see this happening on my home cameras so I suspect it is something in the network causing regular packet loss.

I'll keep an eye out for it though, again, thanks for the report.

[easytarget]

And... I'm seeing this now. Interesting. Think it is still something network, rather than code related.
Will monitor and reproduce.

[easytarget]

Question: Are you using PlatformIO?

I can reproduce this when I commandline build using platformio; but when I build the same code using the ArduinoIDE the problem disappears.

[Bulduper]

Same issue here when using PlatformIO. The stream frezzes (fails) after a few seconds. I tried with Arduino IDE but I couldn't make the stream run with Arduino IDE at all because of SPIFFS: mount failed, -10025 and error page...

[easytarget]

@Bulduper : With the IDE; double check the partition scheme you selected has SPIFFS and/or lower the x-clock rate. Also make sure you are using the HEAD revision RC1, not one of the older betas (which are more susceptible to this). This is discussed further in the discussion threads.

[easytarget]

I have spent some time testing this: I see no obvious fix and I am not willing to spend more time on it; platformio is a nice-to-have, and not the focus of this work
If others want to investigate this further I will be happy to integrate a fix later.

Working; Arduino IDE with ESP core 2.0.2:

Firmware: Mar 14 2022 @ 10:48:09 (base: 4.0.rc1)
Sketch Size: 1004944 (total: 1966080, 51.0% used)
ESP sdk: v4.4-beta1-189-ga79dc75f0a

Failing: Platformio (current config)

platform = https://github.com/platformio/platform-espressif32.git#feature/arduino-upstream
platform_packages = framework-arduinoespressif32@https://github.com/espressif/arduino-esp32.git#2.0.2
board = esp32dev
board_build.partitions = min_spiffs.csv
framework = arduino
build_flags =
    -DBOARD_HAS_PSRAM
    -mfix-esp32-psram-cache-issue

Firmware: Mar 14 2022 @ 10:23:19 (base: 4.0.rc1)
Sketch Size: 1020544 (total: 1966080, 51.0% used)
ESP sdk: v4.4-beta1-189-ga79dc75f0a

I note that the Platformio build with (nominally) the same SDK is 16k or so bigger in size. so the two builds are not equivalent despite the SDK giving the same version.

I tried a number of ways of configuring the platform, platform_packages and framework fields in platform.ini to no avail, either the build would fail or the error would persist. Using the latest (master) tag for the arduino-esp32 package seemed to make the problem worse. I also tried other frame sizes, disabling OTA, using a different AP and browser. Nothing worked.

https://github.com/search?q=ESP_ERR_HTTPD_RESP_SEND&type=issues shows that this is a pretty unique error.
Googling ESP_ERR_HTTPD_RESP_SEND isn't much help either.

As noted above; I will not close this; but I do not have any more time to work on it.
I'll document that PlatformIO builds are broken for the 4.0 release; but for the present I wont go any further.
If anybody wants to pursue this as a PlatformIO bug/question please go ahead; I'll assist where I can and hotfix any resolution.

[TheWolf95]

Hi!

Sorry for not answering before. I can confirm I was using PlatformIO to build and flash the code on ESP32-CAM and it wouldn't work.
Some time later I tried using Arduino IDE, as the included example was working fine (the simple one included with ESP libraries for Arduino), and it happened to work just fine. No more crashes and no more error, so the issue is somehow strictly due to PIO.

@Bulduper, I had the same issue too. You could simply disable custom config options by uncommenting #define NO_FS around line 165 of myconfig.h. Instead, if you want to use this code, for now, you have to use Arduino IDE and select under Tools > Partition scheme > Default 4MB or Minimal. This does exclude OTA, but works for me.
Also by commenting with ; before board_build.partitions = min_spiffs.csv in platformIO config does the job too, but stream crashes due to other issues.

@easytarget thank you for taking the time to investigate, as for now using ArduinoIDE is not an issue for me, so I guess we will just wait for PIO to fix their issues

[Bulduper]

Thank you @TheWolf95 ! Indeed your solution for IDE worked.

[FKW9]

Hello!

I have a working version built with platformio (v4.0 release).
The only things I've changed are the following:

• in platformio.ini, i simply use platform = espressif32 and commented out platform_packages
• in the main file i commented out line 383 config.grab_mode = CAMERA_GRAB_LATEST; since i got an "... is not defined" error

This program is up and running for several hours now and no problems.
How can i contribute to solve this issue?
Greetings

[TheHoodedFoot]

Thank you @mitsubishievo99 - I can confirm that making those changes worked for me, and that building with PlatformIO now results in a working program.

[marspd]

Thank you @mitsubishievo99 - I can ALSO confirm that making those changes worked for me, and that building with PlatformIO now results in a working program.

[easytarget]

@mitsubishievo99 @TheHoodedFoot @marspd
Yes; this is very valid as a workaround for the time being. But be warned that it may break in the future.

What you are doing is relying on the fact that the PlatformIO espressif32 package provides a older version of the Ardionio Core (V2.0.0, I believe, maybe 2.0.1, or some commit between the two).
This earlier Arduino Core has an older camera driver which doesnt support the .grab_mode, so you have to comment that out too.
If and when the platformio esp32 package is updated to the latest core this will break again, but (what with one thing and another) I'm not expecting this to happen any time soon. I'm actually investigating the root issue separately, so I hope to have a working platformio build eventually.

[llamaonaskateboard]

The latest git version of framework-arduinoespressif32 (appears to be a RC of 2.0.3) has mysteriously fixed this issue for me.

platform = https://github.com/platformio/platform-espressif32.git#feature/arduino-upstream
platform_packages = framework-arduinoespressif32@https://github.com/espressif/arduino-esp32.git

PLATFORM: Espressif 32 (3.3.1+sha.3784198) > Espressif ESP32 Dev Module
HARDWARE: ESP32 240MHz, 320KB RAM, 4MB Flash
DEBUG: Current (esp-prog) External (esp-prog, iot-bus-jtag, jlink, minimodule, olimex-arm-usb-ocd, olimex-arm-usb-ocd-h, olimex-arm-usb-tiny-h, olimex-jtag-tiny, tumpa)
PACKAGES:
 - framework-arduinoespressif32 2.0.3+sha.b3c203d
 - tool-esptoolpy 1.30100.210531 (3.1.0)
 - tool-mkspiffs 2.230.0 (2.30)
 - toolchain-xtensa-esp32 8.4.0+2021r2-patch3

[easytarget]

Curious; thankyou for the update @llamaonaskateboard , I will re-test when I can, it would be nice if this has been a transient issue.

[easytarget]

Ok, so with the latest Arduino core (2.0.3) specified the problem is resolved.

• Streams are now stable for hours

Thank you again to all the folks who contributed and assisted with this.