Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Very low FPS when trying the AsyncCam example #34

Open
chewbaz1 opened this issue Jun 26, 2022 · 8 comments
Open

Very low FPS when trying the AsyncCam example #34

chewbaz1 opened this issue Jun 26, 2022 · 8 comments

Comments

@chewbaz1
Copy link

chewbaz1 commented Jun 26, 2022 •
edited

Hello

I tried using the AsyncCam example but I get very poor fps (1-2fps). When i use the standard example from espressif I get normal fps. Changing resolution doesnt help. I have an AI Thinker board and it gets very warm with this example. I am using PlatformIO and have tried upload with both AI thinker setting and Wrover setting but fps is bad on both. When I tried using arduino IDE i got a lot of errors therefore I use platformio instead.

Power: 5V

I would be happy if I recieved help with this issue.

Thanks
@ery346
Copy link

ery346 commented Oct 9, 2022

It is a very good library but I am also having the same problem

@agus0
Copy link

agus0 commented Oct 19, 2023

Same problem. cant made a HTTP+JS web joystick for RC Car via websockets with embebed stream video

@Matt-Stedman
Copy link

Same here. I can get a v. good frame rate if I'm on StreamMjpeg, but the async example is depressingly slow!

@yoursunny
Copy link
Owner

When i use the standard example from espressif I get normal fps.

Do they offer an example with AsyncTCP?
If so, can you post a link?

@ForrestFire0
Copy link

I can confirm that the frame rate on the Async example from this repo is quite poor :(. Even when lowering the resolution.

To capture a static jpeg is actually quite fast. I modified the example to have a download button and the download is almost instant.

I attempted to characterize the performance of the library - I found in esp32cam-asyncweb.h this line auto frame = Camera.capture().release(); I slapped a println there and disappointingly found that it is only capturing a frame once per second.

Using platformio with the following settings:

[env:esp32cam]
platform = https://github.com/Jason2866/platform-espressif32.git#Arduino/IDF5
board = esp32cam
framework = arduino
upload_speed = 921600
monitor_speed = 115200
monitor_rts = 0
monitor_dtr = 0

lib_deps =
    AsyncTCP
    ESP Async WebServer
    yoursunny/esp32cam

Using this board.

@bsdshneg
Copy link

+1
very slow video ~1fps

@yoursunny
Copy link
Owner

yoursunny commented Jun 18, 2024 •
edited

I added some debug logging in MjpegResponse and can have some insights in where it's spending the most time.
In my location the 2.4GHz network is very noisy, such that the home router indicates "RX rate 6 Mbit/s" from the ESP32.
I tested with 640x480 resolution for 30 seconds and got 1.393932 fps.

full serial logs 
[   32941] MjpegResponse(0x3ffdb8ec) created
[   32942] MjpegResponse(0x3ffdb8ec) capturing
[   33417] MjpegResponse(0x3ffdb8ec) frame has 7352 octets
[   33532] MjpegResponse(0x3ffdb8ec) sent to client
[   33917] MjpegResponse(0x3ffdb8ec) capturing
[   34417] MjpegResponse(0x3ffdb8ec) frame has 7357 octets
[   34513] MjpegResponse(0x3ffdb8ec) sent to client
[   34517] MjpegResponse(0x3ffdb8ec) capturing
[   34917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   34999] MjpegResponse(0x3ffdb8ec) sent to client
[   35005] MjpegResponse(0x3ffdb8ec) capturing
[   35417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   35701] MjpegResponse(0x3ffdb8ec) sent to client
[   35706] MjpegResponse(0x3ffdb8ec) capturing
[   35917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   36010] MjpegResponse(0x3ffdb8ec) sent to client
[   36417] MjpegResponse(0x3ffdb8ec) capturing
[   36917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   37266] MjpegResponse(0x3ffdb8ec) sent to client
[   37269] MjpegResponse(0x3ffdb8ec) capturing
[   37417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   37505] MjpegResponse(0x3ffdb8ec) sent to client
[   37514] MjpegResponse(0x3ffdb8ec) capturing
[   37917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   38361] MjpegResponse(0x3ffdb8ec) sent to client
[   38417] MjpegResponse(0x3ffdb8ec) capturing
[   38917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   39029] MjpegResponse(0x3ffdb8ec) sent to client
[   39033] MjpegResponse(0x3ffdb8ec) capturing
[   39417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   39514] MjpegResponse(0x3ffdb8ec) sent to client
[   39514] MjpegResponse(0x3ffdb8ec) sent to client
[   39917] MjpegResponse(0x3ffdb8ec) capturing
[   40417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   40513] MjpegResponse(0x3ffdb8ec) sent to client
[   40519] MjpegResponse(0x3ffdb8ec) capturing
[   40917] MjpegResponse(0x3ffdb8ec) frame has 7148 octets
[   41058] MjpegResponse(0x3ffdb8ec) sent to client
[   41417] MjpegResponse(0x3ffdb8ec) capturing
[   41917] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   42172] MjpegResponse(0x3ffdb8ec) sent to client
[   42173] MjpegResponse(0x3ffdb8ec) capturing
[   42417] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   42501] MjpegResponse(0x3ffdb8ec) sent to client
[   42917] MjpegResponse(0x3ffdb8ec) capturing
[   43417] MjpegResponse(0x3ffdb8ec) frame has 7166 octets
[   43754] MjpegResponse(0x3ffdb8ec) sent to client
[   43918] MjpegResponse(0x3ffdb8ec) capturing
[   43921] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   44006] MjpegResponse(0x3ffdb8ec) sent to client
[   44013] MjpegResponse(0x3ffdb8ec) capturing
[   44418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   44562] MjpegResponse(0x3ffdb8ec) sent to client
[   44563] MjpegResponse(0x3ffdb8ec) capturing
[   44904] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   44947] MjpegResponse(0x3ffdb8ec) sent to client
[   45002] MjpegResponse(0x3ffdb8ec) capturing
[   45418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   45520] MjpegResponse(0x3ffdb8ec) sent to client
[   45522] MjpegResponse(0x3ffdb8ec) capturing
[   45918] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   46014] MjpegResponse(0x3ffdb8ec) sent to client
[   46020] MjpegResponse(0x3ffdb8ec) capturing
[   46418] MjpegResponse(0x3ffdb8ec) frame has 7151 octets
[   46501] MjpegResponse(0x3ffdb8ec) sent to client
[   46918] MjpegResponse(0x3ffdb8ec) capturing
[   47418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   47607] MjpegResponse(0x3ffdb8ec) sent to client
[   47918] MjpegResponse(0x3ffdb8ec) capturing
[   48418] MjpegResponse(0x3ffdb8ec) frame has 7125 octets
[   48502] MjpegResponse(0x3ffdb8ec) sent to client
[   48918] MjpegResponse(0x3ffdb8ec) capturing
[   49418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   49514] MjpegResponse(0x3ffdb8ec) sent to client
[   49918] MjpegResponse(0x3ffdb8ec) capturing
[   50418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   50561] MjpegResponse(0x3ffdb8ec) sent to client
[   50566] MjpegResponse(0x3ffdb8ec) capturing
[   50918] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   51041] MjpegResponse(0x3ffdb8ec) sent to client
[   51059] MjpegResponse(0x3ffdb8ec) capturing
[   51418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   51512] MjpegResponse(0x3ffdb8ec) sent to client
[   51918] MjpegResponse(0x3ffdb8ec) capturing
[   52418] MjpegResponse(0x3ffdb8ec) frame has 7356 octets
[   52801] MjpegResponse(0x3ffdb8ec) sent to client
[   52809] MjpegResponse(0x3ffdb8ec) capturing
[   52918] MjpegResponse(0x3ffdb8ec) frame has 7151 octets
[   53207] MjpegResponse(0x3ffdb8ec) sent to client
[   53418] MjpegResponse(0x3ffdb8ec) capturing
[   53553] MjpegResponse(0x3ffdb8ec) frame has 7161 octets
[   53664] MjpegResponse(0x3ffdb8ec) sent to client
[   53918] MjpegResponse(0x3ffdb8ec) capturing
[   54418] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   54529] MjpegResponse(0x3ffdb8ec) sent to client
[   54736] MjpegResponse(0x3ffdb8ec) capturing
[   54918] MjpegResponse(0x3ffdb8ec) frame has 7160 octets
[   54998] MjpegResponse(0x3ffdb8ec) sent to client
[   55418] MjpegResponse(0x3ffdb8ec) capturing
[   55918] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   56000] MjpegResponse(0x3ffdb8ec) sent to client
[   56001] MjpegResponse(0x3ffdb8ec) capturing
[   56418] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   56495] MjpegResponse(0x3ffdb8ec) sent to client
[   56497] MjpegResponse(0x3ffdb8ec) capturing
[   56918] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   57019] MjpegResponse(0x3ffdb8ec) sent to client
[   57020] MjpegResponse(0x3ffdb8ec) capturing
[   57418] MjpegResponse(0x3ffdb8ec) frame has 7150 octets
[   57507] MjpegResponse(0x3ffdb8ec) sent to client
[   57918] MjpegResponse(0x3ffdb8ec) capturing
[   58418] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   58524] MjpegResponse(0x3ffdb8ec) sent to client
[   58918] MjpegResponse(0x3ffdb8ec) capturing
[   59418] MjpegResponse(0x3ffdb8ec) frame has 7160 octets
[   59695] MjpegResponse(0x3ffdb8ec) sent to client
[   59755] MjpegResponse(0x3ffdb8ec) capturing
[   59918] MjpegResponse(0x3ffdb8ec) frame has 7765 octets
[   60270] MjpegResponse(0x3ffdb8ec) sent to client
[   60291] MjpegResponse(0x3ffdb8ec) capturing
[   60418] MjpegResponse(0x3ffdb8ec) frame has 7143 octets
[   60630] MjpegResponse(0x3ffdb8ec) sent to client
[   60918] MjpegResponse(0x3ffdb8ec) capturing
[   61418] MjpegResponse(0x3ffdb8ec) frame has 7164 octets
[   61580] MjpegResponse(0x3ffdb8ec) sent to client
[   61628] MjpegResponse(0x3ffdb8ec) capturing
[   61918] MjpegResponse(0x3ffdb8ec) frame has 7164 octets
[   62021] MjpegResponse(0x3ffdb8ec) sent to client
[   62418] MjpegResponse(0x3ffdb8ec) capturing
[   62918] MjpegResponse(0x3ffdb8ec) frame has 7161 octets
[   63058] MjpegResponse(0x3ffdb8ec) sent to client
[   63130] MjpegResponse(0x3ffdb8ec) capturing
[   63418] MjpegResponse(0x3ffdb8ec) frame has 7152 octets
[   63789] MjpegResponse(0x3ffdb8ec) deleted after 43 frames at fps 1.393932

Some spreadsheet calculation reveals the average wait time:

  1. Waiting to start capture: 133ms
  2. Camera performing the capture: 224ms
  3. Sending the frame to the client: 336ms

In the current codebase, the MjpegController type can only make use of one Frame, even if multiple frames are created via Config::setBufferCount.
If this limitation is resolved, the duration for "camera performing the capture" and "sending the frame to the client" can be overlapped, which would greatly improve the fps.

Separately, it should be possible to refactor MjpegResponse type to reduce or eliminate the "waiting to start capture" duration.
While this is a smaller improvement, it is much easier to achieve.

yoursunny added a commit that referenced this issue Jun 18, 2024
@yoursunny
asyncweb: reduce wait between SEND and CAPTURE
00d1ddb 
refs #34
@yoursunny
Copy link
Owner

yoursunny commented Jun 18, 2024 •
edited

I tried to reduce the SEND-CAPTURE wait in 00d1ddb.
Then I retested 640x480 resolution for 30 seconds, same as above.

full serial logs 
[  152698] MjpegResponse(0x3ffdc148) created
[  152698] MjpegResponse(0x3ffdc148) capturing
[  153189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  154022] MjpegResponse(0x3ffdc148) sent to client
[  154023] MjpegResponse(0x3ffdc148) capturing
[  154032] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  154640] MjpegResponse(0x3ffdc148) sent to client
[  154640] MjpegResponse(0x3ffdc148) capturing
[  154689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  155811] MjpegResponse(0x3ffdc148) sent to client
[  155811] MjpegResponse(0x3ffdc148) capturing
[  156142] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  157066] MjpegResponse(0x3ffdc148) sent to client
[  157066] MjpegResponse(0x3ffdc148) capturing
[  157142] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  158189] MjpegResponse(0x3ffdc148) sent to client
[  158189] MjpegResponse(0x3ffdc148) capturing
[  158689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  159553] MjpegResponse(0x3ffdc148) sent to client
[  159553] MjpegResponse(0x3ffdc148) capturing
[  159689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  161121] MjpegResponse(0x3ffdc148) sent to client
[  161121] MjpegResponse(0x3ffdc148) capturing
[  161189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  161959] MjpegResponse(0x3ffdc148) sent to client
[  161959] MjpegResponse(0x3ffdc148) capturing
[  162146] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  162577] MjpegResponse(0x3ffdc148) sent to client
[  162577] MjpegResponse(0x3ffdc148) capturing
[  162689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  163189] MjpegResponse(0x3ffdc148) sent to client
[  163189] MjpegResponse(0x3ffdc148) capturing
[  163237] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  164896] MjpegResponse(0x3ffdc148) sent to client
[  164897] MjpegResponse(0x3ffdc148) capturing
[  165189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  165713] MjpegResponse(0x3ffdc148) sent to client
[  165713] MjpegResponse(0x3ffdc148) capturing
[  165743] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  166330] MjpegResponse(0x3ffdc148) sent to client
[  166330] MjpegResponse(0x3ffdc148) capturing
[  166338] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  167039] MjpegResponse(0x3ffdc148) sent to client
[  167039] MjpegResponse(0x3ffdc148) capturing
[  167162] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  168189] MjpegResponse(0x3ffdc148) sent to client
[  168189] MjpegResponse(0x3ffdc148) capturing
[  168689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  170725] MjpegResponse(0x3ffdc148) sent to client
[  170725] MjpegResponse(0x3ffdc148) capturing
[  170776] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  171578] MjpegResponse(0x3ffdc148) sent to client
[  171578] MjpegResponse(0x3ffdc148) capturing
[  171689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  172427] MjpegResponse(0x3ffdc148) sent to client
[  172427] MjpegResponse(0x3ffdc148) capturing
[  172514] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  173189] MjpegResponse(0x3ffdc148) sent to client
[  173189] MjpegResponse(0x3ffdc148) capturing
[  173196] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  173680] MjpegResponse(0x3ffdc148) sent to client
[  173680] MjpegResponse(0x3ffdc148) capturing
[  173689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  174072] MjpegResponse(0x3ffdc148) sent to client
[  174072] MjpegResponse(0x3ffdc148) capturing
[  174189] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  174563] MjpegResponse(0x3ffdc148) sent to client
[  174563] MjpegResponse(0x3ffdc148) capturing
[  174689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  175366] MjpegResponse(0x3ffdc148) sent to client
[  175366] MjpegResponse(0x3ffdc148) capturing
[  175479] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  176189] MjpegResponse(0x3ffdc148) sent to client
[  176189] MjpegResponse(0x3ffdc148) capturing
[  176419] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  177189] MjpegResponse(0x3ffdc148) sent to client
[  177189] MjpegResponse(0x3ffdc148) capturing
[  177271] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  177594] MjpegResponse(0x3ffdc148) sent to client
[  177594] MjpegResponse(0x3ffdc148) capturing
[  177689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  178856] MjpegResponse(0x3ffdc148) sent to client
[  178856] MjpegResponse(0x3ffdc148) capturing
[  179163] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  180030] MjpegResponse(0x3ffdc148) sent to client
[  180030] MjpegResponse(0x3ffdc148) capturing
[  180186] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  181577] MjpegResponse(0x3ffdc148) sent to client
[  181577] MjpegResponse(0x3ffdc148) capturing
[  181689] MjpegResponse(0x3ffdc148) frame has 7922 octets
[  182973] MjpegResponse(0x3ffdc148) deleted after 29 frames at fps 0.957886

According to the spreadsheet, each step takes average time as follows:

  1. Waiting to start capture: 0ms
  2. Camera performing the capture: 140ms
  3. Sending the frame to the client: 857ms

The change is effective in eliminating the SEND-CAPTURE delay, but the duration spent within SEND state is much longer.
I can't tell whether it's due to WiFi interference or something in the code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@agus0 @yoursunny @bsdshneg @ForrestFire0 @Matt-Stedman @ery346 @chewbaz1