Working s3 or p4 camera?

[n-IA-hane]

Does anyone have any working configuration for Freenove ESP32 cameras? Or has anyone managed to get the camera working on ESP32-P4? They’d be really useful for me to start thinking about a video intercom

[Psix-anp]

I'm testing the camera on the Waveshare ESP32-P4-DevKit. Or rather, not me, haha — Claude Code is handling it. But apparently it's fixed the problem, and both photos and videos are working now.

[n-IA-hane]

HOOOOOOOOOOOWWWWWWWW?????

[Psix-anp]

esp_video_camera.zip
AI fork component

And this my stand + HA interface

[n-IA-hane]

Hi @Psix-anp, your esp_video_camera.zip really was the piece that unlocked
video on the ESP32-P4 for us. Thank you again for sharing it.

Starting from your snapshot, we now have three working P4 profiles:

1. Full ESPHome profile: the native ESPHome camera is exposed to Home
   Assistant while SIP carries bidirectional audio. This is intended for the
   full dashboard/voice-assistant firmware and gives us P4 camera video in the
   HA call card without making the P4 decode remote video.
2. JPEG SIP videophone: bidirectional SIP audio and standards-based RTP/JPEG
   (payload type 26). The P4 sends its hardware-JPEG camera and also decodes and
   renders the remote video on its display.
3. Experimental H.264 SIP videophone: bidirectional SIP audio/video with a
   hardware H.264 encoder for P4-to-peer video and a constrained software
   decoder path for peer-to-P4 video. It works end-to-end, although the RX path
   is still experimental on our ESP32-P4 rev 1.3 hardware.

Your component immediately solved the camera/P4-to-HA half. We then worked on
the other half as well: SIP/SDP/RTP integration and rendering the remote caller
on the P4 display.

The attached component is a working handoff based on the ZIP you posted here.
The main local changes are:

• proper ESPHome camera::Camera code-generation integration;
• Espressif esp_video 2.3 integration and compile-time JPEG/H.264 dependency
  gating;
• PPA hardware rotation/scaling for the P4 camera path;
• V4L2 framerate configuration with a software fallback;
• reusable STREAMON/STREAMOFF capture state, bounded retry/shutdown and safer
  DMA/PSRAM ownership;
• synchronous borrowed JPEG and RGB565 consumers, so SIP codecs can reuse the
  existing camera pipeline without opening the sensor twice;
• optional debug counters for camera/PPA/JPEG timing.

The SIP packetizers, H.264 adapter and P4 display renderer remain separate
components; they were not folded into your camera component.

Would you still be interested in publishing and maintaining this component in
your own repository, with the license and history you consider appropriate?
We have deliberately not released this camera fork as our own project: we would
prefer your repository to be the upstream, because your shared component is
what unlocked P4 video for us. Once it is available, we can split these changes
into reviewable pull requests instead of maintaining another unrelated fork.

This ZIP is a working development handoff rather than a polished upstream
release:

esp_video_camera-p4-sip-handoff-20260728.zip

SHA-256: d49f99eec7d49bf9c0a460b0ddfcb8ee50df8b39ebbe22f3f6e60a541b657904

[n-IA-hane]

A quick visual follow-up from today's P4 qualification.

Home Assistant side: the VoIP call card is showing the live P4 camera feed during the active call.

P4 side: this phone recording shows the Waveshare P4 display receiving, decoding and presenting a direct SIP/RTP H.264 stream.

Watch the 19.7-second P4 H.264 display demo

For the complete 44.8-second call, the device reported 212 frames admitted and rendered, 211 presented, zero decode drops and a clean remote BYE. That is about 4.7 displayed fps in this conservative moving-content test.

Test footage: Big Buck Bunny, Blender Foundation.

[n-IA-hane]

Yessss, it works! I just tried it on my P4 Touch :D You know I’m going straight into testing now to build a P4 video phone, right?

[Psix-anp]

Yeeeaaahhhh, good!

[Psix-anp]

All that's left is to display the camera feed from the call panel onto the Waveshare ESP32-P4-Touch-4B screen, so that it becomes a full-fledged intercom. But there's an MJPEG decoder involved. I've basically managed to get it working, but I still need to work on it some more.

[n-IA-hane]

Hey! Your component works great on our Waveshare P4. Seeing the camera finally come alive was fantastic 😄

We’d love to experiment with extending it toward optional hardware H.264 output for a real SIP video phone, while keeping your work and attribution clearly recognizable.

Would you be willing to publish the component in one of your GitHub repositories, ideally with a license and an initial version tag? We could then fork it and send any improvements back to you through proper pull requests.

Thanks again for sharing it. It opened up a very interesting path for the project!

[Psix-anp]

Thanks, I'll get to it tomorrow or the day after. I'm currently out of town.

Test video on waveshare 4b (4inch)

[Psix-anp]

1000116958_20260726190406.mp4

[Psix-anp]

Yes, let's do that. It's up: https://github.com/Psix-anp/esphome-esp-video-camera — v0.1.0 is tagged and released.

License is the ESPHome split, GPLv3 for the C++ and MIT for the Python side, since the whole thing is derived from ESPHome and from your PR. NOTICE spells out where everything came from: ESPHome, @youkorr as the original author of the component and of esphome/esphome#16944, and my changes on top. @youkorr stays as the code owner, and every place I touched in the C++ carries a // FORK: comment, which is what GPLv3 asks for anyway.

I kept v0.1.0 to only what I could actually test on my own board (Waveshare P4 PoE + OV5647, JPEG at 1280x960): the capture-side S_FMT fix, moving capture off loopTask into its own task — that was the reboot loop as soon as Home Assistant started polling the entity — the DQBUF ordering, control writes going through S_EXT_CTRLS, the runtime controls, and the warm-up that stops the first frame from coming out black. Nothing I couldn't verify on hardware.

Your handoff is running on my panel right now. Two things stood out immediately: WARMUP_MS instead of my fixed ten-frame count cut the cold frame from about 2 seconds to 0.35, and the esp_h264 stub keeps H.264 out of a JPEG-only build entirely. I'm feeding an MJPEG endpoint from it in my own firmware and getting around 14 fps at 1280x960 with no watchdog trouble. I'll report back properly once it has a few days on it.

Please send the rest as PRs — I'd rather your name stayed on your own work. Split them however is natural for you; from the outside it looks like esp_video 2.3 plus the stub, the camera::Camera codegen, the two frame consumers, rotation, and then the warm-up/retries/logging group. I'll review and merge.

One thing I need to get right: is the buffer in consume_jpeg_frame() only valid for the duration of the call — copy immediately, never block — or does it stay mapped a little longer? I'm copying defensively at the moment, but if I don't have to, that's 100 KB per frame I'd rather not move around.

And the SIP/RTP H.264 going straight to the P4 display — I didn't expect that to be working this soon. Nice one.

[youkorr]

Great work, and thanks for the H.264; I've been working on the High Profile for quite a while—it's not perfect, but it works.

[youkorr]

Your project is nice.
I did work very similar to yours—WebRTC audio/video using a custom signaling Docker container I had to create (instead of the Espressif one)—along with a doorbell that is currently undergoing testing.

[youkorr]

Rtsp

80690544900__2C4E2B28-0473-41F2-BCF1-CCC0EE042F47.mov

[youkorr]

80686434456__55F804E0-C105-402D-ACB2-F448E156BB7A.mov

[youkorr]

#18 (reply in thread)
I really like your camera map for Home Assistant; how did you make it?

[n-IA-hane]

Hi @Psix-anp, the six camera changes you requested are now ready for review:

1. PR #1 fixes the esp_video_init() timeout lifetime.
2. PR #2 fixes the ESPHome camera codegen inheritance.
3. PR #3 adds borrowed raw and JPEG frame consumers.
4. PR #4 adds PPA rotation and downscaling.
5. PR #5 adds the event-driven capture lifecycle.
6. PR #6 updates esp_video and isolates JPEG-only dependencies.

Each PR is isolated against the current main, and its exact validation status
is documented in the description. The remaining connected-hardware checks for
PRs #5 and #6 are stated explicitly. Since several changes touch the same capture
code, I can rebase the remaining branches after each merge to keep the review
clean.

[youkorr]

Hi n-IA-hane, I’ve reviewed part of your work—it’s very good. You should try to avoid PR duplicates, though—not to offend you—and your tests were validated on a single Waveshare P4 with an OV5647 sensor (presumably a P4 revision 1.3). You’re likely aware of the complexities regarding the PPA on certain revisions.

[n-IA-hane]

Thanks for pointing this out, and sorry for the noisy PR layout. No offense taken. I have used GitHub for several months, but this is my first time managing a set of interdependent PRs. I split the work by concern while basing every branch on main, so changes around the capture loop overlap and appear repeatedly. I understand now why that makes review and merging harder.

I will not open more camera PRs for now. After the first accepted change is merged, I will rebase the next branch onto the updated main, remove anything already landed, and proceed one PR at a time in the order @Psix-anp prefers.

One small hardware correction: our qualification device is indeed an ESP32-P4 revision 1.3, but its sensor is SC202CS in the native 800x800 mode, not OV5647. You are right that this is still validation on one P4 revision and does not establish PPA behaviour on other revisions. I will keep that limitation explicit.

Thanks for reviewing the work and for warning us about the revision-specific PPA complexities.

[n-IA-hane]

@Psix-anp and @youkorr, before I change any of the open branches, which review structure would you prefer?

1. Sequential PRs: keep one PR active, then after each merge rebase the next branch onto the updated main so its diff contains only the next incremental change.
2. One consolidated PR: keep the truly independent fixes separate, but close the overlapping PRs Add hacs.json #3, Enable ESP32P4 for esp_aec #4 and Add audio limits to duplex mic and speaker to squash compile failures #5 and replace them with one ordered PR.

I lean toward the sequential approach, but I will follow your preference. Until you answer, I will not rewrite or close any branch.

[Psix-anp]

Sequential, one PR at a time — that is the format I prefer. A single consolidated PR of ~900 lines would not get a real review from anyone, and @youkorr clearly wants to look at the PPA path carefully.

Status on my side: #1, #2 and #6 are merged. #3, #4 and #5 now conflict against the updated main — please rebase them and I will take them in the same order.

One correction that matters for the validation caveat: the camera running here is an OV5647, not SC202CS. The firmware pins it explicitly:

CONFIG_CAMERA_OV5647_MIPI_RAW10_1280X960_BINNING_45FPS: y
CONFIG_CAMERA_OV5647_MIPI_DEFAULT_FMT_RAW10_1280X960_BINNING_45FPS: y

So the PPA transform has been exercised on two different sensors, not one. On this side it is not a lab test either — the code in #3/#4 is what my door panel has been running in production at the gate: 1280x960 binned capture, PPA downscale and 720x720 JPEG served to Home Assistant and to the panel's own web UI, all day, every day.

What I cannot confirm is the other half of @youkorr's point: everything here is a single P4 revision. If PPA behaves differently on another revision, that will have to come from someone with that hardware.

(Review and merge handled with AI assistance on my side.)

[n-IA-hane]

Thanks @Psix-anp and @youkorr. Sequential, one PR at a time, is clear. I will rebase only the next PR on the current main, remove changes that have already been merged, and continue in the requested order. I am holding back the final release refinement until the camera work is settled in its upstream repository, so the release can use the maintained component instead of carrying another fork. As soon as the component is complete and stable there, I will update the relevant external_components declarations in my YAML profiles to use Psix-anp/esphome-esp-video-camera directly.

Also, apologies for the noisy first set of PRs. I am relying heavily on AI assistance for the GitHub workflow because I do not yet have much experience managing a chain of related PRs. If I misunderstand a convention or organize something poorly again, please tell me directly. No offense will be taken, and candid feedback is very welcome.

On the technical side, the integration is working unexpectedly well. We now have a P4 JPEG firmware profile that combines the full ESPHome assistant functionality, including LVGL, micro wake word, voice assistant, AEC and AFE, music and Sendspin, with bidirectional SIP audio and video. Some services are paused during video calls where necessary to protect realtime resources, but they all live in the same full profile and resume afterward.

That result is honestly exciting. Nine months ago this project had one way audio running through ESPHome. Now the same device can operate as a complete voice assistant and a bidirectional video phone. It is a major milestone, and your camera work is a central part of what made it possible. Thank you both.

[youkorr]

I saw your work on bidirectional audio—bravo, it works well and I recommend it, even though I had done similar, simplified work involving analog, digital, and the ES8388 DAC.

[n-IA-hane]

Hi @Psix-anp and @youkorr, have you had time to evaluate
PR #5?

It is now the only remaining PR in the camera series. It is rebased on the
current main and GitHub reports it as cleanly mergeable. Please let me know if
you need any further changes or validation from my side.

[Psix-anp]

Sorry, my friend! It’s just summer — I’ve been busy and haven’t had much time. I’m relaxing by the sea right now 😄)))