Repository navigation
Camera: expose two-way audio support in camera capabilities #1511
ContextMany cameras have a speaker and support two-way audio (talk-back), for example Reolink cameras and doorbells. Home Assistant can't currently tell the frontend that a camera supports this. Without that information, the frontend can't decide whether to show a "talk" / microphone control in the camera card or more-info dialog. WebRTC already supports this. When the client sends an offer with a Whether talk-back works end-to-end depends on three things:
The frontend already requests Implementation PR: home-assistant/core#184716 ProposalAdd a {
"frontend_stream_types": ["hls", "web_rtc"],
"two_way_audio": true
}Entity APIIntegrations declare capabilities with a new frozen dataclass, set through the usual @dataclass(frozen=True, kw_only=True)
class CameraEntityCapabilities:
"""Capabilities declared by a camera entity."""
two_way_audio: bool = False
class Camera(Entity):
_attr_camera_entity_capabilities: CameraEntityCapabilities = (
CameraEntityCapabilities()
)
@cached_property
def camera_entity_capabilities(self) -> CameraEntityCapabilities:
"""Return the capabilities declared by the camera entity."""
return self._attr_camera_entity_capabilitiesThe dataclass is the home for camera capabilities that aren't related to actions. Requirement: an integration must only set Example (Reolink): the user can choose RTSP, RTMP or FLV, and only RTSP carries the backchannel: if "snapshots" not in entity_description.stream:
self._attr_supported_features = CameraEntityFeature.STREAM
self._attr_camera_entity_capabilities = CameraEntityCapabilities(
two_way_audio=(
self._host.api.protocol == "rtsp"
and self._host.api.supported(self._channel, "two_way_audio")
)
)Why a separate
|
Entity two_way_audio |
WebRTC available | Result two_way_audio |
|---|---|---|
False |
any | false |
True |
no (HLS only) | false |
True |
yes, through a provider | true |
True |
yes, native | true |
camera_capabilities is no longer cached. It only does a few cheap checks, and its callers run once per camera open, WebRTC session or media source lookup. Without the cache, it always reflects the current entity capabilities, stream support and WebRTC provider, so there is no invalidation to keep in sync. The property stays public and @final.
If an integration changes its stream source at runtime without reloading, it must update _attr_camera_entity_capabilities to match the new source. Integrations usually change the protocol in an options flow that reloads the config entry (Reolink uses OptionsFlowWithReload), which rebuilds the entity anyway.
Frontend
When two_way_audio is true, the frontend can show a talk control. While talking, it adds an audio transceiver with direction sendrecv (and the microphone track) to the WebRTC offer. The existing camera/webrtc/offer signaling path stays the same.
Implementation
- Adds
CameraEntityCapabilitiesand_attr_camera_entity_capabilitiestoCamera. - Adds
two_way_audiotoCameraCapabilitiesand reports it only when WebRTC is available (native or through a provider). - Removes the caching of
camera_capabilities. - Implements it for Reolink (RTSP only).
Consequences
- Purely additive.
two_way_audiodefaults toFalse, so existing integrations, providers and the current frontend are unaffected. - The websocket response gets one more key. Clients that compare the full payload must allow for it.
- Integrations have to account for their configured stream protocol, not just the device model, when declaring two-way audio.
- Removing the cache also removes the private
_invalidate_camera_capabilities_cachehelper. Nothing outside the camera component uses it. - There is no check for whether the WebRTC provider supports two-way audio for the stream source. A provider that can't forward audio to a given source would therefore still lead to
true. This check is out of scope for now:- Not needed: the entity capability already guarantees that the stream source carries a backchannel, and all existing providers can forward it (see Context).
- Bigger refactoring: whether a provider supports two-way audio depends on the camera's stream source.
stream_source()is async, butcamera_capabilitiesis a sync property. A provider check would need either storing the provider's answer when the provider is selected inasync_refresh_providers, or turningcamera_capabilitiesinto an async method, which would break the public API. - Non-breaking path: if such a provider appears, a provider method that it overrides (e.g.
async_supports_two_way_audio, defaulting toTrue) can be added later without breaking existing integrations or providers.
Changes since the first version
- The open question is decided in favour of the dataclass. Integrations now set
_attr_camera_entity_capabilities = CameraEntityCapabilities(...)instead of_attr_supports_two_way_audio. A new section explains whyCameraCapabilitiescan't be reused for this. supports_two_way_audiois renamed totwo_way_audioin both the entity dataclass and the websocket result. Inside a capabilities object, thesupports_prefix adds nothing.camera_capabilitiesis no longer cached, so the "Cache invalidation" section is gone. The cache saved almost nothing and had to be invalidated in several places.- The "Open questions" section is removed because its only question is decided.
Replies: 3 comments 2 replies
|
Looks good to me, only the _attr_camera_entity_capabilities vs _attr_supports_two_way_audio is something that needs a decission. But I am also fine with putting them together in a dataclass. Out of scope for this proposal, but relevant for the context: |
|
I see the benefit of a dataclass that groups all capabilities together and you get all capabilities in one go instead of needing to call each property |
|
Fine with me, I will change it to a dataclass once this is agreed appon and the proposal is approved. |
|
The architecture meeting discussed this proposal on 2026-10-15 and approved it with the note that the shape of the entity API for this feature should be clarified. As that's done now, and is in line with the meeting sentiment, the proposal is approved. |
The architecture meeting discussed this proposal on 2026-10-15 and approved it with the note that the shape of the entity API for this feature should be clarified. As that's done now, and is in line with the meeting sentiment, the proposal is approved.