Full-duplex voice + wake-word access on Ray-Ban Meta — four SDK capability questions

[Asgermolgaard]

im building with the SDK and want to hear if support full-duplex barge-in — i.e.,
the user can interrupt the AI's TTS mid-speech by talking, and the AI stops within ~200ms. Today most
voice apps on this hardware ship serial speak-then-listen because the default A2DP-for-TTS /
HFP-for-calls profile arrangement doesn't allow mic capture during speaker playback.

The approach we'd like to validate: force HFP-mono profile for the entire conversation session, keep
the mic open continuously, run software AEC (WebRTC AEC3) on the mic stream to cancel speaker leak,
and detect user speech / end-of-utterance on the cleaned mic signal. TTS audio quality drops from
A2DP-stereo to HFP-mono 16 kHz for the duration of the conversation, which we think is an acceptable
trade.

Four questions about Ray-Ban Meta (Gen 2 + Gen 3):

1. Can the host app force / hold the BT audio profile?
   Can we keep the connection in HFP for an extended session (60s+) rather than letting the stack switch
   to A2DP when TTS plays? Or is profile selection driven by activity heuristics and out of app
   control?

2. Is the mic stream accessible during HFP-mode TTS playback?
   While HFP is active and TTS audio is being routed to the speaker, can the app receive a continuous
   mic stream? Or is the mic capture reserved/disabled while speaker output is active (e.g., only
   available during active SCO calls)?

3. Is there hardware/firmware AEC, and is it exposed to apps?

• If yes: roughly what cancellation depth, and is there an API to enable / disable / tune it?
• If no: any guidance on the echo path characteristics (delay range, frequency response) so we can
  configure WebRTC AEC3 correctly?

4. Wake-word access — if and when?
   The glasses ship with a built-in always-on hot-word detector for "Hey Meta" — a low-power hardware
   path that activates the main mic on trigger without continuous app-side polling. Is there a plan to
   expose this capability to third-party SDKs? Two shapes that would be useful:

• Custom-phrase registration — the app declares a phrase (e.g. "Hey Coach"), the on-device detector
  wakes the app when it fires.
• "Hey Meta + handoff" — user says "Hey Meta, ask my coach about my bench press", the wake-word path
  activates and the post-"Hey Meta" payload is handed off to the appropriate registered app.

The hardware-level path matters here because polling the mic continuously from the SDK side is a
battery killer; an OS-level always-on detector is what makes truly always-on wake practical. Even
rough timing ("not on the roadmap" / "internal-only for now" / "Q3-ish 2026") helps us decide whether
to build product around it now or design a fallback (push-to-talk gesture, paired-phone wake) for
the near term.

---

Also welcome: any gotchas anyone has hit trying full-duplex on this hardware, or alternative paths we
haven't considered. Even a clean "this isn't possible on this firmware" answer on any of the four is
useful — it lets us scope our fallbacks confidently rather than chasing dead ends.

[michaelgaultjr]

I've used the glasses for my own AI companion app to do some of the things you're asking about, so I can answer some of those questions:

The SDK doesn't expose anything to do with audio, it's all handled over Bluetooth. So any TTS or STT you're responsible for. Anything you're able to do with a normal Bluetooth headset, you're able to do with the glasses. You can use HFP to have 2 way communication even while audio is playing, I've personally used this to do exactly what you're talking about in terms of "interrupting" TTS. You can use a VAD to detect speech, and then cancel the TTS.

Meta AI does not have any API's, so no custom phrases, actions, handoff etc... nothing. The closest thing you can do it setup a SMS, WhatsApp, or Facebook messenger bot, and then tell the AI to text that bot with whatever command, for example, you'd create a bot called "my coach" and then say "Hey Meta, text my coach about my bench press" and it it would text "my coach" the words "about my bench press"

As for custom wake words, no support for custom wake words on the hardware themselves. Though you could always have them running in HFP mode to text wake words. It's not optimal, but that's the only way it's possible.

As for hardware/firmware AEC, they use beam forming to isolate the users voice as mentioned in the docs. There is no access to raw audio streams from any of the mics even if you wanted to. Though from my personal testing a few months ago, you'll want to apply some additional filters and logic, as it does pick up others people's voices pretty clearly if they're within about 6ft or so.

Here is the documentation for using device microphones and speakers: https://wearables.developer.meta.com/docs/develop/dat/microphones-and-speakers

[Halsted312]

Done 🥸🥇