build(deps): bump pipecat-ai from 1.0.0 to 1.4.0 in /perf/framework_environments/pipecat

[dependabot]

Bumps pipecat-ai from 1.0.0 to 1.4.0.

Release notes

Sourced from pipecat-ai's releases.

v1.4.0

Added

• Added on_user_turn_message_added event handler on LLMUserAggregator, with a new UserTurnMessageAddedMessage arg type. It fires when the user aggregator writes a message to the LLM context, carrying the finalized turn text. In cascade mode it coincides with on_user_turn_stopped; in realtime mode (when realtime_service_mode=True on the aggregator pair) it's the canonical way to subscribe to "context just updated, here's the user text" (since the on_user_turn_stopped event fires before the message is finalized, with UserTurnStoppedMessage.content=None). Note that there's been no change to on_assistant_turn_stopped. (PR #4533)

• Added RealtimeServiceMetadataFrame, broadcast at pipeline start by realtime LLM services (OpenAI Realtime, Azure Realtime, Inworld, Grok/xAI Realtime, Gemini Live, AWS Nova Sonic, Ultravox). This frame can be used by other processors in the pipeline to configure themselves accordingly. Today, it only advertises two things: that a realtime service is present in the pipeline (indicated by the fact that the frame is sent at all), and emits_user_turn_frames, which says whether the realtime service can emit its own UserStartedSpeakingFrame and UserStoppedSpeakingFrames (suggesting local VAD/turn detection may not be needed in the pipeline). (PR #4533)

• Added to our examples "locally-driven-turns" variants for:

  • OpenAI Realtime (realtime-openai-locally-driven-turns.py)
  • Grok Realtime (realtime-grok-locally-driven-turns.py)
  • Inworld Realtime (realtime-inworld-locally-driven-turns.py)

  These join realtime-gemini-live-locally-driven-turns.py in showing how to configure each realtime service so that its turn-taking is dictated by local turn detection (e.g. VAD + smart turn analyzer). (PR #4533)

• Added a startup WARNING log on realtime LLM services that don't emit UserStartedSpeakingFrame/UserStoppedSpeakingFrame (Gemini Live, AWS Nova Sonic, Ultravox). The log is meant to draw attention to a couple of things:

  • That other processors in the pipeline (e.g. RTVI) may expect turn frames, and that the developer can enable local VAD/turn detection to supply them, and, relatedly
  • That when using local turn detection, local turns may NOT perfectly align with the "ground truth" of server-decided turns, so they should be thought of as APPROXIMATE (unless local turn detection is driving the realtime service's turns, in which case there's no separate server-decided ground truth)

  (The warning also serves as a little nudge to the realtime service providers: providing a "ground truth" signal of when the provider thinks the user has started or stopped speaking is very helpful to app developers!) (PR #4533)

• Added a realtime_service_mode: bool kwarg on LLMContextAggregatorPair, for opting into a set of behaviors tailored for use with realtime (speech-to-speech) services. Setting realtime_service_mode=True does three things: 1. Decouples context writes from the UserStoppedSpeakingFrame signal. Instead, the assistant response start triggers the user message writes. This ensures that context is written properly even when the realtime service provides no turn frames and local turn detection (i.e. local VAD) is disabled. This mechanism also enables the next point. 2. Lets UserStoppedSpeakingFrame fire without waiting for transcripts. When local turn detection is configured to drive realtime service conversations, UserStoppedSpeakingFrame is the signal that triggers assistant responses. By letting this frame fire earlier, we reduce latency. 3. Replaces the default turn strategies with ExternalUserTurnStartStrategy and ExternalUserTurnStopStrategy when the realtime service advertises that it emits its own turn frames. Various realtime services (OpenAI Realtime, Azure, Grok, Inworld) emit their own turn frames; in that case the External strategies fire on_user_turn_started / on_user_turn_stopped from the server-emitted UserStartedSpeakingFrame / UserStoppedSpeakingFrame. For realtime services that don't emit those frames — either because they never do (Gemini Live, Nova Sonic, Ultravox) or because server-side turn detection has been disabled at runtime (e.g. OpenAI Realtime with turn_detection=False, in locally-driven-turns setups) — the defaults stay in place so locally-driven turn detection (e.g. local VAD) can fire the events. Passing custom user_turn_strategies opts out of the swap.

  Note that when realtime_service_mode=True, you should listen for the new on_user_turn_message_added event to get the newly-added user message rather than on_user_turn_stopped, which no longer carries it. (PR #4533)

• Added private_endpoint parameter to AzureTTSService and AzureHttpTTSService for connecting via Private Link or custom domain endpoints, matching existing AzureSTTService support. (PR #4549)

• Added will_be_spoken field to AggregatedTextFrame. Set to True by the TTS service just before synthesis, allowing downstream processors and observers to know whether TTS will speak a given text segment before audio begins. (PR #4559)

• Added AggregatedTextProgressFrame — a new frame emitted alongside each TTSTextFrame during word-timestamp playback. It carries accumulated_text (text already spoken) and remaining_text (text not yet spoken) for the active segment, enabling downstream consumers such as the RTVI observer to do word-level highlighting without coupling to internal sequencer state. (PR #4559)

• Added AICQuailVADAnalyzer (pipecat.audio.vad.aic_quail_vad), a noise-robustVoice Activity Detection analyzer powered by the standalone Quail VAD 2.0 model from the ai-coustics SDK (aic-sdk~=2.3.0). It owns its own Processor and works independently of AICFilter, so it can sit before or after enhancement in the pipeline. Defaults to the published quail-vad-2.0-xxs-16khz model; supply model_id/model_path to override. (PR #4588)

• Added continuous_partials and interruption_delay connection parameters to the AssemblyAI streaming STT service (u3-rt-pro only). continuous_partials defaults to True so voice agents receive interim transcripts at a steady cadence during long turns; interruption_delay (0–1000 ms) overrides how soon the first partial is emitted. Both are exposed via AssemblyAISTTService.Settings and are omitted for non-u3-rt-pro models. (PR #4593)

• Added a user_audio_preroll_secs parameter to GeminiLiveLLMService controlling how much "pre-roll" audio is replayed (sent to Gemini Live) when the user turn start is confirmed, in locally-driven-turns mode (server-side VAD disabled). Defaults to None, auto-sizing the pre-roll duration from the upstream VAD's start_secs (which assumes VAD drives turn starts); set it explicitly when using a non-VAD turn-start strategy. (PR #4597)

• Added a user_audio_preroll_secs parameter to OpenAIRealtimeLLMService controlling how much "pre-roll" audio is replayed (re-appended to the input audio buffer) when the user turn start is confirmed, in locally-driven-turns mode (server-side turn detection disabled). Defaults to None, auto-sizing the pre-roll duration from the upstream VAD's start_secs (which assumes VAD drives turn starts); set it explicitly when using a non-VAD turn-start strategy. (PR #4599)

... (truncated)

Changelog

Sourced from pipecat-ai's changelog.

[1.4.0] - 2026-06-16

Added

• Added on_user_turn_message_added event handler on LLMUserAggregator, with a new UserTurnMessageAddedMessage arg type. It fires when the user aggregator writes a message to the LLM context, carrying the finalized turn text. In cascade mode it coincides with on_user_turn_stopped; in realtime mode (when realtime_service_mode=True on the aggregator pair) it's the canonical way to subscribe to "context just updated, here's the user text" (since the on_user_turn_stopped event fires before the message is finalized, with UserTurnStoppedMessage.content=None). Note that there's been no change to on_assistant_turn_stopped. (PR #4533)

• Added RealtimeServiceMetadataFrame, broadcast at pipeline start by realtime LLM services (OpenAI Realtime, Azure Realtime, Inworld, Grok/xAI Realtime, Gemini Live, AWS Nova Sonic, Ultravox). This frame can be used by other processors in the pipeline to configure themselves accordingly. Today, it only advertises two things: that a realtime service is present in the pipeline (indicated by the fact that the frame is sent at all), and emits_user_turn_frames, which says whether the realtime service can emit its own UserStartedSpeakingFrame and UserStoppedSpeakingFrames (suggesting local VAD/turn detection may not be needed in the pipeline). (PR #4533)

• Added to our examples "locally-driven-turns" variants for:

  • OpenAI Realtime (realtime-openai-locally-driven-turns.py)
  • Grok Realtime (realtime-grok-locally-driven-turns.py)
  • Inworld Realtime (realtime-inworld-locally-driven-turns.py)

  These join realtime-gemini-live-locally-driven-turns.py in showing how to configure each realtime service so that its turn-taking is dictated by local turn detection (e.g. VAD + smart turn analyzer). (PR #4533)

• Added a startup WARNING log on realtime LLM services that don't emit UserStartedSpeakingFrame/UserStoppedSpeakingFrame (Gemini Live, AWS Nova Sonic, Ultravox). The log is meant to draw attention to a couple of things:

  • That other processors in the pipeline (e.g. RTVI) may expect turn frames, and that the developer can enable local VAD/turn detection to supply them, and, relatedly
  • That when using local turn detection, local turns may NOT perfectly align with the "ground truth" of server-decided turns, so they should be thought of as APPROXIMATE (unless local turn detection is driving the realtime service's turns, in which case there's no separate server-decided ground truth)

  (The warning also serves as a little nudge to the realtime service providers: providing a "ground truth" signal of when the provider thinks the

... (truncated)

Commits

• 3a69d3f Merge pull request #4789 from pipecat-ai/changelog-1.4.0
• 416798c Update changelog for version 1.4.0
• 8c39194 Merge pull request #4786 from pipecat-ai/fix/deepgram-sagemaker-vad-events
• 20daf45 Add changelog for DeepgramSageMakerSTTService construction fix
• 2fbdb77 Fix DeepgramSageMakerSTTService crash on construction
• 338177d Merge pull request #4785 from pipecat-ai/aleix/rtvi-vad-speaking-events
• 64aaba9 Add changelog for RTVIObserver VAD speaking events
• eca552f Add filter_incomplete_turns_function_calling release-eval scenario
• aafc196 Don't log client disconnect as an error in the websocket server transport
• 4243670 Add filter_incomplete_turns release-eval scenario
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
  You can disable automated security fix PRs for this repo from the Security Alerts page.