AGT-2182: Add adaptive interruption handling and dynamic endpointing#4771
AGT-2182: Add adaptive interruption handling and dynamic endpointing#4771chenghao-mou merged 52 commits intomainfrom
Conversation
Co-authored-by: Chenghao Mou <chenghao.mou@livekit.io> Co-authored-by: Théo Monnom <theo.monnom@outlook.com>
Co-authored-by: Théo Monnom <theo.monnom@outlook.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Théo Monnom <theo.8bits@gmail.com> Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
| _logger_class = logging.getLoggerClass() | ||
| logging.setLoggerClass(Logger) | ||
| logger: Logger = logging.getLogger("livekit.agents") # type: ignore[assignment] | ||
| logging.setLoggerClass(_logger_class) |
There was a problem hiding this comment.
🟡 logging.setLoggerClass globally mutates the logging module, affecting all loggers created concurrently
In log.py, the code temporarily sets the global logger class to Logger via logging.setLoggerClass(Logger), creates the livekit.agents logger, then restores the original class. This is not thread-safe: any logger created by another thread between lines 21 and 23 will inadvertently be a Logger instance. Additionally, any child loggers of livekit.agents created later (e.g., logging.getLogger("livekit.agents.voice")) will use the standard logging.Logger class and won't have the trace() or dev() methods, leading to AttributeError if those methods are ever called on a child logger.
Was this helpful? React with 👍 or 👎 to provide feedback.
| def serialize_model(self, handler: SerializerFunctionWrapHandler) -> Any: | ||
| # remove numpy arrays from the model dump | ||
| copy = self.model_copy(deep=True) | ||
| data = copy.speech_input, copy.probabilities | ||
| copy.speech_input, copy.probabilities = None, None | ||
| try: | ||
| serialized = handler(copy) | ||
| finally: | ||
| copy.speech_input, copy.probabilities = data | ||
| return serialized |
There was a problem hiding this comment.
🟡 OverlappingSpeechEvent.serialize_model corrupts original object's speech_input and probabilities on exception
In interruption.py:166-175, the serialize_model method makes a deep copy but then destructures it incorrectly. It saves data = copy.speech_input, copy.probabilities, sets the copy's fields to None, serializes, then restores from data. However, if handler(copy) raises an exception, the finally block restores the copy's fields — not the original's — so there's no actual corruption of self. But more importantly, model_copy(deep=True) on a Pydantic model containing numpy arrays may fail or produce unexpected results since numpy arrays aren't standard Pydantic-serializable types. The real issue is that data captures (copy.speech_input, copy.probabilities) as a tuple but the restore line copy.speech_input, copy.probabilities = data unpacks it back — this is fine syntactically but the entire deep copy + restore pattern is unnecessary since the copy is never used after the function returns.
Was this helpful? React with 👍 or 👎 to provide feedback.
| input_tokens=self._input_tokens, | ||
| output_tokens=self._output_tokens, |
There was a problem hiding this comment.
🔴 TTS streaming metrics report cumulative _input_tokens/_output_tokens for every segment instead of per-segment
In tts.py, SynthesizeStream._set_token_usage sets instance-level self._input_tokens and self._output_tokens (lines 447-449). These values are then emitted in the TTSMetrics for every segment produced by _emit_metrics (line 558-559). If a TTS provider calls _set_token_usage once with cumulative totals, every subsequent segment's metrics will duplicate those same token counts, inflating the totals in ModelUsageCollector.collect() (usage.py:213-214) which sums them. Unlike characters_count which is per-segment (derived from len(text)), the token counts are not reset between segments.
Was this helpful? React with 👍 or 👎 to provide feedback.
… metrics_collected (#5124) Co-authored-by: Chenghao Mou <chenghao.mou@livekit.io>
Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com> Co-authored-by: Théo Monnom <theo.monnom@outlook.com> Co-authored-by: David Zhao <dz@livekit.io>
No description provided.