Bug Description
I'm trying to migrate to LiveKit agents 1.6.1+ versions, and found some incompatibility between the newer versions and our previous evaluation/testing frameworks based on: https://docs.livekit.io/agents/start/testing/ . Specifically, the behavior of how to await nested AgentTask from a tool call within the parent Agent's on_enter() is changing and we would like your guidance.
I have a minimal reproducible script based on an earlier issue: #5371 (comment) (a related previous PR: #5377) See below for the script and error info.
The following error is obtained with livekit-agents v1.6.4, but the same error appeared since v1.6.1.
ERROR livekit.agents:log.py:22 Error in _llm_inference_task
Traceback (most recent call last):
File "/Users/cj/Library/Caches/pypoetry/virtualenvs/agent-eval-2eDZ1Xtk-py3.11/lib/python3.11/site-packages/livekit/agents/utils/log.py", line 17, in async_fn_logs
return await fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/cj/Library/Caches/pypoetry/virtualenvs/agent-eval-2eDZ1Xtk-py3.11/lib/python3.11/site-packages/opentelemetry/util/_decorator.py", line 71, in async_wrapper
return await func(*args, **kwargs) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/cj/Library/Caches/pypoetry/virtualenvs/agent-eval-2eDZ1Xtk-py3.11/lib/python3.11/site-packages/livekit/agents/voice/generation.py", line 145, in _llm_inference_task
async for chunk in llm_node:
File "/Users/cj/Library/Caches/pypoetry/virtualenvs/agent-eval-2eDZ1Xtk-py3.11/lib/python3.11/site-packages/livekit/agents/voice/agent.py", line 476, in llm_node
activity = agent._get_activity_or_raise()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/cj/Library/Caches/pypoetry/virtualenvs/agent-eval-2eDZ1Xtk-py3.11/lib/python3.11/site-packages/livekit/agents/voice/agent.py", line 415, in _get_activity_or_raise
raise RuntimeError("no activity context found, the agent is not running")
RuntimeError: no activity context found, the agent is not running
WARNING livekit.agents:agent_session.py:1463 session is closing, skipping start activity of agent_task1
WARNING livekit.agents:generation.py:729 ToolError while executing tool: activity doesn't start for agent_task1, likely due to session closing
WARNING livekit.agents:agent_activity.py:1474 attempting to schedule a new SpeechHandle, but the scheduling_task is not running, the speech will be cancelled
FAILED
Based on the pattern in livekit-agents’ own test for nested AgentTask on Github: https://github.com/livekit/agents/blob/main/tests/test_nested_agent_task.py , I modified the script accordingly to make it work with v1.6.1+. The change is to remove the on_enter() method from TestAgent.
Expected Behavior
The original script should continue to work, or if that's an intended behavioral change (breaking existing tests), a clarification paragraph could be added to the official documentation at https://docs.livekit.io/agents/start/testing/ and https://docs.livekit.io/agents/logic/tasks/
Reproduction Steps
The minimal reproducible script: #5371 (comment) (A related previous PR: #5377)
"""
LIVEKIT_EVALS_VERBOSE=1 pytest -s -o log_cli=true test_nested_task.py
Works for livekit-agents versions: 1.5.19rc1, 1.6.0, but not with 1.6.1
1.6.1 fails maybe due to this PR: https://github.com/livekit/agents/pull/6071
"""
import asyncio
# from typing import override
from livekit.agents import Agent, AgentSession, AgentTask, function_tool
from livekit.agents.llm.tool_context import NamedToolChoice
from livekit.plugins import openai
import pytest
class AgentTask1(AgentTask):
def __init__(self) -> None:
super().__init__(instructions="say hello")
print('after init AgentTask1')
# @override
async def on_enter(self) -> None:
print("AgentTask1 on_enter")
tool = NamedToolChoice(type="function", function={"name": "run_agent_task_2"})
await self.session.generate_reply(tool_choice=tool)
@function_tool
async def run_agent_task_2(self) -> None:
print("run_agent_task_2")
await AgentTask2()
self.complete(None)
class AgentTask2(AgentTask):
def __init__(self) -> None:
super().__init__(instructions="say hello")
# @override
async def on_enter(self) -> None:
print("AgentTask2 on_enter")
tool = NamedToolChoice(type="function", function={"name": "run_agent_task_3"})
await self.session.generate_reply(tool_choice=tool)
@function_tool
async def run_agent_task_3(self) -> None:
print("run_agent_task_3")
await AgentTask3()
self.complete(None)
class AgentTask3(AgentTask):
def __init__(self) -> None:
super().__init__(instructions="say hello")
# @override
async def on_enter(self) -> None:
print("AgentTask3 on_enter")
await self.session.generate_reply(instructions="say hello")
@function_tool
async def goodbye(self) -> None:
print("goodbye")
self.complete(None)
class TestAgent(Agent):
def __init__(self) -> None:
super().__init__(instructions="say hello")
# @override
async def on_enter(self) -> None:
print("\nTestAgent on_enter")
tool = NamedToolChoice(type="function", function={"name": "run_agent_task_1"})
await self.session.generate_reply(tool_choice=tool)
@function_tool
async def run_agent_task_1(self) -> None:
print("run_agent_task_1")
await AgentTask1()
@pytest.mark.asyncio
async def test_assistant_greeting() -> None:
llm = openai.LLM(model="gpt-5.4-mini")
async with AgentSession(llm=llm) as session:
await session.start(TestAgent())
print("user turn 1")
await session.run(user_input="hi")
print("user turn 2")
await session.run(user_input="hi there") # hangs here
print("user turn 3")
await session.run(user_input="sayonara")
Based on the pattern in livekit-agents’ own test for nested AgentTask on Github: https://github.com/livekit/agents/blob/main/tests/test_nested_agent_task.py , I modified the script accordingly to make it work with v1.6.1+. The change is to remove the on_enter() method from TestAgent.
Operating System
macOS Sequoia
Models Used
llm = openai.LLM(model="gpt-5.4-mini")
Package Versions
livekit-agents == v1.6.1 or newer
Session/Room/Call IDs
No response
Proposed Solution
Additional Context
No response
Screenshots and Recordings
No response
Bug Description
I'm trying to migrate to LiveKit agents 1.6.1+ versions, and found some incompatibility between the newer versions and our previous evaluation/testing frameworks based on: https://docs.livekit.io/agents/start/testing/ . Specifically, the behavior of how to await nested AgentTask from a tool call within the parent Agent's
on_enter()is changing and we would like your guidance.I have a minimal reproducible script based on an earlier issue: #5371 (comment) (a related previous PR: #5377) See below for the script and error info.
The following error is obtained with livekit-agents v1.6.4, but the same error appeared since v1.6.1.
Based on the pattern in livekit-agents’ own test for nested AgentTask on Github: https://github.com/livekit/agents/blob/main/tests/test_nested_agent_task.py , I modified the script accordingly to make it work with v1.6.1+. The change is to remove the
on_enter()method from TestAgent.Expected Behavior
The original script should continue to work, or if that's an intended behavioral change (breaking existing tests), a clarification paragraph could be added to the official documentation at https://docs.livekit.io/agents/start/testing/ and https://docs.livekit.io/agents/logic/tasks/
Reproduction Steps
The minimal reproducible script: #5371 (comment) (A related previous PR: #5377)
Based on the pattern in livekit-agents’ own test for nested AgentTask on Github: https://github.com/livekit/agents/blob/main/tests/test_nested_agent_task.py , I modified the script accordingly to make it work with v1.6.1+. The change is to remove the
on_enter()method from TestAgent.Operating System
macOS Sequoia
Models Used
llm = openai.LLM(model="gpt-5.4-mini")
Package Versions
Session/Room/Call IDs
No response
Proposed Solution
Additional Context
No response
Screenshots and Recordings
No response