RoomComposite Egress only records the first room, subsequent rooms are not recorded

[quanhavn]

I'm using RoomComposite Egress from LiveKit to record room sessions. However, only the first room gets recorded successfully. All subsequent rooms do not trigger any recording, even though egress requests are made.

Details:

• Running in production mode using the start command
• I've tried launching multiple workers to handle concurrent recordings, but it doesn't help
• There are no clear errors in the logs — it simply seems like new egress sessions are ignored after the first one

Expected behavior:

All egress requests should be processed, allowing multiple rooms to be recorded either sequentially or concurrently (depending on worker availability).

Questions:

• Is there a known limitation with handling multiple RoomComposite egress sessions?
• Is there any additional configuration needed to support multiple room recordings?
• How can I debug or verify why only the first egress runs?

Possible bug:

This might be a bug related to how RoomComposite egress sessions are managed when multiple requests are made. Any insights or workarounds would be appreciated.

Source code

async def entrypoint(ctx: JobContext):
    logger.info(f"connecting to room {ctx.room.name}")
    
    prompt_service = PromptService(ctx.room.name)
    system_prompt = prompt_service.get_system_prompt(ctx.room.name)
    initial_ctx = llm.ChatContext().append(
        role="system",
        text=system_prompt,
    )
    await ctx.connect()

    participant = await ctx.wait_for_participant()
    logger.info(f"starting voice assistant for participant {participant.identity}")

    fnc_ctx = AssistantFnc(ctx)

    agent = VoicePipelineAgent(
        vad=ctx.proc.userdata["vad"],
        stt=stt_config,
        llm=llm_config,
        tts=tts_config,
        chat_ctx=initial_ctx,
        allow_interruptions=True, 
        interrupt_speech_duration=0.2,
        interrupt_min_words=0,
        min_endpointing_delay=0.2,
        fnc_ctx=fnc_ctx,
    )
    
    fnc_ctx.set_agent(agent)
    agent.start(ctx.room, participant)

    await agent.say(hello_message)

    req = api.RoomCompositeEgressRequest(
        room_name=ctx.room.name,
        layout="grid",
        preset=api.EncodingOptionsPreset.H264_720P_30,
        audio_only=False,
        segment_outputs=[
            api.SegmentedFileOutput(
                filename_prefix=path_filename_prefix,
                playlist_name=playlist_name,
                live_playlist_name=live_playlist_name,
                segment_duration=10,
                s3=api.S3Upload(
                    access_key=os.getenv("S3_ACCESS_KEY"),
                    secret=os.getenv("S3_SECRET"),
                    region=os.getenv("S3_REGION"),
                    bucket=os.getenv("S3_BUCKET"),
                    endpoint=os.getenv("S3_ENDPOINT"),
                    force_path_style=True,
                )
            )
        ]
    )
    res = await lkapi.egress.start_room_composite_egress(req)

Logs from agent.py

Don't have any logs on livekit server and livekit egress

2025-05-15 16:08:58,275 - ERROR - root - [PID:77194] [Worker:8252717760] - ('unavailable', 'twirp error unknown: no response from servers')
Traceback (most recent call last):
  File "/Users/back-end/agent.py", line 402, in entrypoint
    res = await lkapi.egress.start_room_composite_egress(req)
  File "/Users/back-end/venv310/lib/python3.10/site-packages/livekit/api/egress_service.py", line 44, in start_room_composite_egress
    return await self._client.request(
  File "/Users/back-end/venv310/lib/python3.10/site-packages/livekit/api/twirp_client.py", line 101, in request
    raise TwirpError(error_data["code"], error_data["msg"])
livekit.api.twirp_client.TwirpError: ('unavailable', 'twirp error unknown: no response from servers')
('unavailable', 'twirp error unknown: no response from servers')
2025-05-15 16:08:58,280 - ERROR - asyncio - [PID:77194] [Worker:8252717760] - Unclosed client session
client_session: <aiohttp.client.ClientSession object at 0x11e8b9d20>
2025-05-15 16:08:58,281 - ERROR - asyncio - [PID:77194] [Worker:8252717760] - Unclosed connector
connections: ['deque([(<aiohttp.client_proto.ResponseHandler object at 0x11e8037c0>, 28718.487585125)])']
connector: <aiohttp.connector.TCPConnector object at 0x11e8b9d80>
2025-05-15 16:08:58,275 - ERROR - root - [PID:77194] [Worker:8252717760] - ('unavailable', 'twirp error unknown: no response from servers')
Traceback (most recent call last):
  File "/Users/back-end/agent.py", line 402, in entrypoint
    res = await lkapi.egress.start_room_composite_egress(req)
  File "/Users/back-end/venv310/lib/python3.10/site-packages/livekit/api/egress_service.py", line 44, in start_room_composite_egress
    return await self._client.request(
  File "/Users/back-end/venv310/lib/python3.10/site-packages/livekit/api/twirp_client.py", line 101, in request
    raise TwirpError(error_data["code"], error_data["msg"])
livekit.api.twirp_client.TwirpError: ('unavailable', 'twirp error unknown: no response from servers')
2025-05-15 16:08:58,281 - DEBUG - livekit.agents - [PID:77194] [Worker:8252717760] - shutting down job task

[quanhavn]

Livekit Server resource

Redis data:

[superRookie007]

Having similar issue here

[davidzhao]

for self-hosted setup, please refer to Egress & livekit-server logs. this isn't an issue with agents.