Skip to content

Voice configuration not taking effect on RealtimeSession #675

@viniciuspalma

Description

@viniciuspalma

Please read this first

  • Have you read the docs? Agents SDK docs
  • Have you searched for related issues? Others may have faced similar issues.

Describe the bug

Voice on realtime Agent not changing doesn't matter which configs I pass, by looking at the typescript types I see that there's a deprecated config and a newly created type definition. Which one should I use, the new one still not well supported?

Debug information

  • Agents SDK version: (e.g. v0.3.0)
  • Runtime environment (e.g. Node.js 20.9.0)

Repro steps

      this.agent = new RealtimeAgent<RealtimeContextData<VoiceAgentContext>>({
        name: `triage_agent`,
        instructions: agentInstructions,
        tools,
      });

      const session = new RealtimeSession<
        RealtimeContextData<VoiceAgentContext>
      >(this.agent, {
        model: 'gpt-realtime',

        config: {
          // Tried here 
          voice: 'marin',

          // Turn detection must also be top level
          turnDetection: {
            type: 'server_vad',
            threshold: 0.5,
            prefix_padding_ms: 300,
            silence_duration_ms: 500,
          },

          // Tried to change voice on audio as well
          //
          audio: {
            input: {
              transcription: {
                model: 'whisper-1',
                language: 'en',
              },
            },
            output: {
              format: 'pcm16',
            },
          },
        },

        context: {
          ...agentContext,
          history: [],
        },
      });

Expected behavior

I would expect to the Agent to take the voice that I'm passing to.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions