Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/src/content/docs/guides/voice-agents/transport.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import historyUpdatedExample from '../../../../../../examples/docs/voice-agents/
import updateHistoryExample from '../../../../../../examples/docs/voice-agents/updateHistory.ts?raw';
import customWebRTCTransportExample from '../../../../../../examples/docs/voice-agents/customWebRTCTransport.ts?raw';
import websocketSessionExample from '../../../../../../examples/docs/voice-agents/websocketSession.ts?raw';
import sipTransportExample from '../../../../../../examples/docs/voice-agents/sipTransport.ts?raw';
import transportEventsExample from '../../../../../../examples/docs/voice-agents/transportEvents.ts?raw';
import thinClientExample from '../../../../../../examples/docs/voice-agents/thinClient.ts?raw';
import cloudflareTransportExample from '../../../../../../examples/docs/extensions/cloudflare-basic.ts?raw';
Expand All @@ -47,6 +48,16 @@ building a phone agent with Twilio.

Use any recording/playback library to handle the raw PCM16 audio bytes.

### Connecting over SIP

Bridge SIP calls from providers such as Twilio by using the `OpenAIRealtimeSIP` transport. The transport keeps the Realtime session synchronized with SIP events emitted by your telephony provider.

1. Accept the incoming call by generating an initial session configuration with `OpenAIRealtimeSIP.buildInitialConfig()`. This ensures the SIP invitation and Realtime session share identical defaults.
2. Attach a `RealtimeSession` that uses the `OpenAIRealtimeSIP` transport and connect with the `callId` issued by the provider webhook.
3. Listen for session events to drive call analytics, transcripts, or escalation logic.

<Code lang="typescript" code={sipTransportExample} />

#### Cloudflare Workers (workerd) note

Cloudflare Workers and other workerd runtimes cannot open outbound WebSockets using the global `WebSocket` constructor. Use the Cloudflare transport from the extensions package, which performs the `fetch()`-based upgrade internally.
Expand Down
57 changes: 57 additions & 0 deletions examples/docs/voice-agents/sipTransport.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
import OpenAI from 'openai';
import {
OpenAIRealtimeSIP,
RealtimeAgent,
RealtimeSession,
type RealtimeSessionOptions,
} from '@openai/agents/realtime';

const openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY!,
webhookSecret: process.env.OPENAI_WEBHOOK_SECRET!,
});

const agent = new RealtimeAgent({
name: 'Receptionist',
instructions:
'Welcome the caller, answer scheduling questions, and hand off if the caller requests a human.',
});

const sessionOptions: Partial<RealtimeSessionOptions> = {
model: 'gpt-realtime',
config: {
audio: {
input: {
turnDetection: { type: 'semantic_vad', interruptResponse: true },
},
},
},
};

export async function acceptIncomingCall(callId: string): Promise<void> {
const initialConfig = await OpenAIRealtimeSIP.buildInitialConfig(
agent,
sessionOptions,
);
await openai.realtime.calls.accept(callId, initialConfig);
}

export async function attachRealtimeSession(
callId: string,
): Promise<RealtimeSession> {
const session = new RealtimeSession(agent, {
transport: new OpenAIRealtimeSIP(),
...sessionOptions,
});

session.on('history_added', (item) => {
console.log('Realtime update:', item.type);
});

await session.connect({
apiKey: process.env.OPENAI_API_KEY!,
callId,
});

return session;
}