Skip to content

Shubhra/clean up real time model logs #729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Shubhrakanti merged 2 commits into from
Sep 25, 2025
Merged

Shubhra/clean up real time model logs #729

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5ddfacc
init
Shubhrakanti Sep 25, 2025
1d8e5af
changeset
Shubhrakanti Sep 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/every-moons-search.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@livekit/agents-plugin-openai': patch
---

update debug logs to be enabled by env variable
32 changes: 17 additions & 15 deletions plugins/openai/src/realtime/realtime_model.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,9 @@ import { AudioFrame, combineAudioFrames } from '@livekit/rtc-node';
import { type MessageEvent, WebSocket } from 'ws';
import * as api_proto from './api_proto.js';

// if LK_OPENAI_DEBUG convert it to a number, otherwise set it to 0
const lkOaiDebug = process.env.LK_OPENAI_DEBUG ? Number(process.env.LK_OPENAI_DEBUG) : 0;

const SAMPLE_RATE = 24000;
const NUM_CHANNELS = 1;
const BASE_URL = 'https://api.openai.com/v1';
Expand Down Expand Up @@ -640,11 +643,8 @@ export class RealtimeSession extends llm.RealtimeSession {
} as api_proto.ConversationItemTruncateEvent);
}

/// Truncates the data field of the event to the specified maxLength to avoid overwhelming logs
Copy link
Copy Markdown
Contributor Author

@Shubhrakanti Shubhrakanti Sep 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

raw audio data is useless to look at

/// with large amounts of base64 audio data.
#loggableEvent(
private loggableEvent(
event: api_proto.ClientEvent | api_proto.ServerEvent,
maxLength: number = 30,
): Record<string, unknown> {
const untypedEvent: Record<string, unknown> = {};
for (const [key, value] of Object.entries(event)) {
Expand All @@ -654,18 +654,14 @@ export class RealtimeSession extends llm.RealtimeSession {
}

if (untypedEvent.audio && typeof untypedEvent.audio === 'string') {
const truncatedData =
untypedEvent.audio.slice(0, maxLength) + (untypedEvent.audio.length > maxLength ? '…' : '');
return { ...untypedEvent, audio: truncatedData };
return { ...untypedEvent, audio: '...' };
}
if (
untypedEvent.delta &&
typeof untypedEvent.delta === 'string' &&
event.type === 'response.audio.delta'
) {
const truncatedDelta =
untypedEvent.delta.slice(0, maxLength) + (untypedEvent.delta.length > maxLength ? '…' : '');
return { ...untypedEvent, delta: truncatedDelta };
return { ...untypedEvent, delta: '...' };
}
return untypedEvent;
}
Expand Down Expand Up @@ -699,7 +695,9 @@ export class RealtimeSession extends llm.RealtimeSession {
azureDeployment: this.oaiRealtimeModel._options.azureDeployment,
});

this.#logger.debug(`Connecting to OpenAI Realtime API at ${url}`);
if (lkOaiDebug) {
this.#logger.debug(`Connecting to OpenAI Realtime API at ${url}`);
}

return new Promise((resolve, reject) => {
const ws = new WebSocket(url, { headers });
Expand Down Expand Up @@ -849,8 +847,8 @@ export class RealtimeSession extends llm.RealtimeSession {
break;
}

if (event.type !== 'input_audio_buffer.append') {
this.#logger.debug(`(client) -> ${JSON.stringify(this.#loggableEvent(event))}`);
if (lkOaiDebug) {
this.#logger.debug(this.loggableEvent(event), `(client) -> ${event.type}`);
}

this.emit('openai_client_event_queued', event);
Expand All @@ -876,7 +874,9 @@ export class RealtimeSession extends llm.RealtimeSession {
const event: api_proto.ServerEvent = JSON.parse(message.data as string);

this.emit('openai_server_event_received', event);
this.#logger.debug(`(server) <- ${JSON.stringify(this.#loggableEvent(event))}`);
if (lkOaiDebug) {
this.#logger.debug(this.loggableEvent(event), `(server) <- ${event.type}`);
}

switch (event.type) {
case 'input_audio_buffer.speech_started':
Expand Down Expand Up @@ -931,7 +931,9 @@ export class RealtimeSession extends llm.RealtimeSession {
this.handleError(event);
break;
default:
this.#logger.debug(`unhandled event: ${event.type}`);
if (lkOaiDebug) {
this.#logger.debug(`unhandled event: ${event.type}`);
}
break;
}
};
Expand Down
3 changes: 2 additions & 1 deletion turbo.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,8 @@
"GOOGLE_GENAI_API_KEY",
"GOOGLE_GENAI_USE_VERTEXAI",
"GOOGLE_CLOUD_PROJECT",
"GOOGLE_CLOUD_LOCATION"
"GOOGLE_CLOUD_LOCATION",
"LK_OPENAI_DEBUG"
],
"pipeline": {
"build": {
Expand Down