-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Description
Summary
The OpenAI Node SDK provides stable audio APIs — audio.transcriptions.create() (Whisper speech-to-text) and audio.speech.create() (TTS text-to-speech) — but neither the wrapper (wrapOpenAI) nor the auto-instrumentation plugin instruments them. Calls to these methods produce no Braintrust spans. These are AI model inference calls (Whisper, TTS-1) that users of voice-enabled AI applications would expect to see traced.
What is missing
- Wrapper (
js/src/wrappers/oai.ts): No proxy foraudioresource. Onlychat.completions,embeddings,moderations, andresponsesare wrapped. - Auto-instrumentation config (
js/src/auto-instrumentations/configs/openai.ts): No config entry for anyaudio.*methods. - Channels (
js/src/instrumentation/plugins/openai-channels.ts): No channel definitions for audio APIs. - Plugin (
js/src/instrumentation/plugins/openai-plugin.ts): No handler for audio calls.
Upstream reference
- OpenAI Audio API: https://platform.openai.com/docs/api-reference/audio
- OpenAI Node SDK audio resources: https://github.com/openai/openai-node/blob/master/src/resources/audio/index.ts
- Key methods:
audio.transcriptions.create()— Whisper model, converts audio to text. Returns transcription text, segments with timestamps, language, duration.audio.speech.create()— TTS-1/TTS-1-HD models, converts text to audio. Returns audio stream (mp3, opus, aac, flac, wav, pcm).audio.translations.create()— Whisper model, translates audio to English text.
- All are stable, production APIs invoking AI models.
Braintrust docs status
The Braintrust wrapper docs page broadly mentions tracing "GPT models, embeddings, and other OpenAI APIs" but does not specifically enumerate audio support (unclear).
Precedent in this repo
OpenAI embeddings and moderations — similarly "secondary" model APIs — are instrumented in all three layers:
- Wrapper:
js/src/wrappers/oai.ts - Auto-instrumentation config:
js/src/auto-instrumentations/configs/openai.ts - Plugin:
js/src/instrumentation/plugins/openai-plugin.ts
Local files inspected
js/src/wrappers/oai.tsjs/src/auto-instrumentations/configs/openai.tsjs/src/instrumentation/plugins/openai-channels.tsjs/src/instrumentation/plugins/openai-plugin.tsjs/src/vendor-sdk-types/openai.tsjs/src/vendor-sdk-types/openai-common.ts
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels