-
Notifications
You must be signed in to change notification settings - Fork 294
Description
Issue Description
I'm encountering a RangeError: byte length of Int16Array should be a multiple of 2
error when processing responses from this package. This code was working correctly until yesterday without any changes on my side, suggesting a possible change in the API response format. My other project using vanilla WebSockets with the same API still works properly.
Environment Details
Node.js version: v22.16.0 as well as v22.12.0
Package versions:
@openai/realtime-api-beta: from github:openai/openai-realtime-api-beta
Minimal Reproduction Example
import { RealtimeClient } from "@openai/realtime-api-beta";
import { Buffer } from "node:buffer";
import http from "node:http";
import { WebSocketServer } from "ws";
import dotenv from "dotenv";
// Load environment variables
dotenv.config();
function convertInt16ArrayToBase64(int16Array) {
const buffer = Buffer.from(int16Array.buffer);
return buffer.toString("base64");
}
function convertBase64ToInt16Array(base64String) {
const buffer = Buffer.from(base64String, "base64");
// Error occurs here when processing API response
return new Int16Array(buffer.buffer);
}
// Setup simple server
const server = http.createServer((req, res) => {
res.writeHead(501);
res.end("Not Implemented");
});
const wss = new WebSocketServer({ server });
const openAiSocket = new RealtimeClient({
apiKey: process.env.OPENAI_API_KEY,
url: "wss://api.openai.com/v1/realtime",
});
openAiSocket.updateSession({
input_audio_format: "g711_ulaw",
output_audio_format: "g711_ulaw",
voice: "ash",
modalities: ["text", "audio"]
});
// Test connection to reproduce the error
wss.on('connection', (socket) => {
openAiSocket.connect().then(() => {
console.log("Connected to RealtimeClient");
openAiSocket.sendUserMessageContent([{ type: "input_text", text: "Hello" }]);
});
// This is where the error occurs
openAiSocket.on("conversation.item.completed", ({ item }) => {
if (item.role === "assistant") {
try {
const base64Audio = convertInt16ArrayToBase64(item.formatted.audio);
console.log("Conversion successful");
} catch (error) {
console.error("Error converting audio:", error);
}
}
});
});
server.listen(3000, () => console.log("Server running on port 3000"));
Error Details
Additional Information
No code changes were made prior to encountering this error
A separate implementation using vanilla WebSockets still works properly
This suggests a potential change in how the Realtime API beta is formatting audio data in responses
The code is triggered by an incoming voice call as described here
Expected Behavior
The API should return audio data that can be properly converted to Int16Array without buffer length errors.