Skip to content

Buffer Handling Issue #113

@IObert

Description

@IObert

Issue Description

I'm encountering a RangeError: byte length of Int16Array should be a multiple of 2 error when processing responses from this package. This code was working correctly until yesterday without any changes on my side, suggesting a possible change in the API response format. My other project using vanilla WebSockets with the same API still works properly.

Environment Details

Node.js version: v22.16.0 as well as v22.12.0
Package versions:
@openai/realtime-api-beta: from github:openai/openai-realtime-api-beta

Minimal Reproduction Example

import { RealtimeClient } from "@openai/realtime-api-beta";
import { Buffer } from "node:buffer";
import http from "node:http";
import { WebSocketServer } from "ws";
import dotenv from "dotenv";

// Load environment variables
dotenv.config();

function convertInt16ArrayToBase64(int16Array) {
  const buffer = Buffer.from(int16Array.buffer);
  return buffer.toString("base64");
}

function convertBase64ToInt16Array(base64String) {
  const buffer = Buffer.from(base64String, "base64");
  // Error occurs here when processing API response
  return new Int16Array(buffer.buffer);
}

// Setup simple server
const server = http.createServer((req, res) => {
  res.writeHead(501);
  res.end("Not Implemented");
});

const wss = new WebSocketServer({ server });
const openAiSocket = new RealtimeClient({
  apiKey: process.env.OPENAI_API_KEY,
  url: "wss://api.openai.com/v1/realtime",
});

openAiSocket.updateSession({
  input_audio_format: "g711_ulaw",
  output_audio_format: "g711_ulaw",
  voice: "ash",
  modalities: ["text", "audio"]
});

// Test connection to reproduce the error
wss.on('connection', (socket) => {
  openAiSocket.connect().then(() => {
    console.log("Connected to RealtimeClient");
    openAiSocket.sendUserMessageContent([{ type: "input_text", text: "Hello" }]);
  });

  // This is where the error occurs
  openAiSocket.on("conversation.item.completed", ({ item }) => {
    if (item.role === "assistant") {
      try {
        const base64Audio = convertInt16ArrayToBase64(item.formatted.audio);
        console.log("Conversion successful");
      } catch (error) {
        console.error("Error converting audio:", error);
      }
    }
  });
});

server.listen(3000, () => console.log("Server running on port 3000"));

Error Details

Additional Information

No code changes were made prior to encountering this error
A separate implementation using vanilla WebSockets still works properly
This suggests a potential change in how the Realtime API beta is formatting audio data in responses

The code is triggered by an incoming voice call as described here

Expected Behavior

The API should return audio data that can be properly converted to Int16Array without buffer length errors.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions