Voice lagging with OpenAi Realtime

[3Omarito]

Hello Dear,
I've an issue while integrate Asterisk with OpenAi using Realtime API. Shortly, I use Audiosocke to connect with Asterisk and then once the connection has begun, I start the OpenAi connection. When I use res.play function out of OpenAi connection function it works fine, but when I use it inside OpenAi connection the played file is lagging and you cannot recognize the voice.

`
audioSocket.onConnection(async (req, res) => {

openAiWs.on( 'message', async (data) => {
await res.play("/root/moh.wav");

        try {
            const response = JSON.parse(data);


            if (response.type === 'response.audio.delta' && response.delta) {
                /*const audioDelta = {
                    event: 'media',
                    streamSid: streamSid,
                    media: { payload: response.delta }
                };*/

                //console.log(typeof data); // Should output 'object' for Buffer
                //console.log(Buffer.isBuffer(data)); // Should output true if data is a Buffer

                //console.log('Firstone:', response);

                const finaldata = Buffer.from(response.delta, "base64");
                //console.log('Final Data is:', finaldata);

                //await res.write(finaldata);
                await res.play("/root/moh.wav");
            }
            } catch (error) {
            console.error('Error processing OpenAI message:', error, 'Raw message:', data);
                            }

});
});
`

[Ulexus]

Sorry, I don't understand what you are saying.

Usually, though, when I see people complaining about corrupt audio, it is because Asterisk is not using slin codec. Unfortunately, at some point after audiosocket was merged, they modified the protocol such that Asterisk reports SLIN codec even though it just uses whatever it uses. Therefore, in modern installations, we have to explicitly declare the slin encoding for AudioSocket channels.

[disolaterX]

Sorry, I don't understand what you are saying.

Usually, though, when I see people complaining about corrupt audio, it is because Asterisk is not using slin codec. Unfortunately, at some point after audiosocket was merged, they modified the protocol such that Asterisk reports SLIN codec even though it just uses whatever it uses. Therefore, in modern installations, we have to explicitly declare the slin encoding for AudioSocket channels.

I am running Asterisk 20 i also get almost no sound complete corrupt audio using golang ai to create external media

uuidStr := uuid.New().String()
channel, err := client.Channel().StageExternalMedia(nil, ari.ExternalMediaOptions{
	App:           ARI_APP_NAME,
	Encapsulation: "audiosocket",
	Transport:     "tcp",
	ExternalHost:  "0.0.0.0:9090",
	Format:        "ulaw",
	Direction:     "both",
	Data:          uuidStr,
	Variables:     variables,
})

this send data but when i try to send data back to asterisk from audiosocket server fully distorted cracked sound