Client-transcribe-streaming - Websocket is closing.

[saravanakumargs]

Describe the bug

While calling the StartMedicalStreamTranscriptionCommand, getting the error as websocket-handler.ts:101 WebSocket is already in CLOSING or CLOSED state

Your environment

Local system Testing.

SDK version number

@aws-sdk/client-transcribe-streaming@3.16.0

Is the issue in the browser/Node.js/ReactNative?

Browser/Node.js/ReactNative
Browser - Node Js React App.

Details of the browser/Node.js/ReactNative version

Paste output of npx envinfo --browsers or node -v or react-native -v

Steps to reproduce

Please share code or minimal repo, and steps to reproduce the behavior.

var params = {
  LanguageCode: "en-US",
  MediaSampleRateHertz: 16000,
  MediaEncoding: "pcm",
  AudioStream: null,
  Specialty: Specialty.CARDIOLOGY,
  Type: Type.CONVERSATION,
};

const client = new TranscribeStreamingClient({
  region: "us-east-2",
  credentials: {
    accessKeyId: REACT_APP_AWS_ACCESS_KEY,
    secretAccessKey: REACT_APP_AWS_SECRET_KEY,
  },
  maxAttempts: 5,
});

async function* audioStream() {
  console.log("inside audiostream");
  for await (const chunk of microphoneStream) {
    yield {
      AudioEvent: {
        AudioChunk: chunk,
      },
    };
  }
}

async function handleClick() {
  console.log("transcrit");
  console.log(currentStream);
  console.log(process.env);
  console.log(REACT_APP_AWS_ACCESS_KEY);
  console.log(REACT_APP_AWS_SECRET_KEY);
  console.log(REACT_APP_MY_ENV);
  if (microphoneStream === null) {
    microphoneStream = new MicrophoneStream();
  }
  microphoneStream.setStream(currentStream);
  console.log(microphoneStream);

  var transcribeparams = params;
  transcribeparams.AudioStream = audioStream();

  console.log(transcribeparams);
  command = new StartMedicalStreamTranscriptionCommand(params);
  console.log(command);
  try {
    const data = await client.send(command);
    console.log(data);
  } catch (error) {
    console.log("Inside error");
    console.log(error);
  } finally {
    console.log("finally");
  }
}    

Observed behavior

Websocket is in closed state. Even though I get 200 status code from the send command.

{$metadata: {…}, ContentIdentificationType: undefined, EnableChannelIdentification: undefined, LanguageCode: "en-US", MediaEncoding: "pcm", …}
$metadata:
attempts: 1
cfId: undefined
extendedRequestId: undefined
httpStatusCode: 200
requestId: undefined
totalRetryDelay: 0
__proto__: Object
ContentIdentificationType: undefined
EnableChannelIdentification: undefined
LanguageCode: "en-US"
MediaEncoding: "pcm"
MediaSampleRateHertz: 16000
NumberOfChannels: undefined
RequestId: undefined
SessionId: "3ffd0fcc-e99d-4f80-a8c9-30ada953d725"
ShowSpeakerLabel: undefined
Specialty: "CARDIOLOGY"
TranscriptResultStream:
Symbol(Symbol.asyncIterator): ƒ ()
__proto__: Object
Type: "CONVERSATION"
VocabularyName: undefined
__proto__: Object

Expected behavior

The transcripted output should come.

[pkoelbl]

I have had this maddening problem as well. after a fair bit of checking I found that the problem is that
for await (const chunk of microphoneStream)
can sometimes pull off empty chunks and send them right over the websocket, which makes the websocket close.
You can probably put in some kind of if statement to stop it from doing that but when you are actually done sending data I don't think it will close the websocket correctly. I ended up switching to use my own websocket instead of using the client. It is a lot more legwork but I don't see a good way around it at this time.

[kristajg]

I am having this issue as well. It would be helpful to get an error message, it's very difficult to debug. I tried the if statement @pkoelbl mentioned like this:

transcribeInput = async function* () {
  for await(const chunk of micStream) {
    if (chunk && chunk.length) {
      yield { AudioEvent: { AudioChunk: pcmEncodeChunk(chunk) } }
    }
  }
}

After cycling through a few chunks it closes the websocket without explanation

[Abel-Liu]

I am facing this issue, I tried the if statement and it didn't work, any other solution? @kristajg @pkoelbl
Thanks.

[ajredniwja]

Hey guys, can anyone share the audio file they are using in order for me to reproduce this issue?

[Abel-Liu]

Hi @ajredniwja we didn't have audio file, we are reading microphone stream.
Here is sample code: code.zip

Packages used:
"@aws-sdk/client-cognito-identity": "^3.32.0",
"@aws-sdk/client-transcribe-streaming": "^3.36.0",
"@aws-sdk/credential-provider-cognito-identity": "^3.32.0",

(For my situation, this code worked fine a month ago)

[alonsogodinez]

I'm facing the same problem, any thoughts?

[alonsogodinez]

HI @ajredniwja, I tested Transcribe streaming client using your sdk websocket handler (copied and added some logs).
And found that the reason for losing socket connection is that at some point I was sending a chunk with a larger size than 96200 which is the maximum for these sockets (48000 Hz * 2).
I believe I should be able to prevent it somehow with the microphone-stream library (used in aws-docs sample) or whatever I decide to use this time, but there is certainly a missed case for error handling in AWS-SDK so that it should throw a standard exception

this is how I'm handling it for now:

const MAX_AUDIO_CHUNK_SIZE = 48000
export const getAudioStreamAsyncGenerator = (micStream: MicrophoneStream) => {
  return async function* () {
    for await (const chunk of micStream as unknown as Iterable<Buffer>) {
      if (chunk.length <= MAX_AUDIO_CHUNK_SIZE) {
        yield {
          AudioEvent: {
            AudioChunk: encodeChunkToPcm(chunk),
          },
        }
      }
    }
  }
}

[yashjais]

Thanks, @alonsogodinez. The error disappeared after implementing this check.