Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: separate strings out of primary SynthesizeStream pipe #957

Merged
merged 3 commits into from
Sep 17, 2019
Merged

feat: separate strings out of primary SynthesizeStream pipe #957

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c2fc175
feat: separate strings out of primary SynthesizeStream pipe
MasterOdin Sep 16, 2019
a3acd2d
fix: review comments
MasterOdin Sep 17, 2019
e7aabca
fix: address error emit comment
MasterOdin Sep 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 31 additions & 6 deletions lib/synthesize-stream.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ class SynthesizeStream extends Readable {
* Note that the WebSocket connection is not established until the first chunk of data is recieved. This allows for IAM token request management by the SDK.
*
* @param {Object} options
* @param {String} options.text - The text that us to be synthesized. Provide plain text or text that is annotated with SSML. SSML input can include the SSML <mark> element. Pass a maximum of 5 KB of text.
* @param {String} options.text - The text that us to be synthesized. Provide plain text or text that is annotated with SSML. SSML input can include the SSML <mark> element. Pass a maximum of 5 KB of text.
* @param {String} options.accept - The requested audio format (MIME type) of the audio.
* @param {String[]} [options.timings] - An array that specifies whether the service is to return word timing information for all strings of the input text
* @param {String} [options.voice='en-US_MichaelVoice'] - The voice that is to be used for the synthesis.
Expand Down Expand Up @@ -98,7 +98,7 @@ class SynthesizeStream extends Readable {

const url =
(options.url || 'wss://stream.watsonplatform.net/text-to-speech/api')
.replace(/^http/, 'ws') +
.replace(/^http/, 'ws') +
'/v1/synthesize?' +
queryString;

Expand Down Expand Up @@ -126,17 +126,42 @@ class SynthesizeStream extends Readable {

socket.onmessage = message => {
const chunk = message.data;
// some messages are strings - emit those unencoded, but push them to
// the stream as binary
const data = typeof chunk === 'string' ? chunk : Buffer.from(chunk);
// some info messages are sent as strings, telling the content_type and
// timings. Emit them as separate events, but do not send them along the
// pipe.
if (typeof chunk === 'string') {
try {
const json = JSON.parse(chunk);
MasterOdin marked this conversation as resolved.
Show resolved Hide resolved
if (json['binary_streams']) {
self.emit('binary_streams', message, chunk);
}
else if (json['marks']) {
self.emit('marks', message, chunk);
}
else if (json['words']) {
self.emit('words', message, chunk);
}
else if (json['error']) {
self.emit('error', message, chunk);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only one that makes me uneasy because the other existing error event has one arg, which is an Error object that's been formatted a little:

    socket.onerror = event => {
      const err = new Error('WebSocket connection error');
      err.name = SynthesizeStream.WEBSOCKET_CONNECTION_ERROR;
      err['event'] = event;
      self.emit('error', err);
      self.push(null);
    };

it is very likely that users already have handlers set up for the Error event expecting this type of object. I think it would be prudent here to either a) rename this event to something unique or b) try to put together a similarly formatted Error object to send back as the first arg. There are pros and cons to both - what do you think?

Copy link
Contributor Author

@MasterOdin MasterOdin Sep 17, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deleted my previous response, after re-reading the docs, it is not clear to me what the behavior on error from the socket, so I'd like to test some stuff first before giving a response.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, so played around a bit more with this and it looks like for things like messing up the codec or forgetting text argument, the JSON object is sent over the onmessage channel to the user and then with an additional message on the onclose that says "see previous message for the error details". I'd probably lean towards making them emit on the same channel (and unify warnings to a similar emit format, though on a different channel), as the end result is all the same regardless of the exact reason.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good to me, thanks for looking into it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything else looks good. Once you update the error handler, I'll approve and merge 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, updated the error handler to look like the onerror signature. I did not touch the warnings one as I'm not sure there's a good way to mimic the warnings as JS does not really have a good concept of warnings like python unfortunately.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I think that's fine and there's no existing API there so I'm not worried about it

}
else if (json['warnings']) {
self.emit('warnings', message, chunk);
}
}
finally {
self.emit('message', message, chunk);
MasterOdin marked this conversation as resolved.
Show resolved Hide resolved
}
return;
}

/**
* Emit any messages received over the wire, mainly used for debugging.
*
* @event SynthesizeStream#message
* @param {Object} message - frame object received from service
* @param {Object} data - a data attribute of the frame that's either a string or a Buffer/TypedArray
*/
self.emit('message', message, data);
self.emit('message', message, chunk);
MasterOdin marked this conversation as resolved.
Show resolved Hide resolved
self.push(Buffer.from(chunk));
};

Expand Down