New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
#1upx91t fix (cpp): Modify ASRStream implementation to better respond to results. #70
Conversation
It turns out that the bool returned by the underlying gRPC stream Write() method, didn't quite mean what I thought it meant (C++ API docs aren't very clear on this point). The Write() method will return false when the stream is closed, but not necessarily when the result is available. However, the result IS available as soon as the stream's Finish() method returns. Experiments confirm that this function can return a good 400-500ms before the Write() method returns false. So, to improve the notification of when a result is available, the ASRStream implementation was updated to listen for a result (via the blocking Finish method) on a separate thread. Using an atomic bool as a flag, we then let the calling thread know when a result is available by returning false from the ASRStream's send methods once the flag is set. This greatly improves the time it takes for a client to get the ASR result. Also simplified the creation of the ASRStream by passing in the client Stub pointer, and better organized the class by storing internal class data in an opaque pointer (also helps with issues related to storing a std::thread object in a class).
This PR is not linked to a task in ClickUp. Please add the task ID starting with "CU-" (e.g. CU-1umz2q2) as the first word in the title. To create a new task in the list click here. |
Your Render PR Server URL is https://sdk-diatheke-pr-70.onrender.com. Follow its progress at https://dashboard.render.com/static/srv-c7jfu2d0mal0g1o5aa50. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good! must have been a nightmare figuring this out.
Please add the ClickUp ID to the PR title |
This PR is not linked to a task in ClickUp. Please add the task ID starting with "CU-" (e.g. CU-1umz2q2) as the first word in the title. To create a new task in the list click here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wow! Great job finding it and fixing it in a thread-safe manner!
It turns out that the bool returned by the underlying gRPC stream
Write() method, didn't quite mean what I thought it meant (C++ API
docs aren't very clear on this point). The Write() method will
return false when the stream is closed, but not necessarily when
the result is available. However, the result IS available as soon
as the stream's Finish() method returns. Experiments confirm that
this function can return a good 400-500ms before the Write() method
returns false. So, to improve the notification of when a result is
available, the ASRStream implementation was updated to listen for
a result (via the blocking Finish method) on a separate thread.
Using an atomic bool as a flag, we then let the calling thread know
when a result is available by returning false from the ASRStream's
send methods once the flag is set. This greatly improves the time
it takes for a client to get the ASR result.
Also simplified the creation of the ASRStream by passing in the
client Stub pointer, and better organized the class by storing
internal class data in an opaque pointer (also helps with issues
related to storing a std::thread object in a class).