Bidirectional streaming via gRPC

[naktinis]

Google Cloud Speech API has the "Streaming Speech Recognition" feature, that is available when using gRPC requests, which "allows you to stream audio to the Cloud Speech API and receive a stream [of] speech recognition results in real time as the audio is processed".

I believe this is possible due to the Bidirectional streaming RPC that is described in gRPC documentation whereby "the server and client could “ping-pong”: the server gets a request, then sends back a response, then the client sends another request based on the response, and so on".

So now I'm wondering whether I can achieve something similar to Google Cloud Speech API streaming recognition using TensorFlow Serving. The only reference I could find about this in the official docs of TF Serving (unless I missed something) was when describing possible future improvements: "Servables can be of any type and interface, enabling flexibility and future improvements such as: streaming results [...]".

I've asked on StackOverflow about it, but received no responses by the time of writing this.

If this is in fact already possible, then feel free to point me to the documentation. Otherwise, this would be a really useful feature to me.

[chrisolston]

TF-Serving does not support this use-case out of the box at this time, but as you allude to it would be possible to leverage its extensiblity to do so without doing major surgery on the internals. We do not currently have any short-term plans to add it to the official distribution.

[naktinis]

@chrisolston could you maybe give a rough sketch of what would be the best way to proceed if I wanted to add this functionality?

Also, would you consider accepting a pull request with such functionality?

[chrisolston]

Unfortunately this falls outside the scope of what we can support at this time, so you'll have to devise a solution and keep it in your own repo.

[ivelin]

@naktinis there is a thread related to this issue on TF SIG IO that you may want to look at:
tensorflow/io#50