Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
net/rpc: Streaming API for RPC #6569
package net/rpc does not have any provision to support streaming. Use-case (1) is retrieving 1000s of records of data from a database using SQL, in a loop. We may not know how many rows are going to be retrieved nor we would like to store all the records in memory and do the marshalling of the whole data at once, which may not be memory efficient. Use case (2) is to transmit bulk data, for example the contents of a file, whose size is several mega/giga bytes. For the above mentioned cases, we avoid the RPC layer and directly work on the underlying network connection. One interesting solution for streaming rpc is https://github.com/youtube/vitess/tree/master/go/rpcplus We need a provision to process data in chunks (streaming mode) instead of waiting for the whole datastructure to be populated first.
Some previous discussion on this is at: https://groups.google.com/d/topic/golang-dev/q-xC7SnHYkY/discussion https://groups.google.com/d/topic/golang-nuts/zQdDCkPfyuU/discussion