New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/rpc: Streaming API for RPC #6569

Closed
gopherbot opened this Issue Oct 11, 2013 · 7 comments

Comments

Projects
None yet
6 participants
@gopherbot

gopherbot commented Oct 11, 2013

by tejorupan:

package net/rpc does not have any provision to support streaming.

Use-case (1) is retrieving 1000s of records of data from a database using SQL, in a
loop. We may not know how many rows are going to be retrieved nor we would like to store
all the records in memory and do the marshalling of the whole data at once, which may
not be memory efficient.

Use case (2) is to transmit bulk data, for example the contents of a file, whose size is
several mega/giga bytes.

For the above mentioned cases, we avoid the RPC layer and directly work on the
underlying network connection.

One interesting solution for streaming rpc is
https://github.com/youtube/vitess/tree/master/go/rpcplus‎ 

We need a provision to process data in chunks (streaming mode) instead of waiting for
the whole datastructure to be populated first.
@ugorji

This comment has been minimized.

@robpike

This comment has been minimized.

Contributor

robpike commented Oct 11, 2013

Comment 2:

Labels changed: added priority-later, removed priority-triage.

Status changed to Accepted.

@rsc

This comment has been minimized.

Contributor

rsc commented Nov 27, 2013

Comment 3:

Labels changed: added go1.3maybe.

@dsymonds

This comment has been minimized.

Member

dsymonds commented Nov 27, 2013

Comment 4:

Labels changed: added packagechange, feature.

@rsc

This comment has been minimized.

Contributor

rsc commented Dec 4, 2013

Comment 5:

Labels changed: added release-none, removed go1.3maybe.

@rsc

This comment has been minimized.

Contributor

rsc commented Dec 4, 2013

Comment 6:

Labels changed: added repo-main.

@mikioh

This comment has been minimized.

Contributor

mikioh commented Nov 27, 2016

Closing as per #16844.

@mikioh mikioh closed this Nov 27, 2016

@golang golang locked and limited conversation to collaborators Nov 27, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.