Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chaining pipe for streams - probably stupid question but... #2384

Closed
Ayms opened this issue Jan 3, 2020 · 5 comments
Closed

Chaining pipe for streams - probably stupid question but... #2384

Ayms opened this issue Jan 3, 2020 · 5 comments
Labels

Comments

@Ayms
Copy link

Ayms commented Jan 3, 2020

The test code is simply:

const stream=require('stream');
let stream1=new stream.Duplex();
let stream2=new stream.Duplex();
let stream3=new stream.Duplex();
stream1._read=function() {let data=this.read();if (data) {console.log('stream1 read '+data.toString())}};
stream2._read=function() {let data=this.read();if (data) {console.log('stream2 read '+data.toString())}};
stream3._read=function() {let data=this.read();if (data) {console.log('stream3 read '+data.toString())}};
stream1._write=function(data) {console.log('stream1 receive write '+data.toString())};
stream2._write=function(data) {console.log('stream2 receive write '+data.toString())};
stream3._write=function(data) {console.log('stream3 receive write '+data.toString())};
stream3.pipe(stream2).pipe(stream1);
stream3.push(new Buffer('hello','utf8'));

The result is:

stream2 receive write hello
stream3 read hello

Then stream1 receives nothing, there are of course different ways to solve this but probably I am missing something obvious because if we have to implement something like this.push(data) in stream2._write then I don't see the use of chaining pipes (we can do stream3.write(data))

The doc states that we can pipe from a readable and chain writable streams (the result of the pipe here is a duplex stream, the doc does not say how the pipe handles the write from the writable to the read of the readable to send the data forward to the piped stream)

The example is always the same using gzip, after some research I did not find any other examples except duplex pairs which looks strange since we can wonder then why we are using Duplex streams

That's why I am asking

@mscdex mscdex transferred this issue from nodejs/node Jan 3, 2020
@addaleax
Copy link
Member

addaleax commented Jan 4, 2020

I’m not sure if that’s what you’re asking, but there is no connection between the writable side of stream2 and the readable side of stream2 here; and because no data is read from stream2, only written to it, no data is written to stream1 either.

@Ayms
Copy link
Author

Ayms commented Jan 5, 2020

Indeed, if you don't do anything with stream2 then no magic happens with pipe

Now I still have some hard time to get the pipe behavior, once data are processed then you push on the related stream and then pipe writes on the piped stream, OK

Now the use case is: a.pipe(b).etc.pipe(z) bidirectionnal

Which I think is used everywhere by node (http.pipe(gzip).pipe(socket)) so it should be easy

The way forward from a to z is clear, how do you handle the way backward without piping z to a or using something like a duplex pair?

Personnally I would have thought that one way is write/write/write and the other way read/read/read, here we are using both to go only in one direction, or missing something again, probably some link to some code doing the intended behavior would clarify

@Ayms
Copy link
Author

Ayms commented Jan 7, 2020

This is impossible to do without implementing something like duplex pairs

And for me this is clearly a misspecification of the pipe method so I suggest you transfer it back to the bug tracker

A and B duplex stream

._write=function(data){this.push(transformed data)}

A.pipe(B).pipe(A) --> B should not write back to A if write on B comes from a read from A

And the doc states that writables can be chained via pipes, this is not correct since all pipe streams must be duplex stream and not only writables

@github-actions
Copy link

There has been no activity on this issue for 3 years and it may no longer be relevant. It will be closed 1 month after the last non-automated comment.

@github-actions github-actions bot added the stale label Jan 23, 2023
@github-actions
Copy link

There has been no activity on this issue and it is being closed. If you feel closing this issue is not the right thing to do, please leave a comment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants