stream: eos on closed #28748

ronag · 2019-07-18T11:16:40Z

stream.finished should be invoked if stream is already closed/destroyed. Currently there is a potential deadlock depending on when stream.finished is called.

For writable premature close is close before finish, i.e. it should look for finished instead of ended.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
documentation is changed or added
commit message follows commit guidelines

Trott · 2019-07-23T00:00:53Z

/ping @nodejs/streams

nodejs-github-bot · 2019-07-23T00:01:28Z

CI: https://ci.nodejs.org/job/node-test-pull-request/24624/

mcollina

Would you mind to add the equivalent test for Readable as well?

lib/internal/streams/end-of-stream.js

test/parallel/test-stream-eos.js

ronag · 2019-07-23T07:33:23Z

@mcollina updated

lib/internal/streams/end-of-stream.js

ronag · 2019-08-02T06:35:42Z

@benjamingr ping

mcollina

LGTM

mcollina · 2019-08-03T16:53:48Z

@nodejs/tsc this needs another review.

ronag · 2019-08-06T09:26:39Z

@mcollina: Should this maybe result in a ERR_STREAM_DESTROYED?

ronag · 2019-09-01T14:15:38Z

Yes please.

@mcollina: done

ronag · 2019-09-01T15:59:24Z

@Trott: Updated PR description to describe the included changes from the other PR (which is now closed).

ronag · 2019-09-01T15:59:48Z

This needs another CITGM (once Travis is ok) given the recent changes.

ronag · 2019-09-01T16:00:49Z

lib/internal/streams/end-of-stream.js

-      return callback.call(stream, err);
+      callback.call(stream, new ERR_STREAM_PREMATURE_CLOSE());
+    } else if (writable && !writableFinished) {
+      callback.call(stream, new ERR_STREAM_PREMATURE_CLOSE());
    }


There is an added test for this change. We should always be emitting premature close unless we've seen finish or end events. I don't think this actually changes anything in practice.

nodejs-github-bot · 2019-09-01T17:39:14Z

CI: https://ci.nodejs.org/job/node-test-pull-request/25315/

nodejs-github-bot · 2019-09-01T18:01:57Z

CI: https://ci.nodejs.org/job/node-test-pull-request/25317/

mcollina · 2019-09-02T14:33:39Z

This will need a rebase, otherwise it looks good to land.

ronag · 2019-09-02T14:53:18Z

rebased

nodejs-github-bot · 2019-09-02T15:11:00Z

CI: https://ci.nodejs.org/job/node-test-pull-request/25326/

mcollina · 2019-09-03T08:29:26Z

Landed in b03845b

Make stream.finished callback invoked if stream is already closed/destroyed. PR-URL: #28748 Reviewed-By: Matteo Collina <matteo.collina@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Trivikram Kamat <trivikr.dev@gmail.com>

mafintosh · 2019-09-23T13:19:19Z

A little late to the party here, but this includes a big change in behaviour for streams.

Since the finished flag is used instead of the ended flag, this PR changes the behaiviour of eos to the following:

const stream = require('stream')
const ws = new stream.Writable({
  write (data, enc, cb) {
    process.nextTick(cb, null)
  }
})

finished(ws, function (err) {
  console.log('This used to *not* error:', err)
})

ws.write('data')
ws.end()
ws.emit('close') // modules tend to emit close on resource close but in a non error state

I was just bitten by this in a debugging session, so unsure if that was an intentional change?
If so I think it should be clearly documented :)

Streams are hard.

ronag force-pushed the stream-eos-on-closed branch 8 times, most recently from 3c57be4 to c9d1e19 Compare July 22, 2019 06:42

mcollina requested changes Jul 23, 2019

View reviewed changes

lib/internal/streams/end-of-stream.js Outdated Show resolved Hide resolved

mcollina requested changes Jul 23, 2019

View reviewed changes

lib/internal/streams/end-of-stream.js Outdated Show resolved Hide resolved

test/parallel/test-stream-eos.js Outdated Show resolved Hide resolved

ronag mentioned this pull request Jul 23, 2019

stream: finished depends on internal API #28813

Closed

1 task

ronag force-pushed the stream-eos-on-closed branch from c9d1e19 to 3816f11 Compare July 23, 2019 07:30

ronag force-pushed the stream-eos-on-closed branch 2 times, most recently from aefe068 to 1588c78 Compare July 23, 2019 07:37

mcollina requested changes Jul 23, 2019

View reviewed changes

lib/internal/streams/end-of-stream.js Outdated Show resolved Hide resolved

mcollina added the semver-major PRs that contain breaking changes and should be released in the next major version. label Jul 23, 2019

ronag force-pushed the stream-eos-on-closed branch 3 times, most recently from 40b4c0b to 39ca23a Compare July 24, 2019 09:46

Fishrock123 added the stream Issues and PRs related to the stream subsystem. label Jul 24, 2019

mcollina approved these changes Aug 2, 2019

View reviewed changes

benjamingr approved these changes Aug 3, 2019

View reviewed changes

mcollina requested a review from a team August 3, 2019 16:53

ronag mentioned this pull request Aug 6, 2019

stream: cleanup #29000

Closed

2 tasks

ronag mentioned this pull request Sep 1, 2019

stream: eos premature close #28720

Closed

4 tasks

ronag commented Sep 1, 2019

View reviewed changes

ronag added 12 commits September 2, 2019 16:49

stream: eos on closed

820074e

fixup: check ended and finished

729cd05

fixup: http response doesn't have destroyed, use aborted instead

e662280

fixup: tests

3ec6ecc

fixup: more tests

27a9624

fixup: merge nodejs#28720

5641590

fixup: sligthly simplify and add more tests

41a692a

fixup: test descriptions

3be0454

fixup: linting

53c04a3

fixup: merge duplicate test files

e8284ec

fixup: comments styling

a4542b7

fixup: typo when merging tests

9b5a84e

ronag force-pushed the stream-eos-on-closed branch from db6eb4f to 9b5a84e Compare September 2, 2019 14:52

mcollina closed this Sep 3, 2019

BethGriggs mentioned this pull request Sep 9, 2019

2019-10-22, Version 13.0.0 (Current) #29504

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

stream: eos on closed #28748

stream: eos on closed #28748

ronag commented Jul 18, 2019 •

edited

Trott commented Jul 23, 2019

nodejs-github-bot commented Jul 23, 2019

mcollina left a comment

ronag commented Jul 23, 2019

ronag commented Aug 2, 2019

mcollina left a comment

mcollina commented Aug 3, 2019

ronag commented Aug 6, 2019

ronag commented Sep 1, 2019

ronag commented Sep 1, 2019

ronag commented Sep 1, 2019

ronag Sep 1, 2019 •

edited

nodejs-github-bot commented Sep 1, 2019

nodejs-github-bot commented Sep 1, 2019

mcollina commented Sep 2, 2019

ronag commented Sep 2, 2019

nodejs-github-bot commented Sep 2, 2019

mcollina commented Sep 3, 2019

mafintosh commented Sep 23, 2019

stream: eos on closed #28748

stream: eos on closed #28748

Conversation

ronag commented Jul 18, 2019 • edited

Checklist

Trott commented Jul 23, 2019

nodejs-github-bot commented Jul 23, 2019

mcollina left a comment

Choose a reason for hiding this comment

ronag commented Jul 23, 2019

ronag commented Aug 2, 2019

mcollina left a comment

Choose a reason for hiding this comment

mcollina commented Aug 3, 2019

ronag commented Aug 6, 2019

ronag commented Sep 1, 2019

ronag commented Sep 1, 2019

ronag commented Sep 1, 2019

ronag Sep 1, 2019 • edited

Choose a reason for hiding this comment

nodejs-github-bot commented Sep 1, 2019

nodejs-github-bot commented Sep 1, 2019

mcollina commented Sep 2, 2019

ronag commented Sep 2, 2019

nodejs-github-bot commented Sep 2, 2019

mcollina commented Sep 3, 2019

mafintosh commented Sep 23, 2019

ronag commented Jul 18, 2019 •

edited

ronag Sep 1, 2019 •

edited