Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stream: finished on closed OutgoingMessage #34313

Closed
wants to merge 3 commits into from

Conversation

ronag
Copy link
Member

@ronag ronag commented Jul 11, 2020

finished should invoke callback on closed OutgoingMessage the
same way as for regular streams.

Fixes: #34274

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@ronag ronag added stream Issues and PRs related to the stream subsystem. dont-land-on-v10.x labels Jul 11, 2020
@ronag ronag requested review from mcollina and lpinca July 11, 2020 21:01
@ronag
Copy link
Member Author

ronag commented Jul 11, 2020

@nodejs/streams

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rickyes
Copy link
Contributor

rickyes commented Jul 12, 2020

finished should invoke callback on closed OutgoingMessage the
same way as for regular streams.

Fixes: #34274

The fix should be #34301

finished should invoke callback on closed OutgoingMessage the
same way as for regular streams.

Fixes: nodejs#34301
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member Author

ronag commented Jul 14, 2020

@Trott Help. I can't get node-test-commit-custom-suites-freestyle (test-worker) to pass (test.parallel/test-bootstrap-modules).

@Trott
Copy link
Member

Trott commented Jul 14, 2020

@Trott Help. I can't get node-test-commit-custom-suites-freestyle (test-worker) to pass (test.parallel/test-bootstrap-modules).

Does it fail for you locally with tools/test.py --worker test/parallel/test-bootstrap-modules.js?

@richardlau
Copy link
Member

@Trott Help. I can't get node-test-commit-custom-suites-freestyle (test-worker) to pass (test.parallel/test-bootstrap-modules).

Does it fail for you locally with tools/test.py --worker test/parallel/test-bootstrap-modules.js?

👍

This test is designed to highlight when we cause extra modules to be loaded during the bootstrap and make sure that we're are doing that conciously. In this case, we're now dragging in https://github.com/nodejs/node/blob/master/lib/internal/http.js during bootstrap with workers because workers uses streams. This is in turn also dragging in the internal performance binding:

const { PerformanceEntry, notify } = internalBinding('performance');

If loading those extra modules now when we bootstrap worker threads is acceptable, add them to the appropriate parts of the test (there's a conditional section for workers). Otherwise consider lazy loading the extra module(s) so they are only loaded if needed.

@ronag
Copy link
Member Author

ronag commented Jul 14, 2020

I'm unsure how to fix this then... where do we put kClosed which is only used for http?

lib/internal/streams/end-of-stream.js Outdated Show resolved Hide resolved
@nodejs-github-bot
Copy link
Collaborator

@ronag
Copy link
Member Author

ronag commented Jul 15, 2020

@nodejs/http

@ronag ronag added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label Jul 15, 2020
@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ronag
Copy link
Member Author

ronag commented Jul 16, 2020

Landed in a55b77d

@ronag ronag closed this Jul 16, 2020
ronag added a commit that referenced this pull request Jul 16, 2020
finished should invoke callback on closed OutgoingMessage the
same way as for regular streams.

Fixes: #34301

PR-URL: #34313
Fixes: #34274
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
@vweevers
Copy link
Contributor

Could be an argument for reconsidering #33990

@ronag
Copy link
Member Author

ronag commented Jul 16, 2020

Could be an argument for reconsidering #33990

I don't think internal difficulties should motivate public APIs?

@vweevers
Copy link
Contributor

OutgoingMessage has ._closed, fs streams have .closed, might be worth consolidating. Without it, can finished() detect already-closed userland streams?

@ronag
Copy link
Member Author

ronag commented Jul 16, 2020

Without it, can finished() detect already-closed userland streams?

Yes? Using the internal property _state.closed? Depends on what you mean by "userland streams". You mean streams that are streamlike but not actually streams?

OutgoingMessage has ._closed

internal property

fs streams have .closed

public property

I'm not against it (I opened that PR after all) but I'm not for it either. I don't think this specific case motivates it.

cjihrig pushed a commit that referenced this pull request Jul 23, 2020
finished should invoke callback on closed OutgoingMessage the
same way as for regular streams.

Fixes: #34301

PR-URL: #34313
Fixes: #34274
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. stream Issues and PRs related to the stream subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stream.pipeline does not wait for the last stream to flush before calling the final callback
8 participants