Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

lib: use Writable#_final for 'finish' tracking #27

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

indutny-signal
Copy link

Instead overriding emit method and using this._realFinish to detect
unexpected end of data - implement _final method and track the logic
there.

The only behavior change from the use of _final is that we would no
longer emit finish when the input data is terminated early. Instead we
would emit error as before and stop the processing. Note that this is
the behavior provided by stream.Writable, and it is thus conformant
with the specification.

Additionally, replace uses of private _events property with either
straight emit() with return value check, or listenerCount() in the
situations where there might be an overhead from the constructed event
arguments.

Fix: #26

cc @mscdex (this is @indutny under disguise)

@indutny-signal
Copy link
Author

I didn't run the formidable benchmarks, but the ones in the repo appears to be giving roughly the same timing as before the patch:

indutny@Fedors-MacBook-Pro-2 dicer % git co master 
Switched to branch 'master'
Your branch is up to date with 'origin/master'.
indutny@Fedors-MacBook-Pro-2 dicer % node bench/bench.js dicer
dicer: 419.234ms
indutny@Fedors-MacBook-Pro-2 dicer % node bench/bench.js dicer
dicer: 432.782ms
indutny@Fedors-MacBook-Pro-2 dicer % git co fix/pipeline-incompat 
Switched to branch 'fix/pipeline-incompat'
Your branch is up to date with 'sig/fix/pipeline-incompat'.
indutny@Fedors-MacBook-Pro-2 dicer % node bench/bench.js dicer    
dicer: 418.913ms
indutny@Fedors-MacBook-Pro-2 dicer % node bench/bench.js dicer
dicer: 423.359ms

@indutny-signal
Copy link
Author

FWIW, I've tested it on our codebase (Signal-Desktop) and the pipeline() test pass there as well.

@indutny-signal
Copy link
Author

Welp, sorry for CI failures. Force pushed.

@indutny-signal
Copy link
Author

Oh, it is actually failing on older node.js versions... Weird.

@indutny-signal
Copy link
Author

Ah, stream/promises.

Instead overriding `emit` method and using `this._realFinish` to detect
unexpected end of data - implement `_final` method and track the logic
there.

The only behavior change from the use of `_final` is that we would no
longer emit `finish` when the input data is terminated early. Instead we
would emit `error` as before and stop the processing. Note that this is
the behavior provided by `stream.Writable`, and it is thus conformant
with the specification.

Additionally, replace uses of private `_events` property with either
straight `emit()` with return value check, or `listenerCount()` in the
situations where there might be an overhead from the constructed event
arguments.

Replace `emit('error', error)` with `destroy(error)` calls as well, as
otherwise the behavior is undefined.

Fix: mscdex#26
@indutny-signal
Copy link
Author

Oof, this was tricky. Node 10 and 12 have different event behavior from 14 and 16. In particular, I get both error and end events on part and dicer on <= 12, and only 'error' on > 12.

Additionally, I had to delay the cb() in _final because it wasn't giving the part a chance to fire its error on Node <= 12.

Weird stuff!

@indutny-signal
Copy link
Author

Anyway, it is all fixed now, although I'm much less happy with the PR than I was at the start. (Hacks 😭)

@indutny
Copy link

indutny commented Mar 2, 2022

Yay, all green! 😁

@indutny-signal
Copy link
Author

Totally no rush with this. Just let me know if you'll ever need any additional information or more detailed description of the changes to help you review this. Thanks!

@indutny-signal
Copy link
Author

(Friendly weekly reminder so that this PR doesn't fall through the cracks of your inbox)

@indutny-signal
Copy link
Author

(Friendly reminder after one month 😉)

@indutny-signal
Copy link
Author

(This PR is still open 😁 )

@JoshuaWise
Copy link

I ran into this issue today, using pipeline from stream/promises. Wondering why this PR hasn't been merged 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dicer hangs when used with stream/promises' pipeline
3 participants