Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: do not reuse HTTPParser #25094

Closed
wants to merge 3 commits into from

Conversation

@Drieger
Copy link
Contributor

commented Dec 17, 2018

Change resource being used, previously HTTParser was being reused. We are now using IncomingMessage and ClientRequest objects. The goal here is to make the async resource unique for each async operation

Refs: #24330
Refs: nodejs/diagnostics#248
Refs: #21313

Co-authored-by: Matheus Marchini matheus@sthima.com

I continued @mmarchini work on #24330 and finished what was missing. I also added a test based on @AndreasMadsen comment nodejs/diagnostics#248 (comment)

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@mmarchini mmarchini referenced this pull request Dec 17, 2018

Closed

WIP http_parser: don't reuse HTTPParser #24330

3 of 3 tasks complete

@Drieger Drieger force-pushed the Drieger:avoid_reuse_of_HTTPParser branch from 368187e to 5d9eebd Dec 17, 2018

@mmarchini

This comment has been minimized.

Copy link
Member

commented Dec 17, 2018

@mmarchini

This comment has been minimized.

Copy link
Member

commented Dec 17, 2018

@mcollina
Copy link
Member

left a comment

LGTM

@Drieger

This comment has been minimized.

Copy link
Contributor Author

commented Dec 17, 2018

The problem with linter is that common is being imported but not used, but when I do not import common linter error is that it is mandatory. Any ideas how to solve this one?

@richardlau

This comment has been minimized.

Copy link
Member

commented Dec 17, 2018

The problem with linter is that common is being imported but not used, but when I do not import common linter error is that it is mandatory. Any ideas how to solve this one?

Just write:

require('../common');

or alternatively use common (e.g. common.mustCall).

@Flarna

This comment has been minimized.

Copy link
Member

commented Dec 17, 2018

If I understand this correct the resource type is still HTTPPARSER but depending on incoming/outgoing the resource passed in init differs. I think this is confusing.

Wouldn't it better to have two resource types instead?

@mscdex

This comment has been minimized.

Copy link
Contributor

commented Dec 18, 2018

Perhaps the commit message could be more clear. 'do not reuse HTTPParser' makes it sound as if parser objects are not being reused, when in fact it's the associated async resource that's no longer being reused (if I understand correctly).

@AndreasMadsen
Copy link
Member

left a comment

Thanks for spending time on this.

Show resolved Hide resolved src/async_wrap.cc
@Drieger

This comment has been minimized.

Copy link
Contributor Author

commented Dec 19, 2018

@Flarna maybe we could use IncomingMessage and ClientRequest instead? What do you think?

@Flarna

This comment has been minimized.

Copy link
Member

commented Dec 19, 2018

Not sure here; maybe prefix with Http as these names are quite generic in the global AsyncHooks scope.

parser.reinitialize(HTTPParser.REQUEST, parser[is_reused_symbol]);
parser.reinitialize(HTTPParser.REQUEST,
parser[is_reused_symbol],
server[kIncomingMessage]);

This comment has been minimized.

Copy link
@Flarna

Flarna Dec 19, 2018

Member

Isn't this reusing the IncomingMessage constructor function instead of the concrete instance? If I understand the code correct the IncomingMessage instance is created in parserOnHeadersComplete.
If I'm correct I wonder why the dedicated test-httparser-reuse.js is not detecting this.

This comment has been minimized.

Copy link
@Flarna

Flarna Jan 10, 2019

Member

@Drieger This one is still open and if I'm correct this is a no go.

This comment has been minimized.

Copy link
@Drieger

Drieger Jan 10, 2019

Author Contributor

You are right @Flarna I'll investigate this more in depth.

This comment has been minimized.

Copy link
@Drieger

Drieger Jan 11, 2019

Author Contributor

@Flarna kIncomingMessage is really a function as you said. I fixed the test to catch this.
I will also create a new object to be the resource for the server. Some other changes will be necessary as well. I will update the PR as soon as I this is working.

@Drieger

This comment has been minimized.

Copy link
Contributor Author

commented Jan 9, 2019

Not sure here; maybe prefix with Http as these names are quite generic in the global AsyncHooks scope.

@Flarna I just updated the PR, adding two new providers HTTPINCOMINGMESSAGE and HTTPCLIENTREQUEST, could you take a look if that is what you had in mind?

I'll check why is the CI failing.

@Flarna

This comment has been minimized.

Copy link
Member

commented Jan 9, 2019

Yes, looks ok. I think an alternative would be HTTP_REQUEST/HTTP_RESPONSE like in code which would point to what happens instead naming the resource type used (currently).

I would love if others give also their comments on these names as they are actually public API (even it's just in experimental part).

Is HTTPPARSER still needed?

https://github.com/nodejs/node/blob/master/doc/api/async_hooks.md needs also some update once agreement on the names is reached.

@Drieger

This comment has been minimized.

Copy link
Contributor Author

commented Jan 9, 2019

You are right, I think HTTPPARSER is not necessary anymore. I'll update doc as soon as the "final" name is chosen.

@Drieger

This comment has been minimized.

Copy link
Contributor Author

commented Jan 10, 2019

Fix test issues!
I'll update commit message after all changes have been approved.

/cc @mmarchini @mcollina @AndreasMadsen @richardlau @mscdex could you check again? What do you think about the naming suggested by @Flarna?

@Drieger

This comment has been minimized.

Copy link
Contributor Author

commented Jan 14, 2019

I changed the resource in the _http_server.js file and fixed the test that was not able to capture the previous situation (function was being reused). I think it would be a good idea to run benchmarks if everyone agrees with the current implementation.

Obs.: Documentation and commit message will be updated if everything is ok.

// Force reinitilalization to make sure AsyncReset is called
parser.reinitialize(
HTTPParser.REQUEST,
true,

This comment has been minimized.

Copy link
@mcollina

mcollina Jan 15, 2019

Member

I think we should remove this parameter, or is it used somewhere else?

This comment has been minimized.

Copy link
@Drieger

Drieger Jan 15, 2019

Author Contributor

Removed parameter and renamed the method.

@mcollina

This comment has been minimized.

@mcollina

This comment has been minimized.

Copy link
Member

commented Jan 19, 2019

@mcollina mcollina referenced this pull request Jan 28, 2019

Open

process: improve nextTick performance #25461

2 of 2 tasks complete
@Drieger

This comment has been minimized.

Copy link
Contributor Author

commented Mar 6, 2019

I think I addressed all previous comments, any other comments on this? @nodejs/diagnostics @nodejs/async_hooks

@mcollina
Copy link
Member

left a comment

LGTM

@mcollina

This comment has been minimized.

Copy link
Member

commented Mar 7, 2019

I've tagged this semver-major because of the impact it can have on async_hooks users. It's not technically required because async_hooks are experimental, but they are widely used.
I'm fine in being semver-minor with a dont-land-on-8.x and dont-land-on-10.x label as well.

@refack refack referenced this pull request Apr 23, 2019

Closed

benchmark: fix http/bench-parser.js #27360

3 of 3 tasks complete

Flarna added a commit to Flarna/node that referenced this pull request Apr 23, 2019

test: replace HTTPPARSER by new resource names
The async resource `HTTPPARSER` has been replaced by
`HTTPINCOMINGMESSAGE` and `HTTPCLIENTREQUEST` but some tests have
not been adapted.

As test test-graph.http.js has not failed even it asserted on
´HTTPPARSER` I improved `verify-graph` and adapted the asserts in
test-graph.tls-write which failed because of the stricter check.

Refs: nodejs#25094

Flarna added a commit to Flarna/node that referenced this pull request Apr 23, 2019

test: replace HTTPPARSER by new resource names
The async resource `HTTPPARSER` has been replaced by
`HTTPINCOMINGMESSAGE` and `HTTPCLIENTREQUEST` but some tests have
not been adapted.

As test test-graph.http.js has not failed even it asserted on
´HTTPPARSER` I improved `verify-graph` and adapted the asserts in
test-graph.tls-write which failed because of the stricter check.

Refs: nodejs#25094

Flarna added a commit to Flarna/node that referenced this pull request Apr 23, 2019

test: replace HTTPPARSER by new resource names
The async resource `HTTPPARSER` has been replaced by
`HTTPINCOMINGMESSAGE` and `HTTPCLIENTREQUEST` but some tests have
not been adapted.

As test test-graph.http.js has not failed even it asserted on
´HTTPPARSER` I improved `verify-graph` and adapted the asserts in
test-graph.tls-write which failed because of the stricter check.

Refs: nodejs#25094

@Flarna Flarna referenced this pull request Apr 23, 2019

Closed

test: replace HTTPPARSER by new resource names #27372

3 of 3 tasks complete

Flarna added a commit to Flarna/node that referenced this pull request Apr 23, 2019

test: replace HTTPPARSER by new resource names
The async resource `HTTPPARSER` has been replaced by
`HTTPINCOMINGMESSAGE` and `HTTPCLIENTREQUEST` but some tests have
not been adapted.

As test test-graph.http.js has not failed even it asserted on
´HTTPPARSER` I improved `verify-graph` and adapted the asserts in
test-graph.tls-write which failed because of the stricter check.

Refs: nodejs#25094

Flarna added a commit to Flarna/node that referenced this pull request Apr 29, 2019

async_hooks: Fixup do not reuse HTTPParser
Fix some issues introduced/not fixed via
nodejs#25094:
* Init hook is not emitted for a reused HTTPParser
* HTTPParser was still used as resource in init hook
* type used in init hook was always HTTPINCOMINGMESSAGE even for client
requests
* some tests have not been adapted to new resource names

With this change the async hooks init event is emitted during a call
to Initialize() as the type and resource object is available at this
time. As a result Initialize() must be called now which could be seen
as breaking change even HTTPParser is not part of documented API.

It was needed to put the ClientRequest instance into a wrapper object
instead passing it directly as async resource otherwise
test-domain-multi fails. I think this is because adding an EventEmitter
to a Domain adds a property 'domain' and the presence of this changes
the context propagation in domains.

Besides that tests still refering to resource HTTPParser have been
updated/improved.

Fixes: nodejs#27467
Fixes: nodejs#26961
Refs: nodejs#25094

Flarna added a commit to Flarna/node that referenced this pull request Apr 29, 2019

async_hooks: Fixup do not reuse HTTPParser
Fix some issues introduced/not fixed via
nodejs#25094:
* Init hook is not emitted for a reused HTTPParser
* HTTPParser was still used as resource in init hook
* type used in init hook was always HTTPINCOMINGMESSAGE even for client
requests
* some tests have not been adapted to new resource names

With this change the async hooks init event is emitted during a call
to Initialize() as the type and resource object is available at this
time. As a result Initialize() must be called now which could be seen
as breaking change even HTTPParser is not part of documented API.

It was needed to put the ClientRequest instance into a wrapper object
instead passing it directly as async resource otherwise
test-domain-multi fails. I think this is because adding an EventEmitter
to a Domain adds a property 'domain' and the presence of this changes
the context propagation in domains.

Besides that tests still refering to resource HTTPParser have been
updated/improved.

Fixes: nodejs#27467
Fixes: nodejs#26961
Refs: nodejs#25094

@Flarna Flarna referenced this pull request Apr 29, 2019

Closed

async_hooks: fixup do not reuse HTTPParser #27477

3 of 3 tasks complete

Flarna added a commit to Flarna/node that referenced this pull request Apr 29, 2019

async_hooks: fixup do not reuse HTTPParser
Fix some issues introduced/not fixed via
nodejs#25094:
* Init hook is not emitted for a reused HTTPParser
* HTTPParser was still used as resource in init hook
* type used in init hook was always HTTPINCOMINGMESSAGE even for client
requests
* some tests have not been adapted to new resource names

With this change the async hooks init event is emitted during a call
to Initialize() as the type and resource object is available at this
time. As a result Initialize() must be called now which could be seen
as breaking change even HTTPParser is not part of documented API.

It was needed to put the ClientRequest instance into a wrapper object
instead passing it directly as async resource otherwise
test-domain-multi fails. I think this is because adding an EventEmitter
to a Domain adds a property 'domain' and the presence of this changes
the context propagation in domains.

Besides that tests still refering to resource HTTPParser have been
updated/improved.

Fixes: nodejs#27467
Fixes: nodejs#26961
Refs: nodejs#25094

addaleax added a commit that referenced this pull request May 3, 2019

async_hooks: fixup do not reuse HTTPParser
Fix some issues introduced/not fixed via
#25094:
* Init hook is not emitted for a reused HTTPParser
* HTTPParser was still used as resource in init hook
* type used in init hook was always HTTPINCOMINGMESSAGE even for client
requests
* some tests have not been adapted to new resource names

With this change the async hooks init event is emitted during a call
to Initialize() as the type and resource object is available at this
time. As a result Initialize() must be called now which could be seen
as breaking change even HTTPParser is not part of documented API.

It was needed to put the ClientRequest instance into a wrapper object
instead passing it directly as async resource otherwise
test-domain-multi fails. I think this is because adding an EventEmitter
to a Domain adds a property 'domain' and the presence of this changes
the context propagation in domains.

Besides that tests still refering to resource HTTPParser have been
updated/improved.

Fixes: #27467
Fixes: #26961
Refs: #25094

PR-URL: #27477
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>

targos added a commit that referenced this pull request May 4, 2019

async_hooks: fixup do not reuse HTTPParser
Fix some issues introduced/not fixed via
#25094:
* Init hook is not emitted for a reused HTTPParser
* HTTPParser was still used as resource in init hook
* type used in init hook was always HTTPINCOMINGMESSAGE even for client
requests
* some tests have not been adapted to new resource names

With this change the async hooks init event is emitted during a call
to Initialize() as the type and resource object is available at this
time. As a result Initialize() must be called now which could be seen
as breaking change even HTTPParser is not part of documented API.

It was needed to put the ClientRequest instance into a wrapper object
instead passing it directly as async resource otherwise
test-domain-multi fails. I think this is because adding an EventEmitter
to a Domain adds a property 'domain' and the presence of this changes
the context propagation in domains.

Besides that tests still refering to resource HTTPParser have been
updated/improved.

Fixes: #27467
Fixes: #26961
Refs: #25094

PR-URL: #27477
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>

Flarna added a commit to dynatrace-oss-contrib/node that referenced this pull request May 30, 2019

[v10.x] backport avoid double-destroy HTTPParser
Avoid that destroy hook is invoked twice - once via `Parser::Free()`
and once again via `Parser::Reinitialize()` by clearing the async_id
in `EmitDestroy()`.

Partial backport of nodejs#27477, a full
backport would require also nodejs#25094
which has a dont-land-on-v10.x label on it.

Fixes: nodejs#26961

Flarna added a commit to dynatrace-oss-contrib/node that referenced this pull request May 30, 2019

[v10.x] backport avoid double-destroy HTTPParser
Avoid that destroy hook is invoked twice - once via `Parser::Free()`
and once again via `Parser::Reinitialize()` by clearing the async_id
in `EmitDestroy()`.

Partial backport of nodejs#27477, a full
backport would require also nodejs#25094
which has a dont-land-on-v10.x label on it.

Fixes: nodejs#26961

@Flarna Flarna referenced this pull request May 30, 2019

Closed

[v10.x] async_hooks: avoid double-destroy HTTPParser #27986

2 of 2 tasks complete

Flarna added a commit to dynatrace-oss-contrib/node that referenced this pull request May 30, 2019

async_hooks: avoid double-destroy HTTPParser
Avoid that destroy hook is invoked twice - once via `Parser::Free()`
and once again via `Parser::Reinitialize()` by clearing the async_id
in `EmitDestroy()`.

Partial backport of nodejs#27477, a full
backport would require also nodejs#25094
which has a dont-land-on-v10.x label on it.

Fixes: nodejs#26961

BethGriggs added a commit that referenced this pull request Jun 28, 2019

async_hooks: avoid double-destroy HTTPParser
Avoid that destroy hook is invoked twice - once via `Parser::Free()`
and once again via `Parser::Reinitialize()` by clearing the async_id
in `EmitDestroy()`.

Partial backport of #27477, a full
backport would require also #25094
which has a dont-land-on-v10.x label on it.

Fixes: #26961

Backport-PR-URL: #27986
PR-URL: #27477
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>

BethGriggs added a commit that referenced this pull request Jul 17, 2019

async_hooks: avoid double-destroy HTTPParser
Avoid that destroy hook is invoked twice - once via `Parser::Free()`
and once again via `Parser::Reinitialize()` by clearing the async_id
in `EmitDestroy()`.

Partial backport of #27477, a full
backport would require also #25094
which has a dont-land-on-v10.x label on it.

Fixes: #26961

Backport-PR-URL: #27986
PR-URL: #27477
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.