New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http: Eliminate ClientRequest capture by Agent socket listeners #10134

Closed
wants to merge 4 commits into
base: master
from

Conversation

Projects
None yet
10 participants
@evantorrie
Contributor

evantorrie commented Dec 6, 2016

Checklist
  • make -j8 test (UNIX), or vcbuild test nosign (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines
Affected core subsystem(s)

http

Description of change

Eliminate capture of createSocket callback in onFree/onClose/onRemote listeners by moving them to a separate function.

Fixes #10133

This reduces the heap usage by eliminating the capture in a prior context of the ClientRequest object associated with the first call that opens a socket. Let me know the best way to provide tests for this, as it's non-obvious to me how to do so.

I have provided a heapsnapshot screen shot in the accompanying issue (#10133), and will attach a similar heapsnapshot screen shot showing usage after this fix is included.
heap_noretainer

Show outdated Hide outdated lib/_http_agent.js
@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 6, 2016

Contributor

@cjihrig Changed per review

Contributor

evantorrie commented Dec 6, 2016

@cjihrig Changed per review

Show outdated Hide outdated lib/_http_agent.js
@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 7, 2016

Contributor

@cjihrig Changes made as requested.

Contributor

evantorrie commented Dec 7, 2016

@cjihrig Changes made as requested.

Show outdated Hide outdated lib/_http_agent.js
@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 8, 2016

Contributor

Removed the blank line. @cjihrig

Contributor

evantorrie commented Dec 8, 2016

Removed the blank line. @cjihrig

@bnoordhuis

LGTM but can you make sure the commit log conforms to the style guide from CONTRIBUTING.md?

@evantorrie evantorrie changed the title from Eliminate capture of "cb" in createSocket context to http: Eliminate capture of "cb" by Agent socket listeners Dec 9, 2016

@evantorrie evantorrie changed the title from http: Eliminate capture of "cb" by Agent socket listeners to http: Eliminate ClientRequest capture by Agent socket listeners Dec 9, 2016

@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 9, 2016

Contributor

@bnoordhuis Does that look better? Do you want me to rebase/squash? Or is that something you do on your end?

Contributor

evantorrie commented Dec 9, 2016

@bnoordhuis Does that look better? Do you want me to rebase/squash? Or is that something you do on your end?

@cjihrig

This comment has been minimized.

Show comment
Hide comment
@cjihrig

cjihrig Dec 9, 2016

Contributor

The only CI failure was the linter. Does make lint pass for you?

Contributor

cjihrig commented Dec 9, 2016

The only CI failure was the linter. Does make lint pass for you?

evantorrie added some commits Dec 5, 2016

Eliminate capture of "cb" in createSocket context
Move the onFree/onClose/onRemote listeners to a separate function
http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.
@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 9, 2016

Contributor

@cjihrig Yes, works for me:

[root@98dc4c605317 node]# git log --oneline -5 --decorate
518d33d (HEAD, agent-retainer) http: eliminate capture of ClientRequest in Agent
eb5b0d3 change self to agent
f774e72 change to local function instead of prototype
3967101 Eliminate capture of "cb" in createSocket context
bc335c0 (upstream/master) doc: buffer allocation throws for negative size
[root@98dc4c605317 node]# scl enable devtoolset-4 python27 -- gmake lint
./node tools/eslint/bin/eslint.js --cache --rulesdir=tools/eslint-rules \
	  benchmark lib test tools
Total errors found: 0
[root@98dc4c605317 node]#
Contributor

evantorrie commented Dec 9, 2016

@cjihrig Yes, works for me:

[root@98dc4c605317 node]# git log --oneline -5 --decorate
518d33d (HEAD, agent-retainer) http: eliminate capture of ClientRequest in Agent
eb5b0d3 change self to agent
f774e72 change to local function instead of prototype
3967101 Eliminate capture of "cb" in createSocket context
bc335c0 (upstream/master) doc: buffer allocation throws for negative size
[root@98dc4c605317 node]# scl enable devtoolset-4 python27 -- gmake lint
./node tools/eslint/bin/eslint.js --cache --rulesdir=tools/eslint-rules \
	  benchmark lib test tools
Total errors found: 0
[root@98dc4c605317 node]#
@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 13, 2016

Contributor

@cjihrig Is there anything more I need to do here?

Contributor

evantorrie commented Dec 13, 2016

@cjihrig Is there anything more I need to do here?

@bnoordhuis

This comment has been minimized.

Show comment
Hide comment
@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 14, 2016

Contributor

test/arm and test/smartos look like flaky failures unrelated to this change.

Contributor

evantorrie commented Dec 14, 2016

test/arm and test/smartos look like flaky failures unrelated to this change.

@evanlucas

LGTM. Running CI one more time to try and get a green build: https://ci.nodejs.org/job/node-test-pull-request/5418/

Thanks!

@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 19, 2016

Contributor

These CI failures still look unrelated to this change - are the particular failing tests known to be flaky?

Contributor

evantorrie commented Dec 19, 2016

These CI failures still look unrelated to this change - are the particular failing tests known to be flaky?

@evantorrie

This comment has been minimized.

Show comment
Hide comment
@evantorrie

evantorrie Dec 22, 2016

Contributor

@evanlucas CI failure seems to be unrelated to this change.

Contributor

evantorrie commented Dec 22, 2016

@evanlucas CI failure seems to be unrelated to this change.

@jasnell

jasnell approved these changes Jan 6, 2017

jasnell added a commit that referenced this pull request Jan 6, 2017

http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.

PR-URL: #10134
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@jasnell

This comment has been minimized.

Show comment
Hide comment
@jasnell

jasnell Jan 6, 2017

Member

Landed in c04d4df

Member

jasnell commented Jan 6, 2017

Landed in c04d4df

@jasnell jasnell closed this Jan 6, 2017

italoacasas added a commit to italoacasas/node that referenced this pull request Jan 18, 2017

http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.

PR-URL: nodejs#10134
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>

italoacasas added a commit to italoacasas/node that referenced this pull request Jan 19, 2017

http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.

PR-URL: nodejs#10134
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>

italoacasas added a commit to italoacasas/node that referenced this pull request Jan 24, 2017

http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.

PR-URL: nodejs#10134
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>

italoacasas added a commit to italoacasas/node that referenced this pull request Jan 27, 2017

http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.

PR-URL: nodejs#10134
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>

@italoacasas italoacasas referenced this pull request Jan 29, 2017

Merged

v7.5.0 proposal #11062

@MylesBorins

This comment has been minimized.

Show comment
Hide comment
@MylesBorins

MylesBorins Mar 7, 2017

Member

This does not land cleanly in LTS. Added dont-land label. Please feel free to manually backport

Member

MylesBorins commented Mar 7, 2017

This does not land cleanly in LTS. Added dont-land label. Please feel free to manually backport

MylesBorins added a commit that referenced this pull request Sep 22, 2017

http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.

Backport-PR-URL: #15500
PR-URL: #10134
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>

@MylesBorins MylesBorins referenced this pull request Sep 22, 2017

Merged

v6.11.4 proposal #15506

MylesBorins added a commit that referenced this pull request Sep 26, 2017

http: eliminate capture of ClientRequest in Agent
Keepalive sockets that are returned to the agent's freesocket pool were
previously capturing a reference to the ClientRequest that initiated the
request.

This commit eliminates that by moving the installation of the socket
listeners to a different function.

Backport-PR-URL: #15500
PR-URL: #10134
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment