Remove `common.PORT` usage from `parallel` tests #12376

Trott · 2017-04-12T22:11:03Z

Version: master
Platform: all
Subsystem: test

Tests in parallel that use common.PORT risk getting EADDRINUSE if another test in parallel uses port 0 (to get an open port assigned by the operating system) at the same time the test runs. This appears to have happened recently. (See #12363 (comment).)

IMO, all instances of common.PORT in parallel should either be changed to use port 0 (if possible in the context of the test) or else moved to sequential (if using port 0 is not possible).

Here are the current tests that use common.PORT:

test/parallel/test-cluster-basic.js (test: remove common.PORT from test-cluster-basic #12377)
test/parallel/test-cluster-bind-twice.js (test: use dynamic port in test-cluster-bind-twice #12418)
test/parallel/test-cluster-dgram-1.js (test: dynamic port in cluster worker dgram #12487)
test/parallel/test-cluster-dgram-2.js (test: dynamic port in cluster worker dgram #12487) (test: replace common.PORT in test-cluster-dgram-2 #12492)
test/parallel/test-cluster-dgram-reuse.js (test: replace port in cluster dgram reuse test #12901)
test/parallel/test-cluster-disconnect-leak.js (Migrate cluster tests from common.PORT #12441)
test/parallel/test-cluster-disconnect-race.js (Migrate cluster tests from common.PORT #12441)
test/parallel/test-cluster-disconnect.js
test/parallel/test-cluster-eaddrinuse.js
test/parallel/test-cluster-inspector-debug-port.js
test/parallel/test-cluster-ipc-throw.js
test/parallel/test-cluster-master-error.js (tests: remove common.PORT from shared,master,rr tests #12451)
test/parallel/test-cluster-master-kill.js (tests: remove common.PORT from shared,master,rr tests #12451)
test/parallel/test-cluster-message.js
test/parallel/test-cluster-net-send.js (tests: remove common.PORT from shared,master,rr tests #12451)
test/parallel/test-cluster-process-disconnect.js (Migrate cluster tests from common.PORT #12441)
test/parallel/test-cluster-rr-domain-listen.js (tests: remove common.PORT from shared,master,rr tests #12451)
test/parallel/test-cluster-rr-ref.js (tests: remove common.PORT from shared,master,rr tests #12451)
test/parallel/test-cluster-send-deadlock.js (tests: remove common.PORT from cluster send tests #12472)
test/parallel/test-cluster-send-handle-twice.js (tests: remove common.PORT from cluster send tests #12472)
test/parallel/test-cluster-server-restart-none.js
test/parallel/test-cluster-server-restart-rr.js
test/parallel/test-cluster-shared-handle-bind-error.js
test/parallel/test-cluster-shared-leak.js (tests: remove common.PORT from shared,master,rr tests #12451)
test/parallel/test-cluster-worker-disconnect-on-error.js (tests: remove common.PORT from cluster-worker-disconnect-on-error #12457)
test/parallel/test-cluster-worker-disconnect.js (tests: remove common.PORT from cluster worker disconnect, exit and kill #12443)
test/parallel/test-cluster-worker-exit.js (tests: remove common.PORT from cluster worker disconnect, exit and kill #12443)
test/parallel/test-cluster-worker-kill.js (tests: remove common.PORT from cluster worker disconnect, exit and kill #12443)
test/parallel/test-cluster-worker-no-exit.js (tests: remove common.PORT from shared,master,rr tests #12451)
test/parallel/test-cluster-worker-wait-server-close.js (tests: remove common.PORT from cluster worker wait server close #12466)
test/parallel/test-debugger-repeat-last.js (test: moved test-debugger-repeat-last.js from parallel to sequential #12470)
test/parallel/test-dgram-bind-shared-ports.js (test: use dynamic port in test-dgram-bind-shared-ports #12452)
test/parallel/test-dgram-close-in-listening.js
test/parallel/test-dgram-close-is-not-callback.js
test/parallel/test-dgram-close.js
test/parallel/test-dgram-exclusive-implicit-bind.js
test/parallel/test-dgram-implicit-bind-failure.js
test/parallel/test-dgram-oob-buffer.js
test/parallel/test-dgram-send-address-types.js (test: replace port in dgram send address types test #13007)
test/parallel/test-dgram-send-callback-buffer-empty-address.js (test: replace port in dgram cd address empty test #12929)
test/parallel/test-dgram-send-callback-buffer-length-empty-address.js (test: replace port in dgram cb address test #12944)
test/parallel/test-dgram-send-callback-buffer-length.js (test: replace port in dgram cb length test #12943)
test/parallel/test-dgram-send-callback-buffer.js (test: replace port in dgram cb test #12942)
test/parallel/test-https-connect-address-family.js (test: replace port in https address family test #12915)
test/parallel/test-net-better-error-messages-port-hostname.js (Remove common.PORT usage from parallel tests #12473)
test/parallel/test-net-better-error-messages-port.js
test/parallel/test-net-connect-handle-econnrefused.js
test/parallel/test-net-connect-immediate-destroy.js
test/parallel/test-net-connect-immediate-finish.js
test/parallel/test-net-connect-local-error.js
test/parallel/test-net-listen-shared-ports.js
test/parallel/test-net-localerror.js
test/parallel/test-net-localport.js
test/parallel/test-net-options-lookup.js
test/parallel/test-net-reconnect-error.js
test/parallel/test-net-server-bind.js (test: refactor test-net-server-bind #13273)
test/parallel/test-net-socket-destroy-twice.js(Remove common.PORT usage from parallel tests #12473)
test/parallel/test-regress-GH-5051.js
test/parallel/test-regress-GH-5727.js
test/parallel/test-tls-client-abort.js (being worked on by @ahmed-taj) (tests: remove common.PORT from client abort #12461)
test/parallel/test-tls-client-abort2.js (being worked on by @ahmed-taj) (tests: remove common.PORT from client abort #12461)
test/parallel/test-tls-client-default-ciphers.js
test/parallel/test-tls-connect.js
test/parallel/test-tls-ticket-cluster.js

The text was updated successfully, but these errors were encountered:

richardlau · 2017-04-12T23:40:50Z

IMO, all instances of common.PORT in parallel should either be changed to use port 0 (if possible in the context of the test) or else moved to sequential (if using port 0 is not possible).

Maybe update the description of common.PORT in test/README.md with this recommendation and/or (if possible) introduce a linting rule after the list is resolved?

evanlucas · 2017-04-13T10:45:28Z

For a few of these, (test-net-better-error-messages-*) they don't actually have a server, they try to connect to a server that is not running. Are those ones that you think we should still change @Trott?

orafaelfragoso · 2017-04-14T13:01:12Z

@evanlucas @richardlau I would love to contribute to this one, but I don't feel comfortable doing so. Can I count on you to guide me on my first contributions? Anything I should read on OS port attribution?

thefourtheye · 2017-04-15T08:05:48Z

So we are slowly removing common.PORT?

Remove common.PORT from test-cluster-dgram-reuse to eliminate possibility that a dynamic port used in another test will collide with common.PORT. Refs: nodejs#12376

gibfahn · 2017-04-15T21:00:32Z

I would love to contribute to this one, but I don't feel comfortable doing so. Can I count on you to guide me on my first contributions?

@rafaelfragosom go for it! You should have plenty of people willing to help you if you have any issues (count me in).

orafaelfragoso · 2017-04-15T21:11:18Z

@gibfahn I'm making progress so far. Thank you!

Trott · 2017-04-16T00:28:03Z

So we are slowly removing common.PORT?

@thefourtheye From tests in parallel only. It's fine in sequential. And if we find tests in parallel that cannot be re-written to avoid common.PORT, then moving those tests to sequential is probably the way we'll go.

Once we get common.PORT out of parallel, we can probably alter common.js and maybe test.py to simply hardcode common.PORT to a single value. (In other words, there's special logic in there to make sure that parallel tests have different common.PORT values. That won't be necessary anymore.)

orafaelfragoso · 2017-04-16T13:35:37Z

@Trott @gibfahn Is it really necessary to make 1 PR for each altered file on this issue? Seems exhausting.

gibfahn · 2017-04-16T13:51:25Z

Is it really necessary to make 1 PR for each altered file on this issue?

@rafaelfragosom no not at all. The general rule (see CONTRIBUTING.md) is one commit per logical change. If you're doing basically the same thing to 20 files, that's one commit. You can also have multiple commits per PR. My general rule is one PR per easily reviewed thing.

EDIT: In fact loads of very similar PRs can get pretty exhausting for reviewers as well.

orafaelfragoso · 2017-04-16T14:18:33Z

@gibfahn Awesome!

I'm trying to wrap my head inside the tests, specially from clusters, and what's bothering me right now is how common.PORT is being used, like the issue says.

I wasn't here when these tests were written so I don't know what I'm saying but, isn't it better to write a getAvailablePortSync() helper function to retrieve an available port? Generators would be awesome for that.

What are your thoughts on this?

I'm currently looking at parallel/test-cluster-disconnect.js and I'm faced with this:

// ...
if (cluster.isWorker) {
  net.createServer((socket) => {
    socket.end('echo');
  }).listen(common.PORT, '127.0.0.1');

  net.createServer((socket) => {
    socket.end('echo');
  }).listen(common.PORT + 1, '127.0.0.1');
// ...

And could be probably written like this:

// ...
if (cluster.isWorker) {
  var port = common.getAvailablePortSync();

  net.createServer((socket) => {
    socket.end('echo');
  }).listen(port, '127.0.0.1');

  net.createServer((socket) => {
    socket.end('echo');
  }).listen(port + 1, '127.0.0.1');
// ...

I'm really excited for this next PR, this is the last test on test-cluster-disconnect-* and I can't see another way around it. I tried saving the ports in an array but since it's Async, it's not available when the master is called.

Thank you.

thefourtheye · 2017-04-16T15:30:25Z

@thefourtheye From tests in parallel only. It's fine in sequential.

Just playing devil's advocate. There is a remote possibility that the port is already chosen by the OS.

Trott · 2017-04-16T15:57:32Z

Just playing devil's advocate. There is a remote possibility that the port is already chosen by the OS.

Yes, that's always a possibility. I agree it's best to avoid common.PORT altogether where possible.

santigimeno · 2017-04-16T19:32:48Z

I imagine it was taken into account but we have to be sure that we're not removing coverage for the case when a method is called with a specific port.

gibfahn · 2017-04-16T19:32:49Z

@rafaelfragosom I would suggest doing something like #12418 (using process.send to send the ports back to the parent).

The problem with getting a free port through something like common.getAvailablePortSync() is that the port could be used by something else between calling that function and actually starting the server on the port. You also still have the problem that the child processes have to pass their port numbers to the parent process.

cc/ @Trott as there might be a better solution.

Trott · 2017-04-16T19:34:13Z

I imagine it was taken into account but we have to be sure that we're not removing coverage for the case when a method is called with a specific port.

Yup, absolutely, good to make that explicit for sure.

The usage of common.PORT could cause undesired port collisions when run in parallel. The following test was moved to sequential. test-net-reconnect-error.js PR-URL: #13033 Refs: #12376 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com>

Replace common.PORT with available port assigned by the operating system in test-dgram-send-callback-buffer. PR-URL: #12942 Refs: #12376 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michael Dawson <mhdawson@ibm.com>

Remove common.PORT from test-dgram-send-callback-buffer-length to eliminate possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: #12943 Refs: #12376 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Michael Dawson <mhdawson@ibm.com>

Remove common.PORT from test-cluster-worker-disconnect, test-cluster-worker-exit and test-cluster-worker-kill to eliminate the possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: nodejs/node#12443 Ref: nodejs/node#12376 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

PR-URL: nodejs/node#12441 Ref: nodejs/node#12376 Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

Remove common.PORT from test-cluster-bind-twice to eliminate possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: nodejs/node#12418 Ref: nodejs/node#12376 Reviewed-By: Luigi Pinca <luigipinca@gmail.com> Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com> Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

Remove common.PORT from test-cluster-send-deadlock and test-cluster-send-handle-twice to reduce possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: nodejs/node#12472 Ref: nodejs/node#12376 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

Change common.PORT to '0' to avoid the possibility of getting EADDRINUSE error if another test in 'parallel' uses port '0' at the same time. PR-URL: nodejs/node#12461 Ref: nodejs/node#12376 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

Remove common.PORT from test-cluster-worker-disconnect-on-error possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: nodejs/node#12457 Ref: nodejs/node#12376 Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Yuta Hiroto <hello@about-hiroppy.com>

Remove common.PORT from test-cluster-dgram-1 and test-cluster-dgram-2, in order to eliminate the possibility of port collision. PR-URL: nodejs/node#12487 Ref: nodejs/node#12376 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>

Removed common.PORT from test-cluster-message, test-cluster-server-restart-none, test-cluster-server-restart-rr and test-cluster-shared-handle-bind-error to eliminate the possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: nodejs/node#12584 Ref: nodejs/node#12376 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>

Removed common.PORT from test-cluster-ipc-throw to eliminate the possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: nodejs/node#12571 Ref: nodejs/node#12376 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Removed common.PORT from test-cluster-eaddrinuse to eliminate the possibility that a dynamic port used in another test will collide with common.PORT. PR-URL: nodejs/node#12547 Ref: nodejs/node#12376 Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Richard Lau <riclau@uk.ibm.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Change common.PORT to dynamic port to prevent collision with other parallel test which use common.PORT Refs: [nodejs#12376](nodejs#12376)

maclover7 · 2017-08-18T21:05:58Z

A bunch more of the test files from the big list above have been completed (purposefully not putting # before pull request number to avoid creating tons of notification noise):

test/parallel/test-cluster-disconnect.js, 12545
test/parallel/test-cluster-eaddrinuse.js, 12547
test/parallel/test-cluster-inspector-debug-port.js moved to test/inspector/test-inspector-port-cluster.js
test/parallel/test-cluster-ipc-throw.js, 12571
test-cluster-message.js, 12584
test-cluster-server-restart-none.js, 12584
test-cluster-server-restart-rr.js, 12584
test-cluster-shared-handle-bind-error.js, 12584
test-dgram-close-in-listening.js, 12376
test-dgram-close-is-not-callback.js, 12376
test-dgram-close.js, 12376
test-dgram-exclusive-implicit-bind.js, 12376
test-dgram-oob-buffer.js, 12376
test-net-better-error-messages-port.js, 12473
test-net-connect-handle-econnrefused.js, 12473
test-net-connect-immediate-destroy.js, 12473
test-net-connect-local-error.js, 12473
test-net-listen-shared-ports.js, 12473
test-net-localerror.js, 12473
test-net-options-lookup.js, 12473
test-net-reconnect-error.js, 13033
test/parallel/test-regress-GH-5051.js, 12639
test/parallel/test-regress-GH-5727.js, 12639
test-tls-ticket-cluster.js, 12715

Trott · 2017-08-18T22:22:33Z

Thanks, @maclover7! I've updated the list.

Ref: nodejs#12376

PR-URL: #14928 Ref: #12376 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

PR-URL: nodejs/node#14928 Ref: nodejs/node#12376 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

PR-URL: #14928 Ref: #12376 Reviewed-By: James M Snell <jasnell@gmail.com> Reviewed-By: Rich Trott <rtrott@gmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>

Trott added the test Issues and PRs related to the tests. label Apr 12, 2017

evanlucas added the good first issue Issues that are suitable for first-time contributors. label Apr 13, 2017

Trott mentioned this issue Apr 15, 2017

test: use dynamic port in test-cluster-bind-twice #12418

Closed

2 tasks

benjamingr mentioned this issue Apr 15, 2017

test: use dynamic port in test-cluster-dgram-reuse #12426

Closed

2 tasks

tarunbatra mentioned this issue Apr 16, 2017

Migrate cluster tests from common.PORT #12441

Closed

3 tasks

sebastianplesciuc mentioned this issue Apr 16, 2017

tests: remove common.PORT from cluster worker disconnect, exit and kill #12443

Closed

3 tasks

tarunbatra mentioned this issue Apr 16, 2017

tests: remove common.PORT from shared,master,rr tests #12451

Closed

3 tasks

orafaelfragoso mentioned this issue Apr 16, 2017

test: use dynamic port in test-dgram-bind-shared-ports #12452

Closed

3 tasks

sebastianplesciuc mentioned this issue Apr 17, 2017

tests: remove common.PORT from cluster-worker-disconnect-on-error #12457

Closed

3 tasks

z0al mentioned this issue Apr 17, 2017

tests: remove common.PORT from client abort #12461

Closed

4 tasks

oantoro added a commit to oantoro/node that referenced this issue Aug 8, 2017

test: use dynamic port in test-tls-connect.js

71a8dc2

Change common.PORT to dynamic port to prevent collision with other parallel test which use common.PORT Refs: [nodejs#12376](nodejs#12376)

oantoro mentioned this issue Aug 8, 2017

test: use dynamic port in test-tls-connect.js #14687

Closed

2 tasks

maclover7 mentioned this issue Aug 18, 2017

test: use 443, not common.PORT #14928

Closed

2 tasks

Trott removed the good first issue Issues that are suitable for first-time contributors. label Aug 18, 2017

maclover7 added a commit to maclover7/node that referenced this issue Aug 24, 2017

test: simplify test-tls-client-default-ciphers

54defc0

Ref: nodejs#12376

Trott closed this as completed Jan 31, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove `common.PORT` usage from `parallel` tests #12376

Remove `common.PORT` usage from `parallel` tests #12376

Trott commented Apr 12, 2017 •

edited

Loading

richardlau commented Apr 12, 2017

evanlucas commented Apr 13, 2017

orafaelfragoso commented Apr 14, 2017

thefourtheye commented Apr 15, 2017

gibfahn commented Apr 15, 2017

orafaelfragoso commented Apr 15, 2017

Trott commented Apr 16, 2017

orafaelfragoso commented Apr 16, 2017

gibfahn commented Apr 16, 2017 •

edited

Loading

orafaelfragoso commented Apr 16, 2017

thefourtheye commented Apr 16, 2017

Trott commented Apr 16, 2017

santigimeno commented Apr 16, 2017

gibfahn commented Apr 16, 2017

Trott commented Apr 16, 2017

maclover7 commented Aug 18, 2017

Trott commented Aug 18, 2017

Remove common.PORT usage from parallel tests #12376

Remove common.PORT usage from parallel tests #12376

Comments

Trott commented Apr 12, 2017 • edited Loading

richardlau commented Apr 12, 2017

evanlucas commented Apr 13, 2017

orafaelfragoso commented Apr 14, 2017

thefourtheye commented Apr 15, 2017

gibfahn commented Apr 15, 2017

orafaelfragoso commented Apr 15, 2017

Trott commented Apr 16, 2017

orafaelfragoso commented Apr 16, 2017

gibfahn commented Apr 16, 2017 • edited Loading

orafaelfragoso commented Apr 16, 2017

thefourtheye commented Apr 16, 2017

Trott commented Apr 16, 2017

santigimeno commented Apr 16, 2017

gibfahn commented Apr 16, 2017

Trott commented Apr 16, 2017

maclover7 commented Aug 18, 2017

Trott commented Aug 18, 2017

Remove `common.PORT` usage from `parallel` tests #12376

Remove `common.PORT` usage from `parallel` tests #12376

Trott commented Apr 12, 2017 •

edited

Loading

gibfahn commented Apr 16, 2017 •

edited

Loading