Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: try to reduce flakes on Windows #28035

Merged
merged 4 commits into from Jun 6, 2019

Conversation

@refack
Copy link
Member

commented Jun 2, 2019

  • test: always suffix tmpdir
    This makes temp dir names consistent whether we run in stand-alone mode,
    via test.py in single process, or in multi-process.
  • test: shell out to rmdir first on Windows
    cmd's rmdir is hardened to deal with Windows edge cases, like
    lingering processes, indexing, and AV checks. So we give it a try first.
  • test: regression test tmpdir
Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@refack refack requested a review from Trott Jun 2, 2019

@refack refack self-assigned this Jun 2, 2019

@nodejs-github-bot

This comment was marked as outdated.

@refack

This comment has been minimized.

Copy link
Member Author

commented Jun 2, 2019

@mscdex

This comment has been minimized.

Copy link
Contributor

commented Jun 2, 2019

Shouldn't the subsystem prefix here just be test: ?

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 297daf1 to 9c57dea Jun 2, 2019

@richardlau

This comment has been minimized.

Copy link
Member

commented Jun 2, 2019

Please could you add a brief sentence or two in the commit message as to why?

@refack

This comment has been minimized.

Copy link
Member Author

commented Jun 2, 2019

Shouldn't the subsystem prefix here just be test: ?

Fixed.

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 9c57dea to b557192 Jun 2, 2019

@refack

This comment has been minimized.

Copy link
Member Author

commented Jun 2, 2019

Please could you add a brief sentence or two in the commit message as to why?

Wrote some words. Didn't have anything too enlightening 🤷‍♂.
WRT to refresh() nothing is extremely robust, it's just a extra heuristic to overcome some recent failures seen in CI:

@nodejs nodejs deleted a comment from nodejs-github-bot Jun 2, 2019

@nodejs-github-bot

This comment was marked as outdated.

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from c869ded to ccf9100 Jun 2, 2019

@nodejs-github-bot

This comment was marked as outdated.

@mscdex mscdex changed the title test,lib: try `rmdir` first on windows test: try `rmdir` first on windows Jun 3, 2019

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch 2 times, most recently from 55fd037 to a17d5ba Jun 3, 2019

@nodejs nodejs deleted a comment from nodejs-github-bot Jun 3, 2019

@nodejs-github-bot

This comment was marked as outdated.

@refack refack added the test label Jun 3, 2019

@Trott
Trott approved these changes Jun 3, 2019
Copy link
Member

left a comment

LGTM if CI is green and once info about opts is added to the tmpdir.refresh() entry in test/common/README.md.

@Trott

This comment has been minimized.

Copy link
Member

commented Jun 3, 2019

@BridgeAR
Copy link
Member

left a comment

LGTM but it seems like on Windows it will now always try both ways, even though the folder might already be deleted by rmdir.

// On Windows first try to delegate rmdir to a shell.
if (process.platform === 'win32' && st.isDirectory()) {
try {
execSync(`rmdir /q /s ${pathname}`, { timout: 1000 });

This comment has been minimized.

Copy link
@BridgeAR

BridgeAR Jun 4, 2019

Member

If everything passes, should this not return?

This comment has been minimized.

Copy link
@refack

refack Jun 6, 2019

Author Member

Not totally sure why, but it's not trivial https://ci.nodejs.org/job/node-test-binary-windows-2/1275/

@sam-github

This comment has been minimized.

Copy link
Member

commented Jun 5, 2019

Is this intended to fix failures like below?


c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:27
      throw e;
      ^

Error: ENOTEMPTY: directory not empty, rmdir 'c:\workspace\node-test-binary-windows-2\test\.tmp.6'
    at Object.rmdirSync (fs.js:693:3)
    at rmdirSync (c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:48:10)
    at rimrafSync (c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:18:7)
    at Object.refresh (c:\workspace\node-test-binary-windows-2\test\common\tmpdir.js:66:3)
    at Object.<anonymous> (c:\workspace\node-test-binary-windows-2\test\parallel\test-child-process-spawnsync-args.js:18:8)
    at Module._compile (internal/modules/cjs/loader.js:777:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:788:10)
    at Module.load (internal/modules/cjs/loader.js:640:32)
    at Function.Module._load (internal/modules/cjs/loader.js:555:12)
    at Function.Module.runMain (internal/modules/cjs/loader.js:840:10) {
  errno: -4051,
  syscall: 'rmdir',
  code: 'ENOTEMPTY',
  path: 'c:\\workspace\\node-test-binary-windows-2\\test\\.tmp.6'
}

https://ci.nodejs.org/job/node-test-binary-windows-2/1244/COMPILED_BY=vs2017,RUNNER=win2008r2-vs2017,RUN_SUBSET=0/testReport/junit/(root)/test/parallel_test_child_process_spawnsync_args/

@refack

This comment has been minimized.

Copy link
Member Author

commented Jun 5, 2019

Is this intended to fix failures like below?

Exactly.

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 91c51b2 to bf5b8e8 Jun 5, 2019

@refack

This comment has been minimized.

Copy link
Member Author

commented Jun 5, 2019

@Trott added doc

@BridgeAR I wanted this to be best-effort with minimal change to current semantics.
had to rebase and fix some more stuff, so I added a return.

I'm going to land this to reduce flakes in CI, and I'll follow up with teaks.

@nodejs-github-bot

This comment was marked as outdated.

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from bf5b8e8 to 4059eba Jun 5, 2019

@nodejs nodejs deleted a comment from nodejs-github-bot Jun 5, 2019

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 4059eba to e531a68 Jun 5, 2019

@nodejs-github-bot

This comment was marked as outdated.

@refack refack changed the title test: try `rmdir` first on windows test: try to reduce flakes on Windows Jun 5, 2019

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from e531a68 to 803a946 Jun 5, 2019

@nodejs-github-bot

This comment was marked as outdated.

test/common/README.md Outdated Show resolved Hide resolved
@Trott
Trott approved these changes Jun 6, 2019
Copy link
Member

left a comment

LGTM with minor doc update to the function signature

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 7fce3e9 to 60133a1 Jun 6, 2019

@nodejs-github-bot

This comment has been minimized.

refack added 4 commits Jun 2, 2019
test: shell out to `rmdir` first on Windows
cmd's `rmdir` is hardened to deal with Windows edge cases, like
lingering processes, indexing, and AV checks. So we give it a try first.

* Added `opts = { spawn = true }` to opt-out of spawning
* test-pipeconnectwrap.js - spawning messes up async_hooks state

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
test: always suffix `tmpdir`
This makes temp dir names consistent whether we run in stand-alone mode,
via `test.py` in single process, or in multi-process.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
test: regression test `tmpdir`
PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
test: mark test-worker-debug as flaky
Also try to make more traceable.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>

@refack refack force-pushed the refack:testutils-rmdir-first-on-windows branch from 60133a1 to 65a5f7b Jun 6, 2019

@refack refack merged commit 65a5f7b into nodejs:master Jun 6, 2019

7 checks passed

Travis CI - Branch Build Passed
Details
Travis CI - Pull Request Build Passed
Details
node-test-commit flaky tests failed
Details
node-test-commit-freebsd tests passed
Details
node-test-commit-linux tests passed
Details
node-test-commit-plinux flaky tests failed
Details
node-test-pull-request flaky tests failed
Details
@refack

This comment has been minimized.

Copy link
Member Author

commented Jun 6, 2019

@refack refack deleted the refack:testutils-rmdir-first-on-windows branch Jun 6, 2019

@refack refack removed their assignment Jun 7, 2019

@refack refack referenced this pull request Jun 10, 2019
BridgeAR added a commit that referenced this pull request Jun 17, 2019
test: shell out to `rmdir` first on Windows
cmd's `rmdir` is hardened to deal with Windows edge cases, like
lingering processes, indexing, and AV checks. So we give it a try first.

* Added `opts = { spawn = true }` to opt-out of spawning
* test-pipeconnectwrap.js - spawning messes up async_hooks state

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
BridgeAR added a commit that referenced this pull request Jun 17, 2019
test: always suffix `tmpdir`
This makes temp dir names consistent whether we run in stand-alone mode,
via `test.py` in single process, or in multi-process.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
BridgeAR added a commit that referenced this pull request Jun 17, 2019
test: regression test `tmpdir`
PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
BridgeAR added a commit that referenced this pull request Jun 17, 2019
test: mark test-worker-debug as flaky
Also try to make more traceable.

PR-URL: #28035
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
@BridgeAR BridgeAR referenced this pull request Jun 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
9 participants
You can’t perform that action at this time.