Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

investigate flaky test-fs-stat-bigint #31727

Closed
Trott opened this issue Feb 11, 2020 · 8 comments
Closed

investigate flaky test-fs-stat-bigint #31727

Trott opened this issue Feb 11, 2020 · 8 comments
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. freebsd Issues and PRs related to the FreeBSD platform. fs Issues and PRs related to the fs subsystem / file system. test Issues and PRs related to the tests.

Comments

@Trott
Copy link
Member

Trott commented Feb 11, 2020

  • Version: 14.0.0-pre
  • Platform: FreeBSD
  • Subsystem: fs, test

What steps will reproduce the bug?

Run test-fs-stat-bigint many times on FreeBSD.

How often does it reproduce? Is there a required condition?

It reproduces probably about 1% of the time.

What is the expected behavior?

The test should always pass.

What do you see instead?

The test sometimes fails with a block count of 1 vs 9:

AssertionError [ERR_ASSERTION]: 1n !== 9n
key=blocks, val=9
    at verifyStats (/usr/home/iojs/build/workspace/node-stress-single-test/nodes/freebsd11-x64/test/parallel/test-fs-stat-bigint.js:84:14)
    at /usr/home/iojs/build/workspace/node-stress-single-test/nodes/freebsd11-x64/test/parallel/test-fs-stat-bigint.js:127:7
    at FSReqCallback.oncomplete (fs.js:175:5) {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: 1n,
  expected: 9n,
  operator: 'strictEqual'
}

Additional information

Seems to only reproduce on FreeBSD in CI, but I'm running a stress test to confirm.

https://ci.nodejs.org/job/node-stress-single-test/48/

@Trott Trott added fs Issues and PRs related to the fs subsystem / file system. test Issues and PRs related to the tests. flaky-test Issues and PRs related to the tests with unstable failures on the CI. labels Feb 11, 2020
@Trott
Copy link
Member Author

Trott commented Feb 11, 2020

It seems like I can't make it fail on test-digitalocean-freebsd11-x64-1 but can easily make it fail on test-digitalocean-freebsd11-x64-2.

@nodejs/build @nodejs/platform-freebsd

@Trott Trott added the freebsd Issues and PRs related to the FreeBSD platform. label Feb 11, 2020
@Trott
Copy link
Member Author

Trott commented Feb 11, 2020

results on test-digitalocean-freebsd11-x64-2 showing 3 failures in 10000 runs for a failure rate of 0.03%. Maybe saying I can "easily" make it fail on that host was an overstatement.

I can't seem to get CI to kick off a stress test with test-digitalocean-freebsd11-x64-1 even though it's idle right now, but take my "it doesn't fail on that host" with a grain of salt. Maybe it's just failing at the same low rate.

Trott added a commit to Trott/io.js that referenced this issue Feb 14, 2020
Refs: nodejs#31727

PR-URL: nodejs#31728
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
codebytere pushed a commit that referenced this issue Feb 17, 2020
Refs: #31727

PR-URL: #31728
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
codebytere pushed a commit that referenced this issue Mar 15, 2020
Refs: #31727

PR-URL: #31728
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
codebytere pushed a commit that referenced this issue Mar 17, 2020
Refs: #31727

PR-URL: #31728
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@GeoffreyBooth
Copy link
Member

#32280 (comment)

@GeoffreyBooth
Copy link
Member

GeoffreyBooth commented Mar 25, 2020

When I run python3 tools/test.py --repeat=100 -J parallel/test-fs-stat-bigint on my machine, I get 66 failures out of 100 attempts. MacOS 10.13.5, local Node built for #32280.

codebytere pushed a commit that referenced this issue Mar 30, 2020
Refs: #31727

PR-URL: #31728
Reviewed-By: Ruben Bridgewater <ruben@bridgewater.de>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Denys Otrishko <shishugi@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@Trott
Copy link
Member Author

Trott commented Jun 26, 2020

Stress test on FreeBSD showing 1 failure in 1000 runs: https://ci.nodejs.org/view/All/job/node-stress-single-test/85/nodes=freebsd11-x64/console

@jasnell jasnell added this to Known flakes in Flakees in CI via automation Jun 26, 2020
@complexsplit
Copy link

Is this issue stale, or still relevant?

@emaste
Copy link

emaste commented Dec 2, 2021

Is it the case that file(s) used in this test are only from getFilename() - i.e., a tmp file named test-file-### that contains the string test? It seems very strange that we could get st_blocks=9.

@Trott
Copy link
Member Author

Trott commented Dec 3, 2021

https://ci.nodejs.org/view/All/job/node-stress-single-test/85/nodes=freebsd11-x64/console

Searching the automatically-generated test failure reports in the issues at https://github.com/nodejs/reliability/issues, I don't see this test failing any time recently. (Those reports were broken for a while but they started working again a two or three months ago.) I think we can close this (and re-open it if we see the problem recurring).

@Trott Trott closed this as completed Dec 3, 2021
Flakees in CI automation moved this from Known flakes to Closed Dec 3, 2021
lpinca added a commit to lpinca/node that referenced this issue Feb 12, 2024
nodejs-github-bot pushed a commit that referenced this issue Feb 22, 2024
Refs: #31727 (comment)
PR-URL: #51735
Reviewed-By: Michael Dawson <midawson@redhat.com>
marco-ippolito pushed a commit that referenced this issue Feb 26, 2024
Refs: #31727 (comment)
PR-URL: #51735
Reviewed-By: Michael Dawson <midawson@redhat.com>
marco-ippolito pushed a commit that referenced this issue Feb 27, 2024
Refs: #31727 (comment)
PR-URL: #51735
Reviewed-By: Michael Dawson <midawson@redhat.com>
richardlau pushed a commit that referenced this issue Mar 25, 2024
Refs: #31727 (comment)
PR-URL: #51735
Reviewed-By: Michael Dawson <midawson@redhat.com>
richardlau pushed a commit that referenced this issue Mar 25, 2024
Refs: #31727 (comment)
PR-URL: #51735
Reviewed-By: Michael Dawson <midawson@redhat.com>
rdw-msft pushed a commit to rdw-msft/node that referenced this issue Mar 26, 2024
Refs: nodejs#31727 (comment)
PR-URL: nodejs#51735
Reviewed-By: Michael Dawson <midawson@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. freebsd Issues and PRs related to the FreeBSD platform. fs Issues and PRs related to the fs subsystem / file system. test Issues and PRs related to the tests.
Projects
Flakees in CI
  
Closed
Development

No branches or pull requests

4 participants