Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v16.x backport] fs: use signed types for stat data #44129

Conversation

LiviaMedeiros
Copy link
Contributor

@LiviaMedeiros LiviaMedeiros commented Aug 4, 2022

Backport of: #43714

On arm-12+, fsPromises.stat timestamps are returned as 0 even for small positive values. Excluding this platform from test.

LiviaMedeiros added a commit to LiviaMedeiros/node that referenced this pull request Aug 4, 2022
This allows to support timestamps before 1970-01-01.
On Windows, it's not supported due to Y2038 issue.

PR-URL: nodejs#43714
Fixes: nodejs#43707
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Backport-PR-URL: nodejs#44129
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. v16.x labels Aug 4, 2022
@LiviaMedeiros LiviaMedeiros added wip Issues and PRs that are still a work in progress. request-ci Add this label to start a Jenkins CI on a PR. labels Aug 4, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 4, 2022
@nodejs-github-bot

This comment was marked as outdated.

LiviaMedeiros and others added 2 commits August 4, 2022 19:40
This allows to support timestamps before 1970-01-01.
On Windows, it's not supported due to Y2038 issue.

PR-URL: nodejs#43714
Fixes: nodejs#43707
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Backport-PR-URL: nodejs#44129
@LiviaMedeiros LiviaMedeiros added request-ci Add this label to start a Jenkins CI on a PR. and removed wip Issues and PRs that are still a work in progress. labels Aug 4, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 4, 2022
@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot

This comment was marked as outdated.

@nodejs-github-bot
Copy link
Collaborator

@aduh95
Copy link
Contributor

aduh95 commented Aug 7, 2022

On arm-12+, fsPromises.stat timestamps are returned as 0 even for small positive values. Excluding this platform from test.

Doesn't that mean that 9e40873...5311b7d should be fast-tracked on main first?

@richardlau
Copy link
Member

On arm-12+, fsPromises.stat timestamps are returned as 0 even for small positive values. Excluding this platform from test.

Doesn't that mean that 9e40873...5311b7d should be fast-tracked on main first?

FWIW as background arm-12+ is referring to https://ci.nodejs.org/job/node-test-binary-arm-12+/ which is the job that runs tests for armv7l on Raspberry Pi devices. They currently aren't being run on main/v18.x as they need updates which is why the failures haven't been seen until the tests were ported here to v16 (where we're still running tests on the Pi's).

Also FWIW I suspect the issue isn't actually Linux on arm but the filesystem being used.

@LiviaMedeiros
Copy link
Contributor Author

Doesn't that mean that 9e40873...5311b7d should be fast-tracked on main first?

Yes, if similar test environment will be used on newer versions as well.

FWIW as background arm-12+ is referring to https://ci.nodejs.org/job/node-test-binary-arm-12+/ which is the job that runs tests for armv7l on Raspberry Pi devices. They currently aren't being run on main/v18.x as they need updates which is why the failures haven't been seen until the tests were ported here to v16 (where we're still running tests on the Pi's).

Also FWIW I suspect the issue isn't actually Linux on arm but the filesystem being used.

Thanks for clarification! Which FS is used on that test platform? Is there a straightforward way to detect it?
Othewise we can add something like

await fsPromises.utimes(filepath, 1000, 1000);
const { atimeMs, mtimeMs } = await fsPromises.stat(filepath);
if (atimeMs !== 1000 || mtimeMs !== 1000) common.skip('bad platform or fs');

But it still feels weird to literally skip a test if it fails. 😅

@aduh95
Copy link
Contributor

aduh95 commented Aug 8, 2022

Doesn't that mean that 9e40873...5311b7d should be fast-tracked on main first?

Yes, if similar test environment will be used on newer versions as well.

I disagree, it's not because this project doesn't have a CI for this environment that it's not worth landing it on main. If someone with this env wants to compile node and run the test themself, there's no reason we should impose them a false positive test failure.

@richardlau
Copy link
Member

FWIW as background arm-12+ is referring to https://ci.nodejs.org/job/node-test-binary-arm-12+/ which is the job that runs tests for armv7l on Raspberry Pi devices. They currently aren't being run on main/v18.x as they need updates which is why the failures haven't been seen until the tests were ported here to v16 (where we're still running tests on the Pi's).
Also FWIW I suspect the issue isn't actually Linux on arm but the filesystem being used.

Thanks for clarification! Which FS is used on that test platform? Is there a straightforward way to detect it?

They are NFS mounts, e.g.

$ df -Th /home/iojs/build/workspace/node-test-binary-arm/
Filesystem                                                                       Type  Size  Used Avail Use% Mounted on
192.168.1.10:/exports/nodejs/pi/test-requireio_williamkapke-debian10-arm64_pi3-3 nfs   458G  206G  230G  48% /
$

On the Pi's the tests are run in containers and folder above is mounted inside the container on the machine.

@LiviaMedeiros LiviaMedeiros added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 8, 2022
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 8, 2022
@nodejs-github-bot
Copy link
Collaborator

@targos
Copy link
Member

targos commented Aug 8, 2022

If I understand correctly, after #44174 lands, we can close this PR and remove the backport label from #43714 ?

@LiviaMedeiros
Copy link
Contributor Author

I disagree, it's not because this project doesn't have a CI for this environment that it's not worth landing it on main. If someone with this env wants to compile node and run the test themself, there's no reason we should impose them a false positive test failure.

Perhaps I worded it incorrectly: test env was related to fast-tracking, i.e. if it should be ported to main ASAP.

As for local user environments, I think it would be reasonable to keep the test as strict as possible, as long as it doesn't break common cases (example: absence of ipv6 support is common, blocked ports for localhost are not), and doesn't unconditionally break CI.

If there is a better approach for such tests, I'm open for suggestions.

They are NFS mounts

I guess NFS might not support atime? On both first and second tries it returned 0, so it doesn't look like fsync-related issue.

If I understand correctly, after #44174 lands, we can close this PR and remove the backport label from #43714 ?

Yes. If #43714 with #44174 will be mergeable into v16 without any problems, the label can be removed and this PR can be closed.

@LiviaMedeiros
Copy link
Contributor Author

Superseded by #44174

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants