Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

node:fs error path performance improvements #49962

Closed
wants to merge 8 commits into from
Closed

node:fs error path performance improvements #49962

wants to merge 8 commits into from

Conversation

anonrig
Copy link
Member

@anonrig anonrig commented Sep 29, 2023 •
edited

Closes #49863
Closes #49750
Closes #49748

Local benchmarks:

  • readlinkSync, linkSync, symlinkSync
fs/bench-linkSync.js n=1000 type='invalid'               ***     33.31 %       ±1.38% ±1.84%  ±2.40%
fs/bench-linkSync.js n=1000 type='valid'                         -1.06 %       ±3.29% ±4.40%  ±5.77%
fs/bench-readlinkSync.js n=1000 type='invalid'           ***     25.55 %       ±0.96% ±1.28%  ±1.66%
fs/bench-readlinkSync.js n=1000 type='valid'                      4.01 %       ±6.24% ±8.32% ±10.87%
fs/bench-symlinkSync.js n=1000 type='invalid'            ***     31.08 %       ±1.08% ±1.45%  ±1.90%
fs/bench-symlinkSync.js n=1000 type='valid'                       1.37 %       ±3.34% ±4.45%  ±5.79%
  • renameSync
fs/bench-renameSync.js n=2000 type='invalid'        ***     44.88 %       ±4.54% ±6.06% ±7.93%
fs/bench-renameSync.js n=2000 type='valid'                  -1.36 %       ±4.57% ±6.08% ±7.92%
  • chownSync & lchownSync
fs/bench-chownSync.js n=10000 method='chownSync' type='existing'              **      3.08 %       ±1.93% ±2.58% ±3.39%
fs/bench-chownSync.js n=10000 method='chownSync' type='non-existing'         ***     90.16 %       ±1.57% ±2.09% ±2.72%
fs/bench-chownSync.js n=10000 method='lchownSync' type='existing'                    -0.03 %       ±1.99% ±2.65% ±3.45%
fs/bench-chownSync.js n=10000 method='lchownSync' type='non-existing'        ***     87.50 %       ±1.75% ±2.33% ±3.06%
  • mkdtempSync
fs/bench-mkdtempSync.js n=1000 type='invalid'        ***     54.66 %       ±1.72% ±2.30% ±3.01%
fs/bench-mkdtempSync.js n=1000 type='valid'                  -1.20 %       ±3.58% ±4.77% ±6.20%

Ref: nodejs/performance#106

cc @nodejs/performance

@anonrig anonrig added performance Issues and PRs related to the performance of Node.js. needs-benchmark-ci PR that need a benchmark CI run. labels Sep 29, 2023
@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/startup

@anonrig anonrig changed the title Fs improvements node:fs error path performance improvements Sep 29, 2023
@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Sep 29, 2023
@anonrig anonrig mentioned this pull request Sep 29, 2023
Closed
@anonrig anonrig added the commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. label Sep 29, 2023
This was referenced Sep 29, 2023
Closed
Closed
Open
CanadaHonk
CanadaHonk approved these changes Sep 29, 2023
View reviewed changes
Copy link
Member

@CanadaHonk CanadaHonk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from initial pass

@joyeecheung
Copy link
Member

This needs a rebase since #49913 landed

@anonrig anonrig marked this pull request as ready for review September 30, 2023 18:56
@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Sep 30, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Sep 30, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54420/

@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54424/

@anonrig
Copy link
Member Author

anonrig commented Sep 30, 2023

Errors seems to be related. I'll look into it.

@github-actions github-actions bot mentioned this pull request Oct 1, 2023
Open
16 tasks
@anonrig anonrig added the request-ci Add this label to start a Jenkins CI on a PR. label Oct 1, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Oct 1, 2023
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/54425/

@anonrig anonrig force-pushed the fs-improvements branch 2 times, most recently from 573b0bf to c437b20 Compare October 1, 2023 02:27
targos pushed a commit that referenced this pull request Oct 23, 2023
@anonrig @targos
fs: improve error performance of linkSync
81f1527 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Oct 23, 2023
@anonrig @targos
fs: improve error performance of mkdtempSync
2759878 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Oct 23, 2023
@anonrig @targos
fs: improve error performance of readlinkSync
bc6f279 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Oct 23, 2023
@anonrig @targos
fs: improve error performance of symlinkSync
dc9ac8d 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Oct 23, 2023
@anonrig @targos
fs: improve error performance of lchownSync
6eeaa02 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Oct 23, 2023
@anonrig @targos
fs: improve error performance of realpathSync
c5ff000 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
@targos targos mentioned this pull request Oct 23, 2023
Merged
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@StefanStojanovic @alexfernandez
test: set test-worker-nearheaplimit-deadlock flaky
56091e2 
The test is only flaky on x86 Windows.

Fixes: nodejs#50220
PR-URL: nodejs#50238
Refs: nodejs#49962
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Michael Dawson <midawson@redhat.com>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of renameSync
246d153 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of chownSync
5d1165e 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of linkSync
62801fd 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of mkdtempSync
4a0c1c2 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of readlinkSync
c7123f4 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of symlinkSync
05f1858 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of lchownSync
f1b7177 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
alexfernandez pushed a commit to alexfernandez/node that referenced this pull request Nov 1, 2023
@anonrig @alexfernandez
fs: improve error performance of realpathSync
cc3ecdf 
PR-URL: nodejs#49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@StefanStojanovic @targos
test: set test-worker-nearheaplimit-deadlock flaky
228c87f 
The test is only flaky on x86 Windows.

Fixes: #50220
PR-URL: #50238
Refs: #49962
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Michael Dawson <midawson@redhat.com>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of renameSync
39f31a3 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of chownSync
ea7902d 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of linkSync
9dba771 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of mkdtempSync
12cda31 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of readlinkSync
af6a061 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of symlinkSync
89e7878 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of lchownSync
4242cb7 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
targos pushed a commit that referenced this pull request Nov 11, 2023
@anonrig @targos
fs: improve error performance of realpathSync
36df275 
PR-URL: #49962
Refs: nodejs/performance#106
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
@targos targos mentioned this pull request Nov 12, 2023
Merged
targos pushed a commit that referenced this pull request Nov 27, 2023
@StefanStojanovic @targos
test: set test-worker-nearheaplimit-deadlock flaky
54f3d87 
The test is only flaky on x86 Windows.

Fixes: #50220
PR-URL: #50238
Refs: #49962
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Michael Dawson <midawson@redhat.com>
@juanarbol juanarbol mentioned this pull request Dec 6, 2023
Merged
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test: set test-worker-nearheaplimit-deadlock flaky
c16f293 
The test is only flaky on x86 Windows.

Fixes: nodejs/node#50220
PR-URL: nodejs/node#50238
Refs: nodejs/node#49962
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Michael Dawson <midawson@redhat.com>
sercher added a commit to sercher/graaljs that referenced this pull request Apr 25, 2024
@sercher
test: set test-worker-nearheaplimit-deadlock flaky
4c28669 
The test is only flaky on x86 Windows.

Fixes: nodejs/node#50220
PR-URL: nodejs/node#50238
Refs: nodejs/node#49962
Reviewed-By: Richard Lau <rlau@redhat.com>
Reviewed-By: Yagiz Nizipli <yagiz@nizipli.com>
Reviewed-By: Vinícius Lourenço Claro Cardoso <contact@viniciusl.com.br>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mcollina mcollina mcollina approved these changes

@Qard Qard Qard approved these changes

@H4ad H4ad H4ad approved these changes

@CanadaHonk CanadaHonk CanadaHonk approved these changes

@joyeecheung joyeecheung Awaiting requested review from joyeecheung

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. lib / src Issues and PRs related to general changes in the lib or src directory. needs-benchmark-ci PR that need a benchmark CI run. needs-ci PRs that need a full CI run. performance Issues and PRs related to the performance of Node.js.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
10 participants
@anonrig @nodejs-github-bot @joyeecheung @tniessen @RafaelGSS @StefanStojanovic @mcollina @Qard @H4ad @CanadaHonk