Cache non-symlinks in realpathSync #10253

Closed
wants to merge 1 commit into
from

Projects

None yet

7 participants

@yallop
Contributor
yallop commented Dec 13, 2016
Background

The fs.realpathSync function synchronously traverses paths componentwise to resolve symbolic links.

The optional cache argument stores the results of resolution to avoid repeated lookups. The cache can be used to override resolution and to store the results of calls to readlinkSync.

However, the cache only records paths that lstat reveals to be symlinks. Consequently, a call to fs.realpathSync for a path without symlinks may result in many calls to lstat, even if most parts of the path have been resolved in an earlier call.

Checklist
  • make -j4 test (UNIX), or vcbuild test nosign (Windows) passes
  • commit message follows commit guidelines
Affected core subsystem(s)

fs

Description of change

This tiny patch extends fs.realpathSync to record the results for paths that are not symlinks. The code suggests that this was originally the intention, since the cache is expected to contain entries that resolve to themselves:

cache.get(base) === base

With this patch a run of ember build on a fresh application makes around 6,200 fewer lstat calls (out of a total of around 70,000 syscalls). On a typical native file system the performance difference is negligible, but in situations where a syscall has more significant overhead it can be a worthwhile saving. For example, with a file system that forwards operations over a socket via FUSE this patch saves around 10% of the build time.

@yallop @yallop yallop Cache non-symlinks in realpathSync.
Extend `fs.realpathSync` to cache the results for paths that are not
symlinks in addition to caching symlink mappings.

Signed-off-by: Jeremy Yallop <yallop@docker.com>
d3276fd
@mscdex
Contributor
mscdex commented Dec 13, 2016

What about fs.realPath()?

@addaleax
Member

What about fs.realpath()?

We removed the cache argument in v6. The cache in fs.realpathSync is exlusively used by the module loader now, not part of a public API.

@yallop
Contributor
yallop commented Dec 14, 2016

Thanks for the reviews. I'm not sure why the test/arm CI failed, but let me know if there's anything more I need to do here.

@gibfahn
Member
gibfahn commented Dec 23, 2016 edited
@Trott
Member
Trott commented Dec 23, 2016

OS X failure looks like an unrelated race condition in test-http-client-timeout-with-data to me but let's re-run CI to confirm...

CI: https://ci.nodejs.org/job/node-test-pull-request/5561/

@Trott
Member
Trott commented Dec 23, 2016

Hmmm, now an AIX failure, also seemingly unrelated. Peculiar.

Let's try again:

CI: https://ci.nodejs.org/job/node-test-pull-request/5562/

@Trott
Member
Trott commented Dec 23, 2016

There we go. CI is

@jasnell jasnell added a commit that referenced this pull request Dec 24, 2016
@yallop @jasnell yallop + jasnell fs: cache non-symlinks in realpathSync.
Extend `fs.realpathSync` to cache the results for paths that are not
symlinks in addition to caching symlink mappings.

PR-URL: #10253
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
5dc4487
@jasnell
Member
jasnell commented Dec 24, 2016

Landed in 5dc4487

@jasnell jasnell closed this Dec 24, 2016
@yallop yallop deleted the yallop:cache-non-symlinks branch Dec 26, 2016
@joyeecheung joyeecheung added a commit to joyeecheung/node that referenced this pull request Jan 2, 2017
@yallop @joyeecheung yallop + joyeecheung fs: cache non-symlinks in realpathSync.
Extend `fs.realpathSync` to cache the results for paths that are not
symlinks in addition to caching symlink mappings.

PR-URL: nodejs#10253
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
d913e96
@evanlucas evanlucas added a commit that referenced this pull request Jan 3, 2017
@yallop @evanlucas yallop + evanlucas fs: cache non-symlinks in realpathSync.
Extend `fs.realpathSync` to cache the results for paths that are not
symlinks in addition to caching symlink mappings.

PR-URL: #10253
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
5386e06
@evanlucas evanlucas referenced this pull request Jan 3, 2017
Merged

v7.4.0 release proposal #10589

@evanlucas evanlucas added a commit that referenced this pull request Jan 3, 2017
@yallop @evanlucas yallop + evanlucas fs: cache non-symlinks in realpathSync.
Extend `fs.realpathSync` to cache the results for paths that are not
symlinks in addition to caching symlink mappings.

PR-URL: #10253
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
094a960
@evanlucas evanlucas added a commit that referenced this pull request Jan 4, 2017
@yallop @evanlucas yallop + evanlucas fs: cache non-symlinks in realpathSync.
Extend `fs.realpathSync` to cache the results for paths that are not
symlinks in addition to caching symlink mappings.

PR-URL: #10253
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
6629f8f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment