Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swallowing errors #62

Closed
vjpr opened this issue Feb 17, 2018 · 8 comments
Closed

Swallowing errors #62

vjpr opened this issue Feb 17, 2018 · 8 comments
Assignees

Comments

@vjpr
Copy link

vjpr commented Feb 17, 2018

Errors are being swallowed here:

if (this.isEnoentCodeError(err)) {

I had a:

ENOENT: no such file or directory, lstat '/dev/fd/12'

but the err.prev was:

EBADF: bad file descriptor, lstat '/dev/fd/12'

Admittedly, I am using the memfs and unionfs. Perhaps the prev is unique to their custom Error object. But maybe there should be a {throwErrors} option?

Environment

  • OS Version: macOS
  • Node.js Version: ^8

Code sample

const glob = require('fast-glob');
const nodeGlob = require('glob');
const {Volume} = require('../lib/volume');
const {patchFs} = require('fs-monkey');
const {ufs} = require('unionfs');
const fs = require('fs');
var readdir = require('readdir-enhanced');

let vol

function start() {
  vol = new Volume()
  ufs.use(fs).use(vol);
  patchFs(ufs)
}

start()

function reset() {
  vol.reset()
}

vol.writeFileSync('/app/foo.json', 'foo')
glob.sync('/app')
vol.symlinkSync('/app', '/app/foo')
glob.sync('/app') // This is empty array.
nodeGlob.sync('/app/**') // Works
@vjpr
Copy link
Author

vjpr commented Feb 17, 2018

So looks like it can be reproduced (on my machine) with:

const readdirEnhanced = require('@mrmlnc/readdir-enhanced')
readdirEnhanced.readdirSync('/dev/fd')

Throws:

Error: EBADF: bad file descriptor, lstat '/dev/fd/12'
    at Object.fs.lstatSync (fs.js:941:11)
    at exports.lstat (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/fs.js:59:20)
    at Object.safeCall [as safe] (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:24:8)
    at stat (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/stat.js:19:8)
    at DirectoryReader.processItem (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:178:5)
    at array.forEach.item (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/for-each.js:14:5)
    at Array.forEach (<anonymous>)
    at Object.syncForEach [as forEach] (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/for-each.js:13:9)
    at call.safe (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:87:16)
    at onceWrapper (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:45:17)
    at onceWrapper (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:45:17)
    at exports.readdir (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/fs.js:19:5)
    at Object.safeCall [as safe] (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/call.js:24:8)
    at DirectoryReader.readNextDirectory (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:78:10)
    at Readable.DirectoryReader.stream._read (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/directory-reader.js:57:18)
    at Readable.read (_stream_readable.js:442:10)
    at readdirSync (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/sync/index.js:27:21)
    at Function.readdirSyncStat (xxx/node_modules/.registry.npmjs.org/@mrmlnc/readdir-enhanced/2.2.1/node_modules/@mrmlnc/readdir-enhanced/lib/index.js:34:10)
    at ReaderSync.api (xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/providers/reader-sync.js:25:24)
    at ReaderSync.read (xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/providers/reader-sync.js:35:32)
    at Array.map (<anonymous>)
    at getWorks (xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/index.js:18:18)
    at Function.sync (xxx/node_modules/.registry.npmjs.org/fast-glob/2.0.4/node_modules/fast-glob/out/index.js:24:17)
    at Test.fn (xxx/index.test.js:25:18)
    at Test.callFn (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/test.js:305:18)
    at Test.run (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/test.js:318:23)
    at runNext (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at Concurrent.run (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/concurrent.js:41:37)
    at runNext (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at runNext (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:58:44)
    at Sequence.run (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/sequence.js:90:10)
    at Bluebird.try (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/runner.js:224:48)
    at tryCatcher (xxx/node_modules/.registry.npmjs.org/bluebird/3.5.1/node_modules/bluebird/js/release/util.js:16:23)
    at Function.Promise.attempt.Promise.try (xxx/node_modules/.registry.npmjs.org/bluebird/3.5.1/node_modules/bluebird/js/release/method.js:39:29)
    at Runner.run (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/runner.js:224:22)
    at process.on.options (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/main.js:84:10)
    at emitOne (events.js:116:13)
    at process.emit (events.js:211:7)
    at process.on.message (xxx/node_modules/.registry.npmjs.org/ava/0.23.0/node_modules/ava/lib/process-adapter.js:14:10)
    at emitTwo (events.js:126:13)
    at process.emit (events.js:214:7)
    at emit (internal/child_process.js:772:12)
    at _combinedTickCallback (internal/process/next_tick.js:141:11)
    at process._tickCallback (internal/process/next_tick.js:180:9) errno: -9, code: 'EBADF', syscall: 'lstat', path: '/dev/fd/12' } }
$ ls -al /dev/fd

dr-xr-xr-x  1 root     wheel         0 15 Feb 15:26 .
dr-xr-xr-x  3 root     wheel      7075 15 Feb 15:26 ..
crw--w----  1 Me       tty     16,  30 17 Feb 11:49 0
crw--w----  1 Me       tty     16,  30 17 Feb 11:49 1
crw--w----  1 Me       tty     16,  30 17 Feb 11:49 2
dr--r--r--  1 root     wheel         0 15 Feb 15:26 4

This doesn't involve any memfs/unionfs stuff from above.

@vjpr
Copy link
Author

vjpr commented Feb 17, 2018

Using zsh autocompletion:

~ ls /dev/fd/
0%   1%   10%  11%  12   2%   3

Using ls:

 ~ ls /dev/fd
0 1 2 4

Maybe the additional fd links are creating inside the running process...

https://unix.stackexchange.com/questions/74454/somethings-special-about-dev-fd-3

@mrmlnc mrmlnc self-assigned this Feb 17, 2018
@mrmlnc
Copy link
Owner

mrmlnc commented Feb 17, 2018

Hello, @vjpr,

Thanks for reporting! I'll take a look at it very soon.

@mrmlnc
Copy link
Owner

mrmlnc commented Feb 18, 2018

So, I see that your issue has a two problems (EBADF and code sample). Sorry, but I don't fully understand your problems. Please use GitHub Issue template for clarifying your problem.

EBADF

What is happening?

Why readdirSync works?

Because fs.readdirSync just reads the catalog and doesn't try to read the items inside it.

Why readdir-enhanced works?

Because it works like fs.readdirSync without { deep: true } option. Inside fast-glob we use deep option.

Why glob works?

Because glob first tries to understand what he's dealing with. Related issue that will help to fix this problem:

Why fast-glob doesn't it work?

Inside fast-glob we use deep: true option and try to look inside the directories. When we try to get lstat or stat it throws error.

Questions

  1. What's the expected behavior here?

Code sample

fs.writeFileSync('/app/foo.json', 'foo')
// Results
glob.sync('/app') // ['/app']
fg.sync('/app') // [] Because we don't read entries inside directory (will be fixed by #47)

fs.writeFileSync('/app/foo.json', 'foo')
fs.symlinkSync('/app', '/app/foo')
// Results
glob.sync('/app') // ['/app']
fg.sync('/app') // [] Because we don't read entries inside directory (will be fixed by #47)

glob.sync('./app/**'); // [ 'app', 'app/foo', 'app/foo.json' ]
fg.sync('./app/**', { onlyFiles: false }); // [ 'app/foo', 'app/foo.json' ]

Questions

  1. What's the expected behavior here?

@vjpr
Copy link
Author

vjpr commented Feb 18, 2018

What about:

console.log(glob.sync('/app/foo/*')) // ? ['/app/foo/foo.json']
console.log(fg.sync('/app/foo/*')) // ? []

They should return the same.

@mrmlnc
Copy link
Owner

mrmlnc commented Feb 18, 2018

Original FS:

const fs = require('fs');

const glob = require('glob');
const fg = require('fast-glob');
const re = require('readdir-enhanced');

fs.mkdirpSync('./app');
fs.writeFileSync('./app/foo.json', 'foo')
fs.symlinkSync('./app', './app/foo')

console.log('fs', fs.readdirSync('./app'));
console.log('glob', glob.sync('./app/*'));
console.log('re', re.sync('./app'));
console.log('fg', fg.sync('./app/*'));

// fs [ 'foo', 'foo.json' ]
// glob [ './app/foo', './app/foo.json' ]
// re [ 'foo', 'foo.json' ]
// fg [ './app/foo.json' ]

VFS:

const fs = require('fs');

const { Volume } = require('memfs');
const { patchFs } = require('fs-monkey');
const { ufs } = require('unionfs');

const glob = require('glob');
const fg = require('fast-glob');
const re = require('readdir-enhanced');

function start() {
    vol = new Volume()
    ufs.use(fs).use(vol);
    patchFs(ufs)
}

start()

vol.mkdirpSync('./app');
vol.writeFileSync('./app/foo.json', 'foo')
vol.symlinkSync('./app', './app/foo')

console.log('fs', vol.readdirSync('./app'));
console.log('glob', glob.sync('./app/*'));
console.log('re', re.sync('./app'));
console.log('fg', fg.sync('./app/*'));

// fs [ 'foo', 'foo.json' ]
// glob [ './app/foo', './app/foo.json' ]
// re [ 'foo', 'foo.json' ]

// Maximum call stack size exceeded
// From `node_modules/unionfs/lib/union.js`

@vjpr
Copy link
Author

vjpr commented Feb 18, 2018

@streamich Seems like memfs gets stuck in an infinite loop when reading a dir containing a symlink to itself. Original fs as seen above handles this case with no problem.

E.g.

/app
/app/foo -> /app

@mrmlnc
Copy link
Owner

mrmlnc commented Feb 24, 2018

I'm closing this issue.

The problem with deep reading will be solved in #60.

Feel free to create separate issues for your problems using the GitHub Issue template (env & actual & expected).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants