Skip to content

thiagodp/fwalker

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

12 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

Build Status version

fwalker

Fast and rock-solid asynchronous traversing of directories and files for NodeJS

πŸ‘‰ This library started as a fork of node-filewalker, created by Oliver Leics which is not maintained anymore.

Main differences from node-filewalker:

  • New option readStream
  • New option fs
  • Dependency updates (security fixes)

This library is designed to provide maximum reliance paired with maximum throughput/performance. Please feel free to contribute!

Installation

npm install --save fwalker

Usage

Simple directory listing and disk-usage report:

var fwalker = require('fwalker');

fwalker('.')
  .on('dir', function(p) {
    console.log('dir:  %s', p);
  })
  .on('file', function(p, s) {
    console.log('file: %s, %d bytes', p, s.size);
  })
  .on('error', function(err) {
    console.error(err);
  })
  .on('done', function() {
    console.log('%d dirs, %d files, %d bytes', this.dirs, this.files, this.bytes);
  })
.walk();

Calculate md5-hash for every file:

var started = Date.now();

var createHash = require('crypto').createHash,
    fwalker = require('fwalker');

var options = {
  maxPending: 10, // throttle handles
};

fwalker('/', options)
  .on('stream', function(rs, p, s, fullPath) {
    var hash = createHash('md5');
    rs.on('data', function(data) {
      hash.update(data);
    });
    rs.on('end', function(data) {
      console.log(hash.digest('hex'), ('                '+s.size).slice(-16), p);
    });
  })
  .on('error', function(err) {
    console.error(err);
  })
  .on('done', function() {
    var duration = Date.now()-started;
    console.log('%d ms', duration);
    console.log('%d dirs, %d files, %d bytes', this.dirs, this.files, this.bytes);
  })
.walk();

FWalker pseudo-class

Inherits from node-fqueue.

Options

maxPending

  • Default: -1
  • Maximum asynchronous jobs. Useful to throttle the number of simultaneous disk-operations.

maxAttempts

  • Default: 3
  • Maximum reattempts on error.
  • Set to 0 to disable reattempts.
  • Set to -1 for infinite reattempts.

attemptTimeout

  • Default: 5000 ms
  • Minimum time to wait before reattempt, in milliseconds.
  • Useful to let network-drives remount, etc.

matchRegExp

  • Default: null
  • A RegExp-instance the path to a file must match in order to emit a "file" event.
  • Set to null to emit all paths.

recursive

  • Default: true
  • Traverse in a recursive manner.
  • In case you wish to target only the current directory, disable this.

readStream

  • Default: fs.createReadStream()'s default values
  • New in fwalker (it does not exist in node-filewalker)
  • Allow to handle streams better. For example, the following configuration allows to read a text file from a range of bytes:
var options = {
  // will read from byte 90 to 99
  readStream: {
    start: 90,
    end: 99
  },
  // from simple.txt
  matchRegExp: /simple\.txt/
};

fwalker( '.', options )
  ...

fs

  • Default: fs from NodeJS
  • New in fwalker (it does not exist in node-filewalker)
  • Allow to use a filesystem library different from fs, such as memfs.
  • Allow to use in-memory filesystems.
  • Very useful for testing purposes.

Properties

maxPending
maxAttempts
attemptTimeout
matchRegExp

pending
dirs
files
total
bytes
errors
attempts
streamed
open
detectedMaxOpen

Methods

walk()
pause()
resume()

Events

  • file
    • relative path
    • fs.Stats instance
    • absolute path
  • dir
    • relative path
    • fs.Stats instance
    • absolute path
  • stream
    • fs.ReadStream instance
    • relative path
    • fs.Stats instance
    • absolute path
  • pause
  • resume
  • done
  • error
    • instance of Error

Notice: There will be no fs.ReadStream created if no listener listens to the 'stream'-event.

License

MIT (c) Thiago Delgado Pinto and Oliver Leics

About

πŸ“‚ Fast and rock-solid asynchronous traversing of directories and files for NodeJS

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published