nofs

Overview

nofs extends Node's native fs module with some useful methods. It tries to make your functional programming experience better. It's one of the core lib of nokit.

Features

Introduce map and reduce to folders.
Recursive glob, move, copy, remove, etc.
Promise by default.
Unified intuitive API. Supports both Promise, Sync and Callback paradigms.
Very light weight. Only depends on yaku and minimath.

Install

npm install nofs

API Convention

Path & Pattern

Only functions like readFile which may confuse the user don't support pattern.

Promise & Callback

If you call an async function without callback, it will return a promise. For example the nofs.remove('dir', () => 'done!') are the same with nofs.remove('dir').then(() => 'done!').

eachDir

It is the core function for directory manipulation. Other abstract functions like mapDir, reduceDir, glob are built on top of it. You can play with it if you don't like other functions.

nofs & Node Native fs

Only the callback of nofs.exists is slightly different, it will also gets two arguments (err, exists).

nofs only extends the native module, no pollution will be found. You can still require the native fs, and call fs.exists as easy as pie.

Inheritance of Options

A Function's options may inherit other function's, especially the functions it calls internally. Such as the glob extends the eachDir's option, therefore glob also has a filter option.

Quick Start

// You can replace "require('fs')" with "require('nofs')"
let fs = require('nofs');

/*
 * Callback
 */
fs.outputFile('x.txt', 'test', (err) => {
    console.log('done');
});


/*
 * Sync
 */
fs.readFileSync('x.txt');
fs.copySync('dir/a', 'dir/b');


/*
 * Promise & async/await
 */
(async () => {
    await fs.mkdirs('deep/dir/path');
    await fs.outputFile('a.txt', 'hello world');
    await fs.move('dir/path', 'other');
    await fs.copy('one/**/*.js', 'two');

    // Get all files, except js files.
    let list = await fs.glob(['deep/**', '!**/*.js']);
    console.log(list);

    // Remove only js files.
    await fs.remove('deep/**/*.js');
})();


/*
 * Concat all css files.
 */
fs.reduceDir('dir/**/*.css', {
    init: '/* Concated by nofs */\n',
    iter (sum, { path }) {
        return fs.readFile(path).then(str =>
            sum += str + '\n'
        );
    }
}).then(concated =>
    console.log(concated)
);



/*
 * Play with the low level api.
 * Filter all the ignored files with high performance.
 */
let patterns = fs.readFileSync('.gitignore', 'utf8').split('\n');

let filter = ({ path }) => {
    for (let p of patterns) {
        // This is only a demo, not full git syntax.
        if (path.indexOf(p) === 0)
            return false;
    }
    return true;
}

fs.eachDir('.', {
    searchFilter: filter, // Ensure subdirectory won't be searched.
    filter: filter,
    iter: (info) => info  // Directly return the file info object.
}).then((tree) =>
    // Instead a list as usual,
    // here we get a file tree for further usage.
    console.log(tree)
);

Changelog

Goto changelog

Function Name Alias

For some naming convention reasons, nofs also uses some common alias for fucntion names. See src/alias.js.

FAQ

Error: EMFILE?

This is due to system's default file descriptor number settings for one process. Latest node will increase the value automatically. See the issue list of node.

API

No native fs funtion will be listed.

Promise
copyDir(src, dest, opts)
copyFile(src, dest, opts)
copy(from, to, opts)
dirExists(path)
eachDir(spath, opts)
ensureFile(path, opts)
fileExists(path)
glob(pattern, opts)
mapDir(from, to, opts)
mkdirs(path, mode)
move(from, to, opts)
outputFile(path, data, opts)
outputJson(path, obj, opts)
path
pmatch
Promise
PromiseUtils
readJson(path, opts)
reduceDir(path, opts)
remove(path, opts)
touch(path, opts)
watchPath(path, opts)
watchFiles(patterns, opts)
watchDir(root, opts)
writeFile(path, data, opts)
Promise

Here I use Yaku only as an ES6 shim for Promise. No APIs other than ES6 spec will be used. In the future it will be removed.
copyDir(src, dest, opts)

Copy an empty directory.
- param: src { String }
- param: dest { String }
- param: opts { Object }
```
{
    isForce: false,
    mode: auto
}
```
- return: { Promise }
copyFile(src, dest, opts)

Copy a single file.
- param: src { String }
- param: dest { String }
- param: opts { Object }
```
{
    isForce: false,
    mode: auto
}
```
- return: { Promise }

copy(from, to, opts)

Like cp -r.

param: from { String }

Source path.
param: to { String }

Destination path.

param: opts { Object }

Extends the options of eachDir. Defaults:

{
    // Overwrite file if exists.
    isForce: false,
    isIterFileOnly: false

    filter: (fileInfo) => true
}

return: { Promise }

example:

Copy the contents of the directory rather than copy the directory itself.

nofs.copy('dir/path/**', 'dest/path');

nofs.copy('dir/path', 'dest/path', {
    filter: (fileInfo) => {
        return /\d+/.test(fileInfo.path);
    }
});

dirExists(path)

Check if a path exists, and if it is a directory.
- param: path { String }
- return: { Promise }
  
  Resolves a boolean value.

eachDir(spath, opts)

Concurrently walks through a path recursively with a callback. The callback can return a Promise to continue the sequence. The resolving order is also recursive, a directory path resolves after all its children are resolved.

param: spath { String }

The path may point to a directory or a file.

param: opts { Object }

Optional. Defaults:

{
    // Callback on each path iteration.
    iter: (fileInfo) => Promise | Any,

    // Auto check if the spath is a minimatch pattern.
    isAutoPmatch: true,

    // Include entries whose names begin with a dot (.), the posix hidden files.
    all: true,

    // To filter paths. It can also be a RegExp or a glob pattern string.
    // When it's a string, it extends the Minimatch's options.
    filter: (fileInfo) => true,

    // The current working directory to search.
    cwd: '',

    // Call iter only when it is a file.
    isIterFileOnly: false,

    // Whether to include the root directory or not.
    isIncludeRoot: true,

    // Whehter to follow symbol links or not.
    isFollowLink: true,

    // Iterate children first, then parent folder.
    isReverse: false,

    // When isReverse is false, it will be the previous iter resolve value.
    val: any,

    // If it return false, sub-entries won't be searched.
    // When the `filter` option returns false, its children will
    // still be itered. But when `searchFilter` returns false, children
    // won't be itered by the iter.
    searchFilter: (fileInfo) => true,

    // If you want sort the names of each level, you can hack here.
    // Such as `(names) => names.sort()`.
    handleNames: (names) => names
}

The argument of opts.iter, fileInfo object has these properties:

{
    path: String,
    name: String,
    baseDir: String,
    isDir: Boolean,
    children: [fileInfo],
    stats: fs.Stats,
    val: Any
}

Assume we call the function: nofs.eachDir('dir', { iter: (f) => f }), the resolved directory object array may look like:

{
    path: 'some/dir/path',
    name: 'path',
    baseDir: 'some/dir',
    isDir: true,
    val: 'test',
    children: [
        {
            path: 'some/dir/path/a.txt', name: 'a.txt',
            baseDir: 'dir', isDir: false, stats: { ... }
        },
        { path: 'some/dir/path/b.txt', name: 'b.txt', ... }
    ],
    stats: {
        size: 527,
        atime: 'Mon, 10 Oct 2011 23:24:11 GMT',
        mtime: 'Mon, 10 Oct 2011 23:24:11 GMT',
        ctime: 'Mon, 10 Oct 2011 23:24:11 GMT'
        ...
    }
}

The stats is a native fs.Stats object.

return: { Promise }

Resolves a directory tree object.

example:

// Print all file and directory names, and the modification time.
nofs.eachDir('dir/path', {
    iter: (obj, stats) =>
        console.log(obj.path, stats.mtime)
});

// Print path name list.
nofs.eachDir('dir/path', { iter: (curr) => curr })
.then((tree) =>
    console.log(tree)
);

// Find all js files.
nofs.eachDir('dir/path', {
    filter: '**/*.js',
    iter: ({ path }) =>
        console.log(paths)
});

// Find all js files.
nofs.eachDir('dir/path', {
    filter: /\.js$/,
    iter: ({ path }) =>
        console.log(paths)
});

// Custom filter.
nofs.eachDir('dir/path', {
    filter: ({ path, stats }) =>
        path.slice(-1) != '/' && stats.size > 1000
    iter: (path) =>
        console.log(path)
});

ensureFile(path, opts)

Ensures that the file exists. Change file access and modification times. If the file does not exist, it is created. If the file exists, it is NOT MODIFIED.
- param: path { String }
- param: opts { Object }
- return: { Promise }
fileExists(path)

Check if a path exists, and if it is a file.
- param: path { String }
- return: { Promise }
  
  Resolves a boolean value.

glob(pattern, opts)

Get files by patterns.

param: pattern { String | Array }

The minimatch pattern. Patterns that starts with '!' in the array will be used to exclude paths.

param: opts { Object }

Extends the options of eachDir. The filter property is fixed with the pattern, use iter instead. Defaults:

{
    all: false,

    // The minimatch option object.
    pmatch: {},

    // It will be called after each match. It can also return
    // a promise.
    iter: (fileInfo, list) => list.push(fileInfo.path)
}

return: { Promise }

Resolves the list array.

example:

// Get all js files.
nofs.glob(['**/*.js', '**/*.css']).then((paths) =>
    console.log(paths)
);

// Exclude some files. "a.js" will be ignored.
nofs.glob(['**/*.js', '!**/a.js']).then((paths) =>
    console.log(paths)
);

// Custom the iterator. Append '/' to each directory path.
nofs.glob('**/*.js', {
    iter: (info, list) =>
        list.push(info.isDir ? (info.path + '/') : info.path
}).then((paths) =>
    console.log(paths)
);

mapDir(from, to, opts)

Map file from a directory to another recursively with a callback.

param: from { String }

The root directory to start with.
param: to { String }

This directory can be a non-exists path.

param: opts { Object }

Extends the options of eachDir. But cwd is fixed with the same as the from parameter. Defaults:

{
    // It will be called with each path. The callback can return
    // a `Promise` to keep the async sequence go on.
    iter: (src, dest, fileInfo) => Promise | Any,

    // When isMapContent is true, and the current is a file.
    iter: (content, src, dest, fileInfo) => Promise | Any,

    // When isMapContent is true, and the current is a folder.
    iter: (mode, src, dest, fileInfo) => Promise | Any,

    isMapContent: false,

    isIterFileOnly: true
}

return: { Promise }

Resolves a tree object.

example:

nofs.mapDir('from', 'to', {
    iter: (src, dest, info) =>
        console.log(src, dest, info)
});

example:

// Copy and add license header for each files
// from a folder to another.
nofs.mapDir('from', 'to', {
    ismMapContent: true,
    iter: (content) =>
        'License MIT\n' + content
});

mkdirs(path, mode)

Recursively create directory path, like mkdir -p.
- param: path { String }
- param: mode { String }
  
  Defaults: 0o777 & ~process.umask()
- return: { Promise }
move(from, to, opts)

Moves a file or directory. Also works between partitions. Behaves like the Unix mv.
- param: from { String }
  
  Source path.
- param: to { String }
  
  Destination path.
- param: opts { Object }
  
  Defaults:
```
{
    isForce: false,
    isFollowLink: false
}
```
- return: { Promise }
  
  It will resolve a boolean value which indicates whether this action is taken between two partitions.
outputFile(path, data, opts)

Almost the same as writeFile, except that if its parent directories do not exist, they will be created.
- param: path { String }
- param: data { String | Buffer }
- param: opts { String | Object }
  
  Same with the writeFile.
- return: { Promise }
outputJson(path, obj, opts)

Write a object to a file, if its parent directory doesn't exists, it will be created.
- param: path { String }
- param: obj { Any }
  
  The data object to save.
- param: opts { Object | String }
  
  Extends the options of outputFile. Defaults:
```
{
    replacer: null,
    space: null
}
```
- return: { Promise }
path

The path module nofs is using. It's the native io.js path lib. nofs will force all the path separators to /, such as C:\a\b will be transformed to C:/a/b.
- type: { Object }
pmatch

The minimatch lib. It has two extra methods:
- isPmatch(String | Object) -> Pmatch | undefined It helps to detect if a string or an object is a minimatch.
- getPlainPath(Pmatch) -> String Helps to get the plain root path of a pattern. Such as src/js/*.js will get src/js
Documentation

Offline Documentation
- example:
```
nofs.pmatch('a/b/c.js', '**/*.js');
// output => true
nofs.pmatch.isPmatch('test*');
// output => true
nofs.pmatch.isPmatch('test/b');
// output => false
```
Promise

What promise this lib is using.
- type: { Promise }
PromiseUtils

Same as the yaku/lib/utils.
- type: { Object }
readJson(path, opts)

Read A Json file and parse it to a object.
- param: path { String }
- param: opts { Object | String }
  
  Same with the native nofs.readFile.
- return: { Promise }
  
  Resolves a parsed object.
- example:
```
nofs.readJson('a.json').then((obj) =>
    console.log(obj.name, obj.age)
);
```

reduceDir(path, opts)

Walk through directory recursively with a iterator.

param: path { String }

param: opts { Object }

Extends the options of eachDir, with some extra options:

{
    iter: (prev, path, isDir, stats) -> Promise | Any,

    // The init value of the walk.
    init: undefined,

    isIterFileOnly: true
}

return: { Promise }

Final resolved value.

example:

// Concat all files.
nofs.reduceDir('dir/path', {
    init: '',
    iter: (val, { path }) =>
        nofs.readFile(path).then((str) =>
            val += str + '\n'
        )
}).then((ret) =>
    console.log(ret)
);

remove(path, opts)

Remove a file or directory peacefully, same with the rm -rf.
- param: path { String }
- param: opts { Object }
  
  Extends the options of eachDir. But the isReverse is fixed with true. Defaults:
```
{ isFollowLink: false }
```
- return: { Promise }
touch(path, opts)

Change file access and modification times. If the file does not exist, it is created.
- param: path { String }
- param: opts { Object }
  
  Default:
```
{
    atime: Date.now(),
    mtime: Date.now(),
    mode: undefined
}
```
- return: { Promise }
  
  If new file created, resolves true.

watchPath(path, opts)

Watch a file. If the file changes, the handler will be invoked. You can change the polling interval by using process.env.pollingWatch. Use process.env.watchPersistent = 'off' to disable the persistent. Why not use nofs.watch? Because nofs.watch is unstable on some file systems, such as Samba or OSX.

param: path { String }

The file path

param: opts { Object }

Defaults:

{
    handler: (path, curr, prev, isDeletion) => {},

    // Auto unwatch the file while file deletion.
    autoUnwatch: true,

    persistent: process.env.watchPersistent != 'off',
    interval: +process.env.pollingWatch || 300
}

return: { Promise }

It resolves the StatWatcher object:
```
{
    path,
    handler
}
```

example:

process.env.watchPersistent = 'off'
nofs.watchPath('a.js', {
    handler: (path, curr, prev, isDeletion) => {
        if (curr.mtime !== prev.mtime)
            console.log(path);
    }
}).then((watcher) =>
    nofs.unwatchFile(watcher.path, watcher.handler)
);

watchFiles(patterns, opts)

Watch files, when file changes, the handler will be invoked. It is build on the top of nofs.watchPath.
- param: patterns { Array }
  
  String array with minimatch syntax. Such as ['*/**.css', 'lib/**/*.js'].
- param: opts { Object }
  
  Same as the nofs.watchPath.
- return: { Promise }
  
  It contains the wrapped watch listeners.
- example:
```
nofs.watchFiles('*.js', handler: (path, curr, prev, isDeletion) =>
    console.log (path)
);
```

watchDir(root, opts)

Watch directory and all the files in it. It supports three types of change: create, modify, move, delete. By default, move event is disabled. It is build on the top of nofs.watchPath.

param: root { String }

param: opts { Object }

Defaults:

{
    handler: (type, path, oldPath, stats, oldStats) => {},

    patterns: '**', // minimatch, string or array

    // Whether to watch POSIX hidden file.
    all: false,

    // The minimatch options.
    pmatch: {},

    isEnableMoveEvent: false
}

return: { Promise }

Resolves a object that keys are paths, values are listeners.

example:

// Only current folder, and only watch js and css file.
nofs.watchDir('lib', {
 pattern: '*.+(js|css)',
 handler: (type, path, oldPath, stats, oldStats) =>
     console.log(type, path, stats.isDirectory(), oldStats.isDirectory())
});

writeFile(path, data, opts)

A writeFile shim for < Node v0.10.
- param: path { String }
- param: data { String | Buffer }
- param: opts { String | Object }
- return: { Promise }

Benckmark

See the benchmark folder.

Node v0.10, Intel Core i7 2.3GHz SSD, find 91,852 js files in 191,585 files:

node-glob: 9939ms
nofs-glob: 8787ms

Nofs is slightly faster.

Lisence

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 414 Commits
benchmark		benchmark
doc		doc
src		src
test		test
.gitignore		.gitignore
.travis.yml		.travis.yml
appveyor.yml		appveyor.yml
nofile.js		nofile.js
package-lock.json		package-lock.json
package.json		package.json
readme.md		readme.md

ysmood/nofs

Folders and files

Latest commit

History

Repository files navigation

Overview

Features

Install

API Convention

Path & Pattern

Promise & Callback

nofs & Node Native fs

Inheritance of Options

Quick Start

Changelog

Function Name Alias

FAQ

API

Benckmark

Lisence

About

Resources

Stars

Watchers

Forks

Languages