Walk

Directory and file walking utility for node apps

Features:

• Easy to use
• Simple, lightweight, and fast
• Async, sync, streams, and promise api
• Simple filtering with glob pattern matching
• Require or import file contents, or read as stream, buffer, or string
• Resolves require-style path strings
• Normalized file objects with helper methods
• Fast pattern matching via micromatch
• Include and exclude pattern matching options
• Only 2 dependencies: micromatch, lodash

About

I needed a better way to walk directories and read files during build and/or run time. I wanted an api that was simple, supported glob pattern matching like gulp, and returned objects with a similar format as vinyl. This package allows you to simply read any directory (or file), filter results with glob pattern matching, and return an array of file objects. It can also require or import file contents, read them as streams, buffers, or strings, and resolve require-style path strings.

Usage

Add walk as a dependency for your app and install via npm

npm install @danmasta/walk --save

Require the package in your app

const walk = require('@danmasta/walk');

By default walk returns a readable stream interface. You can use the methods: map(), tap(), each(), promise(), then(), and catch() to iterate file objects and return promises

Options

name	type	description
cwd	string	Base directory to start walk from. Default is process.cwd
root	string	Directory or file path as root to walk from. This affects the relative paths used in file objects and matching. Default is ./
src	array|string|regexp	Micromatch pattern for result filtering by including any matches. Can be a path string, glob pattern string, regular expression, or an array of strings. Defaults to *(../)*(**/)*
dot	boolean	Whether or not to include dot files when matching. Default is true
ignore	array|string|regexp	Micromatch pattern for result filtering by ignoring any matches. Can be a path string, glob pattern string, regular expression, or an array of strings. Defaults to *(../)*(**/)(.git|node_modules)
encoding	string	Which encoding to use when reading or writing file contents. Default is utf8
resolve	boolean	Whether or not to attempt to resolve file paths if not found. Default is true
paths	array|string	Which paths to walk for files. Default is undefined

Methods

Name	Description
map(fn)	Runs an iterator function over each file. Returns a promise that resolves with a new array of return values
tap(fn)	Runs an iterator function over each file. Returns a promise that resolves with an array of file objects
each(fn)	Runs an iterator function over each file. Returns a promise that resolves with undefined
promise	Returns a promise that resolves with an array of file objects
then(fn)	Run a callback when the promise is fulfilled. Returns a promise that resolves with an array of file objects
catch(fn)	Run a callback when the promise is rejected. Returns a promise that resolves with an array of file objects

File Objects

Each file object returned from walk has the following signature:

Properties

name	type	description
cwd	string	Current working directory. Defaults to process.cwd
root	string	Base directory to use for relative pathing. Defaults to cwd
path	string	Absolute path of the file on disk
relative	string	Relative path of file based from normalized root
relativeFromCwd	string	Relative path of file based from normalized cwd
dir	string	Parent directory where file is located
base	string	File name with extension
name	string	File name without extension
ext	string	File extension
stat	object	The fs.stat object for the file
contents	string|object	Contents of the file. Default is undefined
encoding	string	Default encoding to use when reading or writing file contents. Default is utf8

Methods

name	description
createReadStream(opts)	Returns a readable stream for the file
createWriteStream(opts)	Returns a writable stream for the file
append(data, opts)	Appends data to the file
read(opts)	Reads the file contents. Returns string or buffer based on encoding
readAsString(opts)	Shortcut for read() that ensures encoding is set or throws an error
readAsBuffer(opts)	Shortcut for read() that sets the encoding to null
readStr	Alias for readAsString
readBuf	Alias for readAsBuffer
write(data, opts)	Writes data to the file
require	Reads the file contents using require
import	Reads the file contents using import. Returns a promise
requireOrImport	Reads the file contents using import or require based on esm-ness. Returns a promise
requireImportOrRead	Reads the file contents using import or require if able, otherwise reads as string or buffer based on encoding. Returns a promise
isModule	Returns true if file.contents is a module
isBuffer	Returns true if file.contents is a buffer
isStream	Returns true if file.contents is a stream
isNull	Returns true if file.contents is null
isNil	Returns true if file.contents is null or undefined
isString	Returns true if file.contents is a string
isDirectory	Returns true if the file is a directory
isSymbolicLink	Returns true if the file is a symbolic link
isBlockDevice	Returns true if the file is a block device
isCharacterDevice	Returns true if the file is a character device
isFIFO	Returns true if the file is a first-in-first-out (FIFO) pipe
isFile	Returns true if the file is a file
isSocket	Returns true if the file is a socket
isEmpty	Returns true if the file is empty (zero bytes)
getEncodingFromBom	Returns the encoding from the file byte order mark if set, otherwise undefined

Sync

This package also supports a fully synchronous api. Instead of requiring the default package just require @danmasta/walk/sync. The api is exactly the same for walking and file objects

Examples

const walk = require('@danmasta/walk');

Walk the current working directory and pipe all files to a destination stream

walk('./').pipe(writeStream());

Walk the current working directory, exclude all .json files

walk('./', { src: '**/*.!(json)' }).then(res => {
    console.log('files:', res);
});

Walk a child directory, include only .json files

walk('./config', { src: '**/*.json' }).then(res => {
    console.log('files:', res);
});

Walk a directory using an absolute path

walk('/usr/local').then(res => {
    console.log('files:', res);
});

Read the contents of all pug files in ./views as a string

walk('./views', { src: '**/*.pug' }).map(file => {
    file.readStr().then(res => {
        console.log('template:', file.path, res);
    });
});

Read the contents of all pug files in ./views as a buffer

walk('./views', { src: '**/*.pug' }).map(file => {
    file.readBuf().then(res => {
        console.log('template:', file.path, res);
    });
});

Require all js files in the ./routes directory and run a callback for each one

walk('./routes', { src: '**/*.js' }).each(route => {
    app.use(route.require());
}).then(() => {
    console.log('all routes loaded');
});

Load all templates from the ./views directory synchronously

const walk = require('@danmasta/walk/sync');

let templates = walk('./views', { src: '**/*.pug' });

console.log('templates:', templates);

Testing

Tests are currently run using mocha and chai. To execute tests run npm run test. To generate unit test coverage reports run npm run coverage

Contact

If you have any questions feel free to get in touch