Skip to content

Commit

Permalink
upgrade braces and picomatch
Browse files Browse the repository at this point in the history
- update changelog
- examples
- benchmarks and tests
  • Loading branch information
jonschlinkert committed Apr 10, 2019
1 parent a6596da commit 976d956
Show file tree
Hide file tree
Showing 7 changed files with 135 additions and 64 deletions.
84 changes: 80 additions & 4 deletions .verb.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ You can mix and match these features to create whatever patterns you need!

## Switching to micromatch

There is one notable difference between micromatch and minimatch in regards to how backslashes are handled. See [the notes about backslashes](#backslashes) for more information.
_(There is one notable difference between micromatch and minimatch in regards to how backslashes are handled. See [the notes about backslashes](#backslashes) for more information.)_

### From minimatch

Expand All @@ -78,6 +78,7 @@ console.log(micromatch(['foo', 'bar', 'baz'], ['f*', '*z'])); //=> ['foo', 'baz'
```

## API

{%= apidocs("index.js") %}

## Options
Expand Down Expand Up @@ -460,20 +461,95 @@ However, it's suprising how many edge cases and rabbit holes there are with glob

There is an important, notable difference between minimatch and micromatch _in regards to how backslashes are handled_ in glob patterns.

- Micromatch exclusively and explicitly reserves backslashes for escaping characters in a glob pattern, even on windows. This is consistent with bash behavior.
- Micromatch exclusively and explicitly reserves backslashes for escaping characters in a glob pattern, even on windows, which is consistent with bash behavior. _More importantly, unescaping globs can result in unsafe regular expressions_.
- Minimatch converts all backslashes to forward slashes, which means you can't use backslashes to escape any characters in your glob patterns.

We made this decision for micromatch for a couple of reasons:

- consistency with bash conventions.
- glob patterns are not filepaths. They are a type of [regular language][regular-language] that is converted to a JavaScript regular expression. Thus, when forward slashes are defined in a glob pattern, the resulting regular expression will match windows or POSIX path separators just fine.
- Consistency with bash conventions.
- Glob patterns are not filepaths. They are a type of [regular language][regular-language] that is converted to a JavaScript regular expression. Thus, when forward slashes are defined in a glob pattern, the resulting regular expression will match windows or POSIX path separators just fine.

**A note about joining paths to globs**

Note that when you pass something like `path.join('foo', '*')` to micromatch, you are creating a filepath and expecting it to still work as a glob pattern. This causes problems on windows, since the `path.sep` is `\\`.

In other words, since `\\` is reserved as an escape character in globs, on windows `path.join('foo', '*')` would result in `foo\\*`, which tells micromatch to match `*` as a literal character. This is the same behavior as bash.

To solve this, you might be inspired to do something like `'foo\\*'.replace(/\\/g, '/')`, but this causes another, potentially much more serious, problem.


## Benchmarks

### Running benchmarks

Install dependencies for running benchmarks:

```sh
$ cd bench && npm install
```

Run the benchmarks:

```sh
$ npm run bench
```


### Latest results

As of {%= date() %} (longer bars are better):

```sh
# .makeRe star
micromatch x 1,738,334 ops/sec ±1.08% (93 runs sampled))
minimatch x 677,791 ops/sec ±1.97% (92 runs sampled)

# .makeRe star; dot=true
micromatch x 1,279,934 ops/sec ±1.34% (91 runs sampled)
minimatch x 571,622 ops/sec ±0.41% (88 runs sampled)

# .makeRe globstar
micromatch x 1,349,874 ops/sec ±0.43% (92 runs sampled)
minimatch x 1,109,129 ops/sec ±1.90% (92 runs sampled))

# .makeRe globstars
micromatch x 1,361,452 ops/sec ±0.48% (91 runs sampled)
minimatch x 548,130 ops/sec ±1.59% (91 runs sampled)

# .makeRe with leading star
micromatch x 1,197,552 ops/sec ±0.47% (95 runs sampled)
minimatch x 517,443 ops/sec ±0.56% (95 runs sampled)

# .makeRe - braces
micromatch x 121,082 ops/sec ±2.14% (81 runs sampled))
minimatch x 116,445 ops/sec ±1.53% (95 runs sampled)

# .makeRe braces - range (expanded)
micromatch x 26,359 ops/sec ±0.53% (94 runs sampled)
minimatch x 4,535 ops/sec ±0.57% (90 runs sampled)

# .makeRe braces - range (compiled)
micromatch x 97,946 ops/sec ±2.12% (84 runs sampled))
minimatch x 1,006 ops/sec ±0.29% (93 runs sampled)

# .makeRe braces - nested ranges (expanded)
micromatch x 18,553 ops/sec ±0.96% (90 runs sampled)
minimatch x 4,435 ops/sec ±1.01% (93 runs sampled)

# .makeRe braces - nested ranges (compiled)
micromatch x 31,550 ops/sec ±2.40% (81 runs sampled))
minimatch x 987 ops/sec ±0.30% (94 runs sampled)d)

# .makeRe braces - set (compiled)
micromatch x 86,669 ops/sec ±1.82% (83 runs sampled))
minimatch x 43,106 ops/sec ±0.78% (94 runs sampled)

# .makeRe braces - nested sets (compiled)
micromatch x 56,419 ops/sec ±2.00% (79 runs sampled)
minimatch x 27,150 ops/sec ±0.79% (92 runs sampled)
```


## Contributing

All contributions are welcome! Please read [the contributing guide](.github/contributing.md) to get started.
Expand Down
48 changes: 19 additions & 29 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,23 +32,26 @@ Changelog entries are classified using the following labels _(from [keep-a-chang

</details>

### [4.0.0] - 2019-03-20
## [4.0.0] - 2019-03-20

**Removed**
### Added

- Removed support for passing an array of brace patterns to `micromatch.braces()`.
- To strictly enforce closing brackets (for `{`, `[`, and `(`), you must now use `strictBrackets=true` instead of `strictErrors`.
- Adds support for `options.onMatch`. See the readme for details
- Adds support for `options.onIgnore`. See the readme for details
- Adds support for `options.onResult`. See the readme for details

**Removed Options**

Support for the following options was removed in v4.0:
### Breaking changes

- `cache` - caching has been removed
- `nodupes` - duplicates are always removed by default. You can override this with customize behavior by using the `onMatch`, `onResult` and `onIgnore` functions.
- `snapdragon` - snapdragon is no longer used
- `sourcemap` - sourcemaps are no longer supports
- Removed support for passing an array of brace patterns to `micromatch.braces()`.
- To strictly enforce closing brackets (for `{`, `[`, and `(`), you must now use `strictBrackets=true` instead of `strictErrors`.
- `cache` - caching and all related options and methods have been removed
- `options.unixify` was renamed to `options.windows`
- `options.nodupes` Was removed. Duplicates are always removed by default. You can override this with custom behavior by using the `onMatch`, `onResult` and `onIgnore` functions.
- `options.snapdragon` was removed, as snapdragon is no longer used.
- `options.sourcemap` was removed, as snapdragon is no longer used, which provided sourcemap support.

### [3.0.0] - 2017-04-11
## [3.0.0] - 2017-04-11

Complete overhaul, with 36,000+ new unit tests validated against actual output generated by Bash and minimatch. More specifically, 35,000+ of the tests:

Expand All @@ -69,21 +72,7 @@ Here are those sub-modules with links to related prs on those modules if you wan

**Added**

- source map support (optionally created when using parse or compile - I have no idea what the use case is yet, but they come for free) (note that source maps are not generated for brace expansion at present, since the braces compiler uses a different strategy. I'll update if/when this changes). Example:
```js
var mm = require('micromatch');
var pattern = '*(*(of*(a)x)z)';

var ast = mm.parse(pattern, {sourcemap: true});
var res = mm.compile(ast);
console.log(res);
// { map:
// { version: 3,
// sources: [ 'string' ],
// names: [],
// mappings: 'AAAA,CAAE,CAAE,EAAE,CAAE,CAAC,EAAC,CAAC,EAAC,CAAC,EAAC',
// sourcesContent: [ '*(*(of*(a)x)z)' ] }},
```
- source map support (optionally created when using parse or compile - I have no idea what the use case is yet, but they come for free) (note that source maps are not generated for brace expansion at present, since the braces compiler uses a different strategy. I'll update if/when this changes).
- parser is exposed, so that implementors can customize or override specific micromatch parsers if necessary
- compiler is exposed, so that implementors can customize or override specific micromatch compilers if necessary

Expand All @@ -93,21 +82,22 @@ Here are those sub-modules with links to related prs on those modules if you wan
- even safer - micromatch has always generated optimized patterns so it's not subject to DoS exploits like minimatch (completely different than the regex DoS issue, minimatch and multimatch are still openly exposed to being used for DoS attacks), but more safeguards were built into this refactor

**Changed**

- the public API of this library did not change in this version and should be safe to upgrade without changing implentor code. However, we have released this as a major version for the following reasons:
- out of an abundance of caution due to the large amount of code changed in this release
- we have improved parser accuracy to such a degree that some implementors using invalid globs have noted change in behavior. If this is the case for you, please check that you are using a valid glob expression before logging a bug with this library

### [1.0.1] - 2016-12-12
## [1.0.1] - 2016-12-12

**Added**

- Support for windows path edge cases where backslashes are used in brackets or other unusual combinations.

### [1.0.0] - 2016-12-12
## [1.0.0] - 2016-12-12

Stable release.

### [0.1.0] - 2016-10-08
## [0.1.0] - 2016-10-08

First release.

Expand Down
9 changes: 6 additions & 3 deletions bench/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,17 @@
"version": "0.0.0",
"private": true,
"main": "index.js",
"scripts": {
"bench": "node ."
},
"dependencies": {
"ansi-colors": "^3.0.3",
"ansi-colors": "^3.2.4",
"benchmark": "^2.1.4",
"minimatch": "^3.0.4",
"minimist": "^1.2.0"
},
"devDependencies": {
"glob-parent": "^3.1.0",
"minimatch": "^3.0.4"
"glob-parent": "^5.0.0"
},
"lintDeps": {
"devDependencies": {
Expand Down
9 changes: 9 additions & 0 deletions examples/micromatch.braces.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
'use strict';

const { braces } = require('..');

console.log(braces('foo/{a,b,c}/bar'));
//=> [ 'foo/(a|b|c)/bar' ]

console.log(braces('foo/{a,b,c}/bar', { expand: true }));
//=> [ 'foo/a/bar', 'foo/b/bar', 'foo/c/bar' ]
36 changes: 12 additions & 24 deletions index.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
'use strict';

const path = require('path');
const util = require('util');
const braces = require('braces');
const picomatch = require('picomatch');
const utils = require('picomatch/lib/utils');
const isEmptyString = val => typeof val === 'string' && (val === '' || val === './');
Expand Down Expand Up @@ -372,7 +372,7 @@ micromatch.capture = (glob, input, options) => {
* @api public
*/

micromatch.makeRe = picomatch.makeRe;
micromatch.makeRe = (...args) => picomatch.makeRe(...args);

/**
* Scan a glob pattern to separate the pattern into segments. Used
Expand All @@ -388,19 +388,7 @@ micromatch.makeRe = picomatch.makeRe;
* @api public
*/

micromatch.scan = (input, options = {}) => picomatch.scan(input, options);

/**
* Split a glob pattern into two parts: the directory part of the glob,
* and the matching part.
*
* @param {String} `pattern`
* @param {Object} `options`
* @return {Array}
* @api public
*/

micromatch.split = (input, options = {}) => picomatch.split(input, options);
micromatch.scan = (...args) => picomatch.scan(...args);

/**
* Parse a glob pattern to create the source string for a regular
Expand All @@ -419,25 +407,25 @@ micromatch.split = (input, options = {}) => picomatch.split(input, options);
micromatch.parse = (patterns, options) => {
let res = [];
for (let pattern of [].concat(patterns || [])) {
for (let str of micromatch.braces(pattern, options)) {
for (let str of braces(pattern, options)) {
res.push(picomatch.parse(str, options));
}
}
return res;
};

/**
* Expand the given brace `pattern`.
* Process the given brace `pattern`.
*
* ```js
* const mm = require('micromatch');
* console.log(mm.braces('foo/{a,b}/bar'));
* //=> ['foo/(a|b)/bar']
* const { braces } = require('micromatch');
* console.log(braces('foo/{a,b,c}/bar'));
* //=> [ 'foo/(a|b|c)/bar' ]
*
* console.log(mm.braces('foo/{a,b}/bar', { expand: true }));
* //=> ['foo/a/bar', 'foo/b/bar']
* console.log(braces('foo/{a,b,c}/bar', { expand: true }));
* //=> [ 'foo/a/bar', 'foo/b/bar', 'foo/c/bar' ]
* ```
* @param {String} `pattern` String with brace pattern to expand.
* @param {String} `pattern` String with brace pattern to process.
* @param {Object} `options` Any [options](#options) to change how expansion is performed. See the [braces][] library for all available options.
* @return {Array}
* @api public
Expand All @@ -448,7 +436,7 @@ micromatch.braces = (pattern, options) => {
if ((options && options.nobrace === true) || !/\{.*\}/.test(pattern)) {
return [pattern];
}
return require('braces')(pattern, options);
return braces(pattern, options);
};

/**
Expand Down
9 changes: 5 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "micromatch",
"description": "Glob matching for javascript/node.js. A drop-in replacement and faster alternative to minimatch and multimatch.",
"description": "Glob matching for javascript/node.js. A replacement and faster alternative to minimatch and multimatch.",
"version": "3.1.10",
"homepage": "https://github.com/micromatch/micromatch",
"author": "Jon Schlinkert (https://github.com/jonschlinkert)",
Expand Down Expand Up @@ -29,16 +29,17 @@
],
"main": "index.js",
"engines": {
"node": ">=6"
"node": ">=8"
},
"scripts": {
"test": "mocha"
},
"dependencies": {
"braces": "micromatch/braces#dev",
"picomatch": "micromatch/picomatch#dev"
"braces": "^3.0.0",
"picomatch": "^2.0.0"
},
"devDependencies": {
"fill-range": "^7.0.1",
"gulp-format-md": "^2.0.0",
"minimatch": "^3.0.4",
"mocha": "^5.2.0",
Expand Down
4 changes: 4 additions & 0 deletions test/api.parse.js
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ describe('.parse()', () => {
let results = mm.parse('a/*');
let { tokens } = results[0];

tokens.forEach(token => {
delete token.prev;
});

assert.deepEqual(tokens, [
{ type: 'bos', value: '', output: '' },
{ type: 'text', value: 'a' },
Expand Down

0 comments on commit 976d956

Please sign in to comment.