Eager ignore

[mixonic]

Paths may be ignored because they contain a large number of files. The getDirs method should not actually descend into those ignored directories. Specifically in Ember.js our tmp/ and node_modules/ directories may both be extremely large and have complex symlinking that trolls the validatePaths method.

Additionally this commit flattens the recursion in getDirs.

[mixonic]

Could use some guidance about what may be annoying travis. The error message is broken (yui/yuitest#29) unfortunately. Test pass locally (or the relevant ones do, there are a number of intermittent errors that were present on master).

[okuryu]

My guess that yui/yuitest#29 doesn't resolves failing build of this PR, because it seems to show different errors with patching yui/yuitest#29.
https://gist.github.com/okuryu/05aee6bfa5ff1b07b424

[mixonic]

@okuryu these failures seem very unrelated though?

[mixonic]

@okuryu looking at yuitest failures again, but they do not seem related here. I sense a disturbance in the npm versioning of a dependency.

This commit has a massive, huge impact on Ember and other codebases. Would appreciate any aid in getting it though, despite the slow progress of these codebases.

[okuryu]

@mixonic Sorry for my delay! Please let me try to investigate this ASAP.

[dschmidt]

Any news? This change reduces our doc build times from about a minute to ~2secs, so I would really like to see this upstreamed.

[stefanpenner]

Test pass locally (or the relevant ones do, there are a number of intermittent errors that were present on master).

@mixonic A rebased version of this PR, consistently demonstrates this issue.

[stefanpenner]

@mixonic not quite sure yet why their is a failure, but it does appear your test case passes without your provided implementation. Which suggests the test isn't quite correct.

[mixonic]

It seems setUp fails silently and doesn't stop the a suite from running

Some tests blindly run on the whole input dir, and since the test I added relies on a fixture that causes a failure those suites attempting to use all of input/ fail.

[stefanpenner]

@mixonic you likely should create a unit test, targeting Y.getDirs directly.

something like:

        var dirs = Y.getDirs(__dirname + '/input/with-symlink', {
          ignorePaths: [
            'a/b',
            'a/d',
            'c'
          ]
        });

            // paths: 'a',
            // ignorePaths: [
            //   'a/b',
            //   'a/d',
            //   'c'
            // ]
        Assert.isArray(dirs);
        console.log(dirs);
        Assert.areSame(1, dirs.length, 'Failed to retrieve all path options');
        Assert.areSame(dirs[0], 'expected Path');

[stefanpenner]

@mixonic once this is green (and reasonable) i'll make sure it gets merged.

[mixonic]

This test fails without the changes to lib/utils.js.

[mixonic]

@stefanpenner The changes here are intended to more efficiently build a list of directories by avoiding traversal of ignored paths. There should be no changed behavior.

However I've added a test for getDirs that shows it now looks for an array of ignored paths, and have confirmed that fails as expected (and the tests confirming old behavior pass) on master.

[mixonic]

These tests were naively presuming all fixtures were valid. I've narrowed the list to ones required for passing tests.

[okuryu]

Why did you need to do this? I think npm test should works after run npm install.

[mixonic]

The test command presumes that instanbul is installed as a global executable:

https://github.com/yui/yuidoc/blob/master/package.json#L90

[stefanpenner]

npm run * prefixes ./node_modules/.bin/ on the path, which should include istanbul as it is devDep https://github.com/yui/yuidoc/blob/master/package.json#L83

[mixonic]

okie dokie, I'll attempt to remove it.

[mixonic]

donezo.

[okuryu]

Merged. I'll publish a new version to npm in this weekend.

[okuryu]

Published v0.10.0.