Listing files.

[sholladay]

Code that moves files individually is unfortunate. I would prefer to avoid hardcoding filenames into my generator.

One thing that would help is if I could programmatically get a list of the filenames within the destinationRoot() directory. Then I can have conventions around how files get renamed and do it in a loop based on whatever files were actually copied over. E.g. all filenames with a given prefix have that prefix removed.

In the generator, it would look something like this:

const cruft = '_';
const filenames = this.fs.list(this.destinationRoot());
filenames.forEach((name) => {
    if (name.startsWith(cruft)) {
        this.fs.move(
            this.destinationPath(name),
            this.destinationPath(name.substring(cruft.length))
        );
    }
});

[sholladay]

Here is an example of what I am doing now (prompts, etc. not included). This code decides whether to copy over the _cli.js template. It also decides whether to rename it to cli.js, simply because attempting to rename it if it wasn't copied over would throw an error. The hardcoded list of filesToMove and the second if statement could disappear if I had something like this.fs.list(), as proposed above.

// All template filenames are prefixed with at least one underscore so that they
// are ignored by tools that care about special files. We remove or replace the
// prefix when we are ready to give the file special meaning.
const unprefix = (name) => {
    const hiddenFilePrefix = '__';
    return name.startsWith(hiddenFilePrefix) ?
        name.replace(hiddenFilePrefix, '.') :
        name.substring('_'.length);
};

module.exports = class extends Generator {
    writing() {
        const templates = [this.sourceRoot()];

        if (!this.props.cli) {
            templates.push('!**/_cli.js');
        }

        this.fs.copyTpl(
            templates,
            this.destinationRoot(),
            this.props
        );

        const filesToMove = [
            '__editorconfig',
            '__gitattributes',
            '__gitignore',
            '_CONTRIBUTING.md',
            '_LICENSE',
            '_README.md',
            '_circle.yml',
            '_index.js',
            '_package.json',
            '_test.js'
        ];

        if (this.props.cli) {
            filesToMove.push('_cli.js');
        }

        filesToMove.forEach((name) => {
            this.fs.move(
                this.destinationPath(name),
                this.destinationPath(unprefix(name))
            );
        });
    }
};

[SBoudrias]

Looping through the files is the correct approach and what we should be doing in mem-fs-editor. End user shouldn't have to think about it.

The current implementation is quite naive in which it does only match files and bypass folder levels operations.

[SBoudrias]

Oh also, it is currently possible to loop over the in-memory files https://github.com/SBoudrias/mem-fs#iterating-over-the-file-system

And I just commited a fix to master for copy and delete operations not working on in-memory directories.

I'll be closing this issue as I believe both these things covers/fix the issue and the question you had.

[sholladay]

@SBoudrias I don't think we're on the same page. I'll try to explain my feature request better.

Assume I have a very basic Yeoman generator that has some templates for .editorconfig, package.json, etc.

The problem I am trying to solve is:

1. By no fault of mem-fs-editor, tools like EditorConfig are prone to doing special things with the template files themselves simply because of their filename, which is undesirable in this case because it's a template and not intended for actual use.
2. I solve problem 1 myself by prefixing all template filenames with an underscore _ so that they are ignored by their respective tools.
3. Now because of 2, I have a new problem. When my generator runs and successfully compiles the templates and places them within the destinationRoot(), the generated files won't have special meaning. In order to gain special meaning, I need to remove the underscore _ prefix to undo the damage I did earlier.
4. There's no magical this.fs.removeUnderscorePrefixes(). :) So I need to get a list of the generated files, loop through them, and use this.fs.move() on them. Normally, I would use Node's fs.readdir() to acquire the list of files. But since mem-fs-editor operates in memory, this isn't possible. How do I get the list?

[SBoudrias]

For this use case, I think registering a transform stream with registerTransformStream will be the simplest implementation. http://yeoman.io/authoring/file-system.html

[sholladay]

Hmm okay that is a solid step in the right direction!

I got pretty close with this:

this.registerTransformStream(new stream.Transform({
    objectMode : true,
    transform(file, encoding, callback) {
        const repaired = path.join(file.dirname, unprefix(file.basename));
        file.path = repaired;
        callback(null, file);
    }
}));

The trouble is, I unexpectedly receive .yo-rc.json and .yo-rc-global.json in the stream. Yikes, I don't want to accidentally rename those!

Solutions I can think of:

1. Implement a blacklist and maintain it in tandem with whatever files Yeoman feels like putting on disk today.
2. Hope and pray that Yeoman never uses files with an underscore _ prefix and use that as a smell test that the files are my templates.
3. Perhaps filter based on whether file.path starts with this.sourceRoot(). But I'm not sure how symlinks will play into this. Still might be brittle.

Semantically speaking, I want a transform stream like this but only for my templates. Also, should I be cloning the files in the stream? Not sure if it matters.

[sholladay]

Oh, no, solution 3 doesn't work because file.path points to the destination not the source. So there's no way to tell if a file came from the template directory.

Even file.history doesn't contain a record of the source.

Here is solution 2, the "hope and pray" solution.

this.registerTransformStream(new stream.Transform({
    objectMode : true,
    transform(file, encoding, callback) {
        // Only process template files.
        if (file.basename.startsWith('_')) {
            file.path = path.join(file.dirname, unprefix(file.basename));
        }
        callback(null, file);
    }
}));

[moritzraho]

I think registerTransformStream can't be used in many cases because of yeoman/generator#819
IMO the easiest way would be to expose a list method on mem-fs-editor