add cat

[stefanpenner]

this would allow someone to quickly cat changed files contents into a function for simple transforms

var tree = cat(tree, function(inputString) { return outputString; })

var dist = cat(tree, function prepend(file) {
  return license + "\n" + file;
});

[rwjblue]

Likely would use @ef4's source map aware concat internally.

[rwjblue]

Can we close this as map allows this fairly straightforwardly?

[stefanpenner]

We likely do need cat to combine

[rwjblue]

map supports the exact syntax in the lead-in description of this issue. Can you update the description to demonstrate what cat needs to add to map?

[stefanpenner]

Afk ATM. It should work like cat normally does

When I get home I'll flesh out the issue :)

[rwjblue]

It should work like cat normally does.

This isn't really helpful to me without some syntax examples. I'm sorry for being dense.

When I get home I'll flesh out the issue :)

Sounds good.

[stefanpenner]

Ya I agree. I'll flesh out the design asap

[stefanpenner]

the following might be a good starting point:

cat is M:1 transform with the following signature cat(tree, inputGlobs..., outputFile);

cat(tree, '**/*.js', 'out.js');
cat(tree, '**/*.js', '**/*.es6', 'out.js');
cat(tree, glob1, glob2, globn..., 'out.js');
cat(tree, '**/*.js', '!app.js', 'out.js'); // concat all .js but app.js into app.js

points:

• source maps by default would be ideal
• as with all other broccoli-stew helpers, all files are linked to the next stage.

questions:

• should we allow a simple callback transform like map? Maybe a callback that is given enough information to do the combine.
• what other features might be needed?
• should we support multiple globs like cat(tree, '*/.js', , ''out.js');

[stefanpenner]

I wonder if cat can be even more general purpose.

cat(tree, '**/*.js', transform);

where transform gets all the needed details passed as args, so any filter can be implemented via it.

@rwjblue / @chadhietala / @Rich-Harris thoughts?

[Rich-Harris]

Not sure I understand what the transform argument is for - wouldn't it be better to keep cat and map functionality separate? That way at least it's fairly straightforward for cat to generate sourcemaps. Or are you thinking that the transform function would determine the concatenation order, so it could do browserify-esque things?

[stefanpenner]

map is 1:1, cat is M:N (primarily M:1) but both likely want to optionally delegate functionality to arbitrary transforms.

An example to help illustrate my early thoughts

cat **/*.js | es6ModuleBundler > out.js # M:1

cat(tree, '**/*.js',  es6ModuleBundler(options), 'out.js'); // M:1

map(tree, '**/*.js',  6to5(options)); // 1:1

maybe:

var pipeline = [
  es6ModuleBundler(options),
  write('out.js')
];

cat(tree, '**/*.js', ...moreGlobs, pipeline); // M:1

[chadhietala]

In the examples above aren't we forcing transforms to be closures? What might be better is to just do.

function pipeline(tree) {
  var es6 = 6to5(tree, {...});
  es6 = es3SafeRecast(es6);
  return write('out.js');
}

var dist = cat(tree, '**/*.js', ...moreGlobs, pipeline);

This approach is leaky though because the developer needs to know that they need to return a write call from the pipeline.

The approach below forces developers into defining a resulting file like in the stdout example above.

function pipeline(tree) {
  var es6 = 6to5(tree, {...});
  return es3SafeRecast(es6)
}

var dist = cat(tree, '**/*.js', ...moreGlobs, pipeline, 'out.js');
// or
var dist = cat(tree, '**/*.js', ...moreGlobs, 'out.js');

[chadhietala]

I suppose we could create write method that allowed to do M:N through the pipeline?

[stefanpenner]

hmm nice, i like the, string as nth arg, being M:1, function/closure return arbitrary tree that can be M:1 or M:N.

[stefanpenner]

https://www.npmjs.com/package/magic-string might be interesting re: source maps cc @chadhietala

[chadhietala]

Comparing the source of fast-source-map-concat and magic string, it seems magic string sourcemap concating would be faster.