Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Added ability to compile a directory and maintain folder structure #1

Closed
wants to merge 2 commits into from

6 participants

@blakeblackshear

Need ability to compile into separate files but maintain directory structure. This allows rjs to optimize the compile files. Needed to use this project with Yeoman as referenced here: yeoman/yeoman#522

@ctalkington

don't care for a few parts of this at quick glance. Ive been diving more into destination type matching recently. look me look into this type of feature with a refreshed mindset and see what comes of it. will also be attempting to support similar function in contrib-stylus.

@blakeblackshear

Made some adjustments.

@tkellen
Owner

Heya Blake! Thanks for taking the time to put this PR together. This is a pretty common request, and we're seeing it across almost all of the contrib tasks. The solution we really need (and the one I'd recommend you use for the time being) is to dynamically generate the file listing for your gruntconfig, producing a 1-to-1 mapping of each file landing in the destination you want.

As it stands now, the coffee task is capable of doing everything you need it to, without further modification. What we really need is a better way to generate the file listing config. Chris has done some interesting work in this area, and I hope we'll have a solution soon.

@ctalkington

@tkellen while I generally agree. Being so requested I feel it should be baked in to prevent confusion. If you check out stylus, I did some work on making it detected wildcard plus ext in dest. For async tasks it produces bit heavier code but think like basePath will be consolidated into lib.

I'm thinking at this point ill code similar support into less and coffee and if we get requests for any others can handle it then.

@tkellen
Owner

Fair enough. Can we pull as much of the logic into contrib-lib as possible and cut a release on monday?

@ctalkington

ok so ive done a lot of updates contrib-wide. still some more to be done but closing out this issue/pr. this should be on npm in the next day or so.

its a little different than your proposed format but you can use:

coffee: {
  compile: {
    files: {
      'dest/*.js': ['path/to/sources/*.coffee', 'path/to/more/*.coffee']
    }
  }
}

this should produce

- dest
-- sources
--- file1,js
--- file2.js
-- more
--- file1.js
--- file2.js

you can also set flatten to true or override basePath (as sometimes its auto isn't perfect, depends on structure desired). I stuck with this way of doing things so its similar between contrib tasks.

@d4tocchini

Does not work as expected if one path is passed in array and path/to/source has one .coffee. So,

coffee: {
  compile: {
    files: {
      'dest/*.js': ['path/to/sources/*.coffee']
    }
  }
}

produces:

- dest
-- file1.js

it should produce:

- dest
-- sources 
--- file1.js
@ctalkington

wildcard feature is experimental and likely to change soon.

part of the behavior you mention comes from auto basePath. it sees sources as the base so everything it does starts relative from that level. there's only a few edge cases where itd output like your example.

I'd recommend just including sources in dest path. ie dest/sources/*.js

@d4tocchini

Workarounds are straight forward, just giving you a heads up that I ran into unexpected behavior based on a reading of the docs.

If the wild card feature is experimental, what do you consider to be the scope of possibility for a more stable solution?

It would be nice to have the source's basePath as a template variable for the destination string, or something similar.

@ctalkington

I do want to clear up docs and provide more detail in the future. The basePath feature has a few edge cases that could use some help.

Far as wildcard, its still a bit rough at this point but its an active discussion that I know @tkellen and @cowboy are working hard to rollout in a smart way. I believe it'll be in form of a helper that builds multiple files object pairs.

@shama
Owner

FWIW, I've been trying to solve this as well in grunt-docs. This is what I have so far: https://github.com/shama/grunt-docs/blob/master/tasks/lib/docs.js#L23 It guesses the base paths using the patterns to keep the sub folder structure in tact.

@d4tocchini

The "guessing" methods seem to be overcomplicating the issue b/c it's trying to maintain subfolder structure given just the asterisk when the destination includes: /*.js. The semantics of that doesn't sense anyway. Why not expose the basePath as a directive, something like:

coffee: 
  compile: 
    files: 
      'compiled-scripts/<file:basePath>/<file:fileName>-did-compile.js': ['path/to/sources/*.coffee', 'path/to/more/*.coffee'] 

This would allow for a much richer API, clear-up the ambiguity, and take the pressure away from relying on a one-size-fits-all guessing convention.

@Iristyle

So what is the recommendation at this point for compiling dir full of coffee -> dir full of js, completely preserving hierarchy?

coffee: {
  app: {
    files:
    {
      'generated/js/' :
        ['app/js/**/*.coffee']
    }
}

This flattens the directory structure, which as mentioned, is not good.

How are people constructing the list right now?

@Iristyle

I see you mention this has been fixed @tkellen ... https://github.com/gruntjs/grunt-contrib/issues/54

But I'm not seeing it.

@ctalkington

you would want a wildcard on the dest side.

ie from repo example

files: {
  'path/to/*.js': ['path/to/sources/*.coffee', 'path/to/more/*.coffee'] // compile individually into dest, maintaining folder structure
}

EDIT: I should note this is very experimental and may be yanked at anytime, the same reason this is one of the only to have this feature. we hope to address it with a more robust solution in grunt v0.5.

@ctalkington

yah those are valid points too, just coffee has the logic built-in but using something similar to the gist for now is prob best.

@Iristyle

Here's an alternate method that should work, by simply relying on grunt.file.expand

files: {
  'generated/js/*.js' : grunt.file.expand({cwd: path.resolve('app/js')},
    '**/*.coffee')
}

You can see in the REPL that my input file list looks OK

> grunt.file.expand({cwd: path.resolve('app/js')}, '**/*.coffee')
[ 'app.coffee',
  'appFake.coffee',
  'controllers/auth.coffee',
  'controllers/controllers.coffee',
  'controllers/inbox.coffee',
  'controllers/order.coffee',
  'directives/directives.coffee',
  'directives/fadeInOut.coffee',
  'directives/modalLogin.coffee',
  'filters/epsrocks.coffee',
  'filters/filters.coffee',
  'main-testacular.coffee',
  'main.coffee',
  'mainFake.coffee',
  'responseInterceptors/dispatcher.coffee',
  'responseInterceptors/responseInterceptors.coffee',
  'services/auth.coffee',
  'services/order.coffee',
  'services/ping.coffee',
  'services/services.coffee',
  'services/userdata.coffee',
  'services/userdatastore.coffee',
  'services/uuid.coffee',
  'statuses.coffee' ]
@ctalkington

are you using the newer version of the task? maybe if you provided a repo that I could clone locally and play with I'd have a more solid answer as to the issue but beyond that its unsupported. im personally ready to yank it out vs dealing with all the issues it brings.

you're also kind of twisting what the gist actually does. you're going to be best off doing something like (using the function in gist)

target: {
  files: globToMultiFiles('app/js/*.coffee', 'generated/js', 'js')
}

the thing is when i wrote that gist it was meant for single dir so it won't actually keep your structure which kind of means your between a rock and hard place.

@Iristyle

Redcated - USER ERROR

@ctalkington

im not actually sure if anyone has tested this task with grunt@devel. lots of things changed in grunt v0.4.

i know we have some work started for the transition but things are a bit bumpy right now. we are starting to get more people involved and everyone kinda has a task that they take charge of, right now @tkellen and i are somewhat sharing the duties on this one and he has also been busy helping get grunt v0.4 out the door.

i will see if i can put together a gist that is smart enough to handle subdirs. as we look for such helper functions to be the route to go for max flexibility since it uses a standard files object vs having each task have to process out wildcards.

@Iristyle

Ohhhhhhhhhhhhhh no.... I think the node_modules folder is still hanging around for the other task!

Stupid stupid stupid.

@Iristyle

This lack of sleep is really catching up with me! I'm really sorry for wasting your time. If I have any real issues, I'll pass them along -- but it would appear that is totally PEBKAC

@ctalkington

no problem. we've all been there. if you do have any other issues, please open a new issue with as much detail as possible.

@Iristyle

I'm giong to to trash a few of those red herring posts from above, because I don't think they add much value here... and might cause confusion. Once again, apologies...

@tkellen
Owner

grunt@devel now has grunt.file.expandMapping

Here is a sample of how you would use it:

coffee: {
  compile: {
    files: grunt.file.expandMapping(['path/to/*.coffee'],'path/to/dest/',{
      rename: function(destBase,destPath) {
        return destBase+destPath.replace(/\.coffee$/,".js");
      }
    })
  }
}
@jjingob jjingob referenced this pull request from a commit in jjingob/grunt-contrib-coffee
@jjingob jjingob Create census
Census Bureau Homepage
#1 source for current population data and the latest Economic Indicators. Households to Business to Government - 
Income & Spending. Race by Sex by Age or poverty & health insurance data. Forei...
http://www.census.gov/
cdc9de5
@jjingob jjingob referenced this pull request from a commit in jjingob/grunt-contrib-coffee
@jjingob jjingob Create atsdr.cdc
ATSDR Home »
Feature 
#1 - Read environmental health success stories; Feature 
#2 - ATSDR recognizes the National ALS Registry; Feature 
#3 - Oak Ridge Reservation - Y-12 Mecury Report. 
The Agency for Toxic Substance...
http://www.atsdr.cdc.gov/
86220ae
@jjingob jjingob referenced this pull request from a commit in jjingob/grunt-contrib-coffee
@jjingob jjingob Create census
Census Bureau Homepage »
#1 source for current population data and the latest Economic Indicators. 
Households to Business to Government - Income & Spending. 
Race by Sex by Age or poverty & health insurance data. Forei...
http://www.census.gov/
e515552
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
View
3  .gitignore
@@ -1,3 +1,4 @@
node_modules
npm-debug.log
-tmp
+tmp
+tmp_dir
View
8 grunt.js
@@ -35,7 +35,7 @@ module.exports = function(grunt) {
// Before generating any new files, remove any previously-created files.
clean: {
- test: ['tmp']
+ test: ['tmp','tmp_dir']
},
// Configuration to be run (and then tested).
@@ -43,10 +43,12 @@ module.exports = function(grunt) {
compile: {
files: {
'tmp/coffee_basic.js': ['test/fixtures/coffee_basic.coffee'],
- 'tmp/coffee_combined.js': ['test/fixtures/*.coffee']
+ 'tmp/coffee_combined.js': ['test/fixtures/*.coffee'],
+ 'tmp_dir': ['test/fixtures/**/*.coffee']
},
options: {
- bare: true
+ bare: true,
+ basePath: 'test/fixtures'
}
}
},
View
73 tasks/coffee.js
@@ -16,6 +16,7 @@ module.exports = function(grunt) {
grunt.registerMultiTask('coffee', 'Compile CoffeeScript files into JavaScript', function() {
var _ = grunt.util._;
+ var path = require('path');
var helpers = require('grunt-contrib-lib').init(grunt);
var options = helpers.options(this);
@@ -24,32 +25,74 @@ module.exports = function(grunt) {
// TODO: ditch this when grunt v0.4 is released
this.files = this.files || helpers.normalizeMultiTaskFiles(this.data, this.target);
+
+ this.files.forEach(function(file) {
+ if(path.extname(file.dest) === ''){
+ compileToDir(file, options);
+ }
+ else{
+ compileAndConcat(file, options);
+ }
+ });
+ });
+
+ var compileToDir = function(file, options){
+ grunt.file.mkdir(file.dest);
+ grunt.log.writeln('Directory ' + file.dest + ' created.');
+
+ var _ = grunt.util._;
+ var path = require('path');
+
+ var srcFiles,
+ destPath,
+ helperOptions;
+
+ srcFiles = grunt.file.expandFiles(file.src);
+
+ srcFiles.forEach(function(srcFile) {
+ destPath = path.dirname(srcFile);
+
+ if(options.basePath){
+ destPath = destPath.replace(new RegExp('^'+options.basePath), '');
+ }
+ destPath = path.join(file.dest, destPath);
+
+ var dest = path.join(destPath, path.basename(srcFile, '.coffee') + '.js');
+
+ helperOptions = _.extend({filename: srcFile}, options);
+
+ grunt.file.write(dest, compileCoffee(grunt.file.read(srcFile), helperOptions));
+ grunt.log.writeln('File ' + dest + ' created.');
+ });
+ };
+
+ var compileAndConcat = function(file, options){
+ var _ = grunt.util._;
+
var srcFiles;
var taskOutput;
var sourceCode;
var sourceCompiled;
var helperOptions;
- this.files.forEach(function(file) {
- srcFiles = grunt.file.expandFiles(file.src);
-
- taskOutput = [];
+ srcFiles = grunt.file.expandFiles(file.src);
- srcFiles.forEach(function(srcFile) {
- helperOptions = _.extend({filename: srcFile}, options);
- sourceCode = grunt.file.read(srcFile);
+ taskOutput = [];
- sourceCompiled = compileCoffee(sourceCode, helperOptions);
+ srcFiles.forEach(function(srcFile) {
+ helperOptions = _.extend({filename: srcFile}, options);
+ sourceCode = grunt.file.read(srcFile);
- taskOutput.push(sourceCompiled);
- });
+ sourceCompiled = compileCoffee(sourceCode, helperOptions);
- if (taskOutput.length > 0) {
- grunt.file.write(file.dest, taskOutput.join('\n'));
- grunt.log.writeln('File ' + file.dest + ' created.');
- }
+ taskOutput.push(sourceCompiled);
});
- });
+
+ if (taskOutput.length > 0) {
+ grunt.file.write(file.dest, taskOutput.join('\n'));
+ grunt.log.writeln('File ' + file.dest + ' created.');
+ }
+ };
var compileCoffee = function(coffeescript, options) {
try {
View
6 test/coffee_test.js
@@ -6,7 +6,7 @@ exports['coffee'] = {
var expect, result;
- test.expect(2);
+ test.expect(3);
expect = "var HelloWorld;\n\nHelloWorld = (function() {\n\n function HelloWorld() {}\n\n HelloWorld.test = 'test';\n\n return HelloWorld;\n\n})();\n";
result = grunt.file.read('tmp/coffee_basic.js');
@@ -16,6 +16,10 @@ exports['coffee'] = {
result = grunt.file.read('tmp/coffee_combined.js');
test.equal(expect, result, 'should compile multiple coffeescript files to a single javascript file');
+ expect = "var HelloWorld;\n\nHelloWorld = (function() {\n\n function HelloWorld() {}\n\n HelloWorld.test = 'test';\n\n return HelloWorld;\n\n})();\n";
+ result = grunt.file.read('tmp_dir/sub/coffee_sub.js');
+ test.equal(expect, result, 'should compile coffeescript to individual files in mirrored directory');
+
test.done();
}
};
View
2  test/fixtures/sub/coffee_sub.coffee
@@ -0,0 +1,2 @@
+class HelloWorld
+ @test: 'test'
Something went wrong with that request. Please try again.