Only running multi-tasks for file targets that have changed

[mehcode]

Let's say we have a repository with several .coffee files under a src/ directory with a root Gruntfile.js. This Gruntfile.js defines options for the grunt-contrib-coffee task (which it has loaded). Upon initially executing grunt coffee, every .coffee file in src/ is ran through the grunt-contrib-coffee task. Upon executing grunt coffee again, nothing happens. Because grunt knows that the .coffee files haven't changed therefore it has filtered the file mapping down to only the destination targets in which one or more of its source files have changed.

If grunt were to maintain a simple hash of source file contents it could easily filter the file list for multi-target tasks.

See gruntjs/grunt-contrib-watch#22 (comment) for more of my rambling.

Does this make sense to anyone? It seems this would be an awesome addition and allow for something like grunt-contrib-watch to just re-run tasks when it thinks files changed and not worry about anything because grunt-itself filters the file lists.

[cowboy]

Any task can store persistent data in a database, temp file, etc. Grunt just runs tasks.

[tkellen]

Hey Ryan,

Thanks for taking the time to share this idea with us! I just wanted to chime in to say that this could potentially be a really cool concept, but Ben is right, the implementation belongs outside of Grunt itself (we're moving in the direction of Grunt itself being a task runner only).

That said, if you take a stab at making this and you find it works well, please let us know! Your solution likely wouldn't fit within the task runner itself, but it might find a home as a dependency somewhere.

[cowboy]

^ What he said

[mehcode]

Hmm..

Okay. I agree that the implementation itself belongs outside of grunt but it would need to be used inside grunt to allow to properly filter file lists.

What I'm thinking is basically a node.js module that exposes a filter method that takes the expanded form of the files object and returns those that have changed. This module would need to keep track of the files and the first time it was asked to filter it would simply record hashes and successive times it would compute and compare to filter the lists.

Or perhaps a method isModified that can be called on a task level in which the task can filter it self if it wants to. This might be better.

[shama]

Reliably determining which files have been modified is not as simple as it seems ;) This can be handled in grunt-contrib-watch without modifying grunt's core. It already essentially does what you're describing... just doesn't have an option to feed the modified files to the tasks yet. I think this just needs to be implemented.

[mehcode]

This can be handled in grunt-contrib-watch without modifying grunt's core.

But it would be a lot more flexible if tasks were able to filter themselves by checking if files were modified with some utility library. That way watch doesn't care. It just re-runs tasks. Also running grunt coffee over and over again doesn't do anything besides what it needs to.

Reliably determining which files have been modified is not as simple as it seems ;)

I'll agree that it is not simple. But, yes, it is easy Maintaining an in-memory database (that is persisted to a file) of MD5 hashes of file contents and referencing this when asked if a file has changed is easy but complex. waf has been doing this for a long time.

I think this just needs to be implemented.

That has hack written all over it. It will work for some cases sure. But its not flexible or powerful enough. All watch would need to do is utilize this same utility library and get notified when things change and notify tasks that are responsible for those files. These tasks would then get out their list of files and ask if these files have been modified before continuing. This allows tasks to be written that for some reason don't want to filter themselves.

[shama]

FWIW, I think it's a cool idea and it has been suggested here #46, many #212, times #193, before #370. I'm just skeptical it is as easy as you make it sound. I have yet to see a pull request or a task running lib that does it.
Looking forward to seeing what you come up with though!

[tschaub]

See the grunt-newer task for an implementation of this at the task level.

Example using grunt-newer with the grunt-contrib-coffee task:

  grunt.initConfig({
    coffee: {
      all: {
        expand: true,
        cwd: 'path/to/src/',
        src: '**/*.coffee',
        dest: 'path/to/dest/',
        ext: '.js'
      }
    }
  });
  grunt.loadNpmTasks('grunt-contrib-coffee');
  grunt.loadNpmTasks('grunt-newer');

  grunt.registerTask('compile', 'newer:coffee');

With the above config, running grunt compile will configure the coffee task to compile only those .coffee scripts that are newer than existing .js scripts generated in previous runs.