Identical blocks being processed multiple times

[iamrandys]

I have a lot of html files that are generated by a static site generator. Since I am using layouts in the SSG, the files have duplicate and blocks that are the same. I have found that useminPrepare creates a concat task for all of the blocks even though they are all identical. concat is then processing the same blocks multiple times. With a large site, this becomes especially slow when duplicate uglify tasks are also created (I was able to get around this by creating a custom flow). Plus, it is also error prone since the last duplicate concat task wins (which may be different).

It would be really nice to have the option to "ignoreDuplicateBlocks" so they won't be processed multiple times. Also if one of the duplicates is composed of different js or css files, an error should be raised and the build should fail.

If someone can give me an idea of the best way to fix this, I would be glad to work on it.

Example:

file1.hml

    <!-- build:js({.tmp,app}) scripts/base.js -->
    <script src="/scripts/script1.js"></script>
    <script src="/scripts/script2.js"></script>
    <!-- endbuild -->

file2.html

    <!-- build:js({.tmp,app}) scripts/base.js -->
    <script src="/scripts/script1.js"></script>
    <script src="/scripts/script2.js"></script>
    <!-- endbuild -->

Would create identical config tasks something like:

concat:
{ dist: {},
generated: 
 { files: 
    [
      { dest: '.tmp/concat/scripts/base.js',
        src: 
         [ '{.tmp,app}/scripts/script1.js',
           '{.tmp,app}/scripts/script2.js' ] },

      { dest: '.tmp/concat/scripts/base.js',  <-- same target processed multiple times
        src: 
         [ '{.tmp,app}/scripts/script1.js',  <-- if there are any differences in the srcs dest/src will win - should be an error if they are different srcs
           '{.tmp,app}/scripts/script2.js' ] },
    ]
  }
}

[eddiemonge]

look at generator-jekyllrb and see how it uses usemin. I have a feeling you are doing the usemin task at the wrong point in the site creation

[iamrandys]

grunt.registerTask('build', [
  'clean',
  'jekyll:dist',     <--- site generation
  'concurrent:dist',
  'useminPrepare',   <--- only scans index.html
  'concat',
  'autoprefixer:dist',
  'cssmin',
  'uglify',
  'imagemin',
  'svgmin',
  'filerev',
  'usemin',
  'htmlmin'
]);

That project does it exactly the same way I'm doing it and in the same order. The only thing different is that it is scanning only the root index.html file. My project is large. Each page does use the same layout, but each page is also able to define css and javascript blocks that get inserted into the main layout header. In this case each page will have the same core javascript files and can have different css and javascript blocks as well. If I just scan the index.html, it won't find these extra blocks.

You did give me a good idea though. I ran useminPrepare against my template files before they were assembled. This way only the unique blocks were added and it is very efficient.

Thanks for the help!

[ariesjia]

I also met same problem ，Each page have own conca js file and concat gblobal js file.
But the concat and uglify will do so many time.
It's very slow.

[danhowitt]

• 1 with this. Very slow.

[rockallite]

Seems like grunt-usemin is currently only suitable in single-html-file scenario.

I think duplicate blocks caused by multiple html files should be eliminated by programs, not human hands -- that's what a "scaffolding tool" is created for!

...And I made a pull request for this.

[addyosmani]

@eddiemonge would be good if you could review 324.

[sindresorhus]

see #382