Skip to content
This repository has been archived by the owner. It is now read-only.

Identical blocks being processed multiple times #289

Closed
iamrandys opened this Issue Feb 2, 2014 · 7 comments

Comments

Projects
None yet
7 participants
@iamrandys
Copy link

iamrandys commented Feb 2, 2014

I have a lot of html files that are generated by a static site generator. Since I am using layouts in the SSG, the files have duplicate and blocks that are the same. I have found that useminPrepare creates a concat task for all of the blocks even though they are all identical. concat is then processing the same blocks multiple times. With a large site, this becomes especially slow when duplicate uglify tasks are also created (I was able to get around this by creating a custom flow). Plus, it is also error prone since the last duplicate concat task wins (which may be different).

It would be really nice to have the option to "ignoreDuplicateBlocks" so they won't be processed multiple times. Also if one of the duplicates is composed of different js or css files, an error should be raised and the build should fail.

If someone can give me an idea of the best way to fix this, I would be glad to work on it.

Example:

file1.hml

    <!-- build:js({.tmp,app}) scripts/base.js -->
    <script src="/scripts/script1.js"></script>
    <script src="/scripts/script2.js"></script>
    <!-- endbuild -->

file2.html

    <!-- build:js({.tmp,app}) scripts/base.js -->
    <script src="/scripts/script1.js"></script>
    <script src="/scripts/script2.js"></script>
    <!-- endbuild -->

Would create identical config tasks something like:

concat:
{ dist: {},
generated: 
 { files: 
    [
      { dest: '.tmp/concat/scripts/base.js',
        src: 
         [ '{.tmp,app}/scripts/script1.js',
           '{.tmp,app}/scripts/script2.js' ] },

      { dest: '.tmp/concat/scripts/base.js',  <-- same target processed multiple times
        src: 
         [ '{.tmp,app}/scripts/script1.js',  <-- if there are any differences in the srcs dest/src will win - should be an error if they are different srcs
           '{.tmp,app}/scripts/script2.js' ] },
    ]
  }
}
@eddiemonge

This comment has been minimized.

Copy link
Member

eddiemonge commented Feb 4, 2014

look at generator-jekyllrb and see how it uses usemin. I have a feeling you are doing the usemin task at the wrong point in the site creation

@iamrandys

This comment has been minimized.

Copy link
Author

iamrandys commented Feb 9, 2014

grunt.registerTask('build', [
  'clean',
  'jekyll:dist',     <--- site generation
  'concurrent:dist',
  'useminPrepare',   <--- only scans index.html
  'concat',
  'autoprefixer:dist',
  'cssmin',
  'uglify',
  'imagemin',
  'svgmin',
  'filerev',
  'usemin',
  'htmlmin'
]);

That project does it exactly the same way I'm doing it and in the same order. The only thing different is that it is scanning only the root index.html file. My project is large. Each page does use the same layout, but each page is also able to define css and javascript blocks that get inserted into the main layout header. In this case each page will have the same core javascript files and can have different css and javascript blocks as well. If I just scan the index.html, it won't find these extra blocks.

You did give me a good idea though. I ran useminPrepare against my template files before they were assembled. This way only the unique blocks were added and it is very efficient.

Thanks for the help!

@ariesjia

This comment has been minimized.

Copy link

ariesjia commented Feb 27, 2014

I also met same problem ,Each page have own conca js file and concat gblobal js file.
But the concat and uglify will do so many time.
It's very slow.

@danhowitt

This comment has been minimized.

Copy link

danhowitt commented Mar 5, 2014

  • 1 with this. Very slow.
@rockallite

This comment has been minimized.

Copy link

rockallite commented Mar 21, 2014

Seems like grunt-usemin is currently only suitable in single-html-file scenario.

I think duplicate blocks caused by multiple html files should be eliminated by programs, not human hands -- that's what a "scaffolding tool" is created for!

...And I made a pull request for this.

@addyosmani

This comment has been minimized.

Copy link
Member

addyosmani commented Mar 24, 2014

@eddiemonge would be good if you could review 324.

boushley pushed a commit to boushley/grunt-usemin that referenced this issue Jun 11, 2014

Aaron Boushley
Skip duplicate configurations.
This should resolve yeoman#289.

This also detects a bad configuration where multiple blocks attempt to write
to the same destination with different sources.

boushley pushed a commit to boushley/grunt-usemin that referenced this issue Jun 11, 2014

Aaron Boushley
Skip duplicate configurations.
This should resolve yeoman#289.

This also detects a bad configuration where multiple blocks attempt to write
to the same destination with different sources.
@sindresorhus

This comment has been minimized.

Copy link
Member

sindresorhus commented Jun 23, 2014

see #382

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.