New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Same Files compiling multiple times in nanoc-4.2.1 #886

Closed
pmalik56 opened this Issue Jul 1, 2016 · 14 comments

Comments

Projects
None yet
5 participants
@pmalik56

pmalik56 commented Jul 1, 2016

Same file iterate inside compile and route block for 5-6 times not sure what is the reason or how nanoc identify whether a file compiled or not? In previous version 3.7.5 it was compiling once for a file.

compile '/*/' do

Same file enter in this blok multiple times

# Compilation logic here

end

screenshot from 2016-07-01 16 51 24

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jul 1, 2016

Member

Can you share the Rules file with me so I can take a look?

Member

ddfreyne commented Jul 1, 2016

Can you share the Rules file with me so I can take a look?

@pmalik56

This comment has been minimized.

Show comment
Hide comment
@pmalik56

pmalik56 Jul 4, 2016

Here below is my rules file.

rules1
rules2

pmalik56 commented Jul 4, 2016

Here below is my rules file.

rules1
rules2

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jul 4, 2016

Member

Can you show me where you are printing Compiling /aspa_Adobe_kayttoehdot.pdf? I can’t see it in the Rules file that you posted.

The rules are often evaluated twice (or more) to identify the processing instructions that are used, although the actual compilation is only done once. I expect this to be the reason.

Member

ddfreyne commented Jul 4, 2016

Can you show me where you are printing Compiling /aspa_Adobe_kayttoehdot.pdf? I can’t see it in the Rules file that you posted.

The rules are often evaluated twice (or more) to identify the processing instructions that are used, although the actual compilation is only done once. I expect this to be the reason.

@pmalik56

This comment has been minimized.

Show comment
Hide comment
@pmalik56

pmalik56 Jul 4, 2016

its printing here

compile /*/ do
if it.binary?
puts 'compiling.....' + item.identifier

I have added that pdf intentionally to verify.... otherwise it give a big /tmp/routes.txt file which have multiple same entry.

pmalik56 commented Jul 4, 2016

its printing here

compile /*/ do
if it.binary?
puts 'compiling.....' + item.identifier

I have added that pdf intentionally to verify.... otherwise it give a big /tmp/routes.txt file which have multiple same entry.

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jul 4, 2016

Member

Is this line always printed exactly five times?

Can you share the output of nanoc compile --debug?

Member

ddfreyne commented Jul 4, 2016

Is this line always printed exactly five times?

Can you share the output of nanoc compile --debug?

@pmalik56

This comment has been minimized.

Show comment
Hide comment
@pmalik56

pmalik56 Jul 4, 2016

nannoc_compile

It's not happening exactly five times always.But same file compiling same number of times after each compilation.

/content/index.html ==> 5 times
/content/abc/index.html ==> 5 times
/content/abc/def/index.html ==> 6 times
/content/abc/def/ghi/index.html ==> 6 times
/content/asiakaspalvelu/aihe/sahkoposti-kotisivut/ohje/kotisivutila-ominaisuudet/index.html ==> 15 times

/content/asiakaspalvelu/aihe/kirja/ohje/kopiosuojausvaltuutus/index.html ==> 12 times

Looks like it's happening for json, index.html and other file type as if in all compilation block executing multiple times for same file.

pmalik56 commented Jul 4, 2016

nannoc_compile

It's not happening exactly five times always.But same file compiling same number of times after each compilation.

/content/index.html ==> 5 times
/content/abc/index.html ==> 5 times
/content/abc/def/index.html ==> 6 times
/content/abc/def/ghi/index.html ==> 6 times
/content/asiakaspalvelu/aihe/sahkoposti-kotisivut/ohje/kotisivutila-ominaisuudet/index.html ==> 15 times

/content/asiakaspalvelu/aihe/kirja/ohje/kopiosuojausvaltuutus/index.html ==> 12 times

Looks like it's happening for json, index.html and other file type as if in all compilation block executing multiple times for same file.

@agross

This comment has been minimized.

Show comment
Hide comment
@agross

agross Jul 5, 2016

Contributor

It might be the case that Nanoc evaluates the compile block multiple times, but does not run any filters. After all, compile is just a definition of what should happen, not the actual compilation.

Have you tried putting that puts debug statement in the actual Filter#run method?

Contributor

agross commented Jul 5, 2016

It might be the case that Nanoc evaluates the compile block multiple times, but does not run any filters. After all, compile is just a definition of what should happen, not the actual compilation.

Have you tried putting that puts debug statement in the actual Filter#run method?

@pmalik56

This comment has been minimized.

Show comment
Hide comment
@pmalik56

pmalik56 Jul 5, 2016

Here is the log when I made some changes in /aihe/asiakaskoulutukset/index.html this file it's filtering multiple times and some other files also coming in that #run method.

nanoc_compile_1
nanoc_compile_2

But when I am not doing any changes to the sites it looks fine.
Here is the log with no change :
nanco_compile_no_change

But most of the cases we made changes to site and publish it and my only concern is if it's taking longer time to process the files.

And this also happen in route block as well. so is it routing the same file again and again?

pmalik56 commented Jul 5, 2016

Here is the log when I made some changes in /aihe/asiakaskoulutukset/index.html this file it's filtering multiple times and some other files also coming in that #run method.

nanoc_compile_1
nanoc_compile_2

But when I am not doing any changes to the sites it looks fine.
Here is the log with no change :
nanco_compile_no_change

But most of the cases we made changes to site and publish it and my only concern is if it's taking longer time to process the files.

And this also happen in route block as well. so is it routing the same file again and again?

@agross

This comment has been minimized.

Show comment
Hide comment
@agross

agross Jul 6, 2016

Contributor

Here's what I did:

  • nanoc create-site test

  • Edited Rules:

    compile '/**/*.html' do
      puts 'rules: ' + item.identifier # added debug statement
      filter :erb # added erb filter for item
    
      layout '/default.*'
    end
    
    # NOTE: the default Rules define an erb filter defined for layouts as well!
  • Edited erb.rb of the Nanoc gem:

    def run(content, params = {})
      puts "erb for #{@item.identifier}: #{content[0...10]}" # added debug statement
      ...
  • rm -rf output; nanoc compile --verbose prints:

    Loading site... done
    Compiling site...
    rules: /index.html
    rules: /index.html
    erb for /index.html:
    <h1>A Bra
    erb for /index.html: <!DOCTYPE
          create  [0.01s]  output/index.html
          create  [0.00s]  output/stylesheet.css
    
        | count    min    avg    max     tot
    ----+-----------------------------------
    erb |     2  0.00s  0.00s  0.01s   0.01s
    
    Site compiled in 0.06s.
    

So while the rule blocks are evaluated more than once (one may ask why, for example if rules are expensive), the actual erb filtering is just done once per item. You'll see two erb filter runs in the output, though:

  • First, the item content is filtered by erb as per my updated compile rule.
  • Second, the layout is filtered by erb, combining the HTML output of content/index.html and the erb input of layouts/default.html.

While the debug output of Nanoc's erb filter indicates that the @item.identifier is the same, the input is not the same (<h1>A Bra... of the item vs. <!DOCTYPE... of the layout).

Contributor

agross commented Jul 6, 2016

Here's what I did:

  • nanoc create-site test

  • Edited Rules:

    compile '/**/*.html' do
      puts 'rules: ' + item.identifier # added debug statement
      filter :erb # added erb filter for item
    
      layout '/default.*'
    end
    
    # NOTE: the default Rules define an erb filter defined for layouts as well!
  • Edited erb.rb of the Nanoc gem:

    def run(content, params = {})
      puts "erb for #{@item.identifier}: #{content[0...10]}" # added debug statement
      ...
  • rm -rf output; nanoc compile --verbose prints:

    Loading site... done
    Compiling site...
    rules: /index.html
    rules: /index.html
    erb for /index.html:
    <h1>A Bra
    erb for /index.html: <!DOCTYPE
          create  [0.01s]  output/index.html
          create  [0.00s]  output/stylesheet.css
    
        | count    min    avg    max     tot
    ----+-----------------------------------
    erb |     2  0.00s  0.00s  0.01s   0.01s
    
    Site compiled in 0.06s.
    

So while the rule blocks are evaluated more than once (one may ask why, for example if rules are expensive), the actual erb filtering is just done once per item. You'll see two erb filter runs in the output, though:

  • First, the item content is filtered by erb as per my updated compile rule.
  • Second, the layout is filtered by erb, combining the HTML output of content/index.html and the erb input of layouts/default.html.

While the debug output of Nanoc's erb filter indicates that the @item.identifier is the same, the input is not the same (<h1>A Bra... of the item vs. <!DOCTYPE... of the layout).

@pmalik56

This comment has been minimized.

Show comment
Hide comment
@pmalik56

pmalik56 Jul 8, 2016

ok now it's clear why it's filtering multiple times. Thanks for your valuable reply.

One more thing I am facing issue on processing static component. Seems like rules are not working on static folder as if it's not processing at all anything inside static. Previously its was working on nanoc 3.7.5 version. And I have some layouts as well in static folder which is getting this error message.

Nanoc::Int::Errors::UnknownLayout: The site does not have a layout with identifier “/static/components/site-common/analytics/google-tag-manager.erb”

I am adding layout this way but still not working getting same error.

new_layout(File.open('./static/components/site-common/analytics/google-tag-manager.erb', 'r').read, {}, '/static/components/site-common/analytics/google-tag-manager.erb')

And new_item also not working as i was trying to add content inside static directory to process.

pmalik56 commented Jul 8, 2016

ok now it's clear why it's filtering multiple times. Thanks for your valuable reply.

One more thing I am facing issue on processing static component. Seems like rules are not working on static folder as if it's not processing at all anything inside static. Previously its was working on nanoc 3.7.5 version. And I have some layouts as well in static folder which is getting this error message.

Nanoc::Int::Errors::UnknownLayout: The site does not have a layout with identifier “/static/components/site-common/analytics/google-tag-manager.erb”

I am adding layout this way but still not working getting same error.

new_layout(File.open('./static/components/site-common/analytics/google-tag-manager.erb', 'r').read, {}, '/static/components/site-common/analytics/google-tag-manager.erb')

And new_item also not working as i was trying to add content inside static directory to process.

@th-h

This comment has been minimized.

Show comment
Hide comment
@th-h

th-h Jul 9, 2016

pmalik56 wrote:

One more thing I am facing issue on processing static component.
Seems like rules are not working on static folder as if it's not
processing at all anything inside static. Previously its was working
on nanoc 3.7.5 version.

Do you mean the static data source? That was removed in nanoc 4.

On the upgrade page, it says:

If you use the static data source, disable it for now and follow the
extended upgrade instructions below.

See
http://nanoc.ws/doc/nanoc-4-upgrade-guide/#upgrading-from-the-static-data-source.

-thh

th-h commented Jul 9, 2016

pmalik56 wrote:

One more thing I am facing issue on processing static component.
Seems like rules are not working on static folder as if it's not
processing at all anything inside static. Previously its was working
on nanoc 3.7.5 version.

Do you mean the static data source? That was removed in nanoc 4.

On the upgrade page, it says:

If you use the static data source, disable it for now and follow the
extended upgrade instructions below.

See
http://nanoc.ws/doc/nanoc-4-upgrade-guide/#upgrading-from-the-static-data-source.

-thh

@pmalik56

This comment has been minimized.

Show comment
Hide comment
@pmalik56

pmalik56 Jul 11, 2016

Is there any way to create a new layout of erb which is not in layout directory but in content directory.

Like I have one component 'site-common' in static folder which I wise to put inside content directory and it has some erb files to be used as layout. But I am not able to create new layout using #new_layout this function.

pmalik56 commented Jul 11, 2016

Is there any way to create a new layout of erb which is not in layout directory but in content directory.

Like I have one component 'site-common' in static folder which I wise to put inside content directory and it has some erb files to be used as layout. But I am not able to create new layout using #new_layout this function.

@denis-soundcloud

This comment has been minimized.

Show comment
Hide comment
@denis-soundcloud

denis-soundcloud Jul 11, 2016

Please use the Nanoc Google Group for support. GitHub issues is only for issues.

I’ll close this for now!

denis-soundcloud commented Jul 11, 2016

Please use the Nanoc Google Group for support. GitHub issues is only for issues.

I’ll close this for now!

@ddfreyne

This comment has been minimized.

Show comment
Hide comment
@ddfreyne

ddfreyne Jul 12, 2016

Member

FYI, this will happen far less in 4.2.4, thanks to #902.

Member

ddfreyne commented Jul 12, 2016

FYI, this will happen far less in 4.2.4, thanks to #902.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment