New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Entire site is rebuilt when making changes #1689

Closed
toolness opened this Issue May 5, 2016 · 6 comments

Comments

Projects
None yet
3 participants
@toolness
Contributor

toolness commented May 5, 2016

I noticed that the entire site seems to be rebuilt whenever I make changes to a file. For instance, the entire site takes 20 seconds to build when I start ./go serve; yet the following actions seem to take another 20 seconds to regenerate the site:

  • Making a minor edit to a SASS file;
  • Making a minor edit to a Markdown post.

This is surprising, since ./go serve runs jekyll with incremental build enabled. I did notice that config.yml contains the line full_rebuild: true, but commenting it out doesn't seem to change anything.

I can reduce the time to about 8 seconds by running jekyll serve --incremental --limit_posts 5, but I'm not sure where that 8 seconds comes from. Will do some profiling to investigate more!

@toolness

This comment has been minimized.

Show comment
Hide comment
@toolness

toolness May 6, 2016

Contributor

Ok, I did some more research/tinkering and here are my findings:

  • The actual rendering of liquid templates doesn't seem to take that long, when using jekyll's --profile option.
  • Deleting everything in _team saves about 3 seconds of build time.
  • Deleting everything in pages saves about 1 second of build time.
  • Deleting everything in assets saves about 3.5 seconds of build time.

So deleting all those is apparently what it takes to get the incremental build time down to a reasonable level of about one second. Unfortunately, it also results in a website with no front page, no CSS, and 5 blog posts.

It's particularly weird to me that processing everything in assets takes 3.5 seconds on an incremental build. Because that's standard jekyll functionality, I'd think that they'd have implemented logic to ignore doing anything with that folder on an incremental build unless something in there has actually changed.

Contributor

toolness commented May 6, 2016

Ok, I did some more research/tinkering and here are my findings:

  • The actual rendering of liquid templates doesn't seem to take that long, when using jekyll's --profile option.
  • Deleting everything in _team saves about 3 seconds of build time.
  • Deleting everything in pages saves about 1 second of build time.
  • Deleting everything in assets saves about 3.5 seconds of build time.

So deleting all those is apparently what it takes to get the incremental build time down to a reasonable level of about one second. Unfortunately, it also results in a website with no front page, no CSS, and 5 blog posts.

It's particularly weird to me that processing everything in assets takes 3.5 seconds on an incremental build. Because that's standard jekyll functionality, I'd think that they'd have implemented logic to ignore doing anything with that folder on an incremental build unless something in there has actually changed.

@toolness

This comment has been minimized.

Show comment
Hide comment
@toolness

toolness May 18, 2016

Contributor

Ok, so I've discovered that something is silently disabling incremental regeneration. This can be shown by creating a _plugins/boop.rb file and putting a hook in it:

require 'jekyll'

Jekyll::Hooks.register :site, :after_reset do |site|
  puts "site.regenerator.disabled? == #{site.regenerator.disabled?}"
end

On a fresh Jekyll site, running jekyll serve --incremental will print out something like this:

 Incremental build: enabled
      Generating...
site.regenerator.disabled? == false

However, on 18f.gsa.gov, it will print out:

 Incremental build: enabled
      Generating...
site.regenerator.disabled? == true

That's not a typo--it actually says that incremental builds are enabled, but then something disables it at a later point.

Next step is to figure out what's disabling it!

Contributor

toolness commented May 18, 2016

Ok, so I've discovered that something is silently disabling incremental regeneration. This can be shown by creating a _plugins/boop.rb file and putting a hook in it:

require 'jekyll'

Jekyll::Hooks.register :site, :after_reset do |site|
  puts "site.regenerator.disabled? == #{site.regenerator.disabled?}"
end

On a fresh Jekyll site, running jekyll serve --incremental will print out something like this:

 Incremental build: enabled
      Generating...
site.regenerator.disabled? == false

However, on 18f.gsa.gov, it will print out:

 Incremental build: enabled
      Generating...
site.regenerator.disabled? == true

That's not a typo--it actually says that incremental builds are enabled, but then something disables it at a later point.

Next step is to figure out what's disabling it!

@toolness

This comment has been minimized.

Show comment
Hide comment
@toolness

toolness May 18, 2016

Contributor

Ok, I was quite confused because there appeared to be no way to disable the incremental regenerator from outside the class, but jekyll-archives found a way:

Jekyll::Hooks.register :site, :after_reset do |site|
  # We need to disable incremental regen for Archives to generate with the
  # correct content
  site.regenerator.instance_variable_set(:@disabled, true)
end

I'd issue a PR to at least display a warning that this is happening, as it would've saved me a lot of debugging time, but apparently there's a fix nearing completion at jekyll/jekyll-archives#58, so that's good at least.

In the meantime, incremental regeneration can be enabled during development by temporarily commenting-out the jekyll-archives gem in the Gemfile. This seems to significantly speed things up on my machine.

Contributor

toolness commented May 18, 2016

Ok, I was quite confused because there appeared to be no way to disable the incremental regenerator from outside the class, but jekyll-archives found a way:

Jekyll::Hooks.register :site, :after_reset do |site|
  # We need to disable incremental regen for Archives to generate with the
  # correct content
  site.regenerator.instance_variable_set(:@disabled, true)
end

I'd issue a PR to at least display a warning that this is happening, as it would've saved me a lot of debugging time, but apparently there's a fix nearing completion at jekyll/jekyll-archives#58, so that's good at least.

In the meantime, incremental regeneration can be enabled during development by temporarily commenting-out the jekyll-archives gem in the Gemfile. This seems to significantly speed things up on my machine.

@gboone

This comment has been minimized.

Show comment
Hide comment
@gboone

gboone May 19, 2016

Contributor

Woah, interesting! I don't know that I would have guessed jekyll-archives was the culprit. Excellent sleuthing, I'll have to keep an eye on that PR.

Contributor

gboone commented May 19, 2016

Woah, interesting! I don't know that I would have guessed jekyll-archives was the culprit. Excellent sleuthing, I'll have to keep an eye on that PR.

@shawnbot

This comment has been minimized.

Show comment
Hide comment
@shawnbot

shawnbot May 27, 2016

Contributor

This is super, super interesting. Kudos, @toolness! We're having the same issue on EITI.

Contributor

shawnbot commented May 27, 2016

This is super, super interesting. Kudos, @toolness! We're having the same issue on EITI.

@gboone

This comment has been minimized.

Show comment
Hide comment
@gboone

gboone Sep 1, 2016

Contributor

This is partially addressed, for local builds, anyway, with the coming redesign, thanks for the discussion. We're watching jekyll/jekyll-archives#58 for a merge on this.

Contributor

gboone commented Sep 1, 2016

This is partially addressed, for local builds, anyway, with the coming redesign, thanks for the discussion. We're watching jekyll/jekyll-archives#58 for a merge on this.

@gboone gboone closed this Sep 1, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment