Entire site is rebuilt when making changes

[toolness]

I noticed that the entire site seems to be rebuilt whenever I make changes to a file. For instance, the entire site takes 20 seconds to build when I start ./go serve; yet the following actions seem to take another 20 seconds to regenerate the site:

• Making a minor edit to a SASS file;
• Making a minor edit to a Markdown post.

This is surprising, since ./go serve runs jekyll with incremental build enabled. I did notice that config.yml contains the line full_rebuild: true, but commenting it out doesn't seem to change anything.

I can reduce the time to about 8 seconds by running jekyll serve --incremental --limit_posts 5, but I'm not sure where that 8 seconds comes from. Will do some profiling to investigate more!

[toolness]

Ok, I did some more research/tinkering and here are my findings:

• The actual rendering of liquid templates doesn't seem to take that long, when using jekyll's --profile option.
• Deleting everything in _team saves about 3 seconds of build time.
• Deleting everything in pages saves about 1 second of build time.
• Deleting everything in assets saves about 3.5 seconds of build time.

So deleting all those is apparently what it takes to get the incremental build time down to a reasonable level of about one second. Unfortunately, it also results in a website with no front page, no CSS, and 5 blog posts.

It's particularly weird to me that processing everything in assets takes 3.5 seconds on an incremental build. Because that's standard jekyll functionality, I'd think that they'd have implemented logic to ignore doing anything with that folder on an incremental build unless something in there has actually changed.

[toolness]

Ok, so I've discovered that something is silently disabling incremental regeneration. This can be shown by creating a _plugins/boop.rb file and putting a hook in it:

require 'jekyll'

Jekyll::Hooks.register :site, :after_reset do |site|
  puts "site.regenerator.disabled? == #{site.regenerator.disabled?}"
end

On a fresh Jekyll site, running jekyll serve --incremental will print out something like this:

 Incremental build: enabled
      Generating...
site.regenerator.disabled? == false

However, on 18f.gsa.gov, it will print out:

 Incremental build: enabled
      Generating...
site.regenerator.disabled? == true

That's not a typo--it actually says that incremental builds are enabled, but then something disables it at a later point.

Next step is to figure out what's disabling it!

[toolness]

Ok, I was quite confused because there appeared to be no way to disable the incremental regenerator from outside the class, but jekyll-archives found a way:

Jekyll::Hooks.register :site, :after_reset do |site|
  # We need to disable incremental regen for Archives to generate with the
  # correct content
  site.regenerator.instance_variable_set(:@disabled, true)
end

I'd issue a PR to at least display a warning that this is happening, as it would've saved me a lot of debugging time, but apparently there's a fix nearing completion at jekyll/jekyll-archives#58, so that's good at least.

In the meantime, incremental regeneration can be enabled during development by temporarily commenting-out the jekyll-archives gem in the Gemfile. This seems to significantly speed things up on my machine.

[gboone]

Woah, interesting! I don't know that I would have guessed jekyll-archives was the culprit. Excellent sleuthing, I'll have to keep an eye on that PR.

[shawnbot]

This is super, super interesting. Kudos, @toolness! We're having the same issue on EITI.

[gboone]

This is partially addressed, for local builds, anyway, with the coming redesign, thanks for the discussion. We're watching jekyll/jekyll-archives#58 for a merge on this.