New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Front Matter defaults performance regression #6691

Closed
mmistakes opened this Issue Jan 10, 2018 · 9 comments

Comments

Projects
None yet
6 participants
@mmistakes

mmistakes commented Jan 10, 2018

  • I believe this to be a bug, not a question about using Jekyll.
  • I updated to the latest Jekyll (or) if on GitHub Pages to the latest github-pages
  • I ran jekyll doctor to check my configuration
  • I read the CONTRIBUTION file at https://jekyllrb.com/docs/contributing/
  • This is a feature request.

  • I am on (or have tested on) macOS 10+
  • I am on (or have tested on) Debian/Ubuntu GNU/Linux
  • I am on (or have tested on) Fedora GNU/Linux
  • I am on (or have tested on) Arch GNU/Linux
  • I am on (or have tested on) Other GNU/Linux
  • I am on (or have tested on) Windows 7+

  • I was trying to install.
  • There is a broken Plugin API.
  • I had an error on GitHub Pages, and I have reproduced it locally.
  • I had an error on GitHub Pages, and GitHub Support said it was a Jekyll Bug.
  • I had an error on GitHub Pages and I did not test it locally.
  • I was trying to build.
  • It was another bug.

My Reproduction Steps

  1. Clone test repo https://github.com/mmistakes/front-matter-defaults.git
  2. Run bundle update to install Jekyll 3.6.2
  3. Run bundle exec jekyll build, and take note of build time ~15 seconds.
  4. Change gem "jekyll", "~> 3.6.2" to 3.7.0 in Gemfile.
  5. Run bundle update to install latest Jekyll version.
  6. Run bundle exec jekyll build again, take note of increased build time of ~40 seconds.

The Output I Wanted

For build time not to double. I noticed this on my rather large Jekyll site when updating to 3.7.0. Build time was usually around two minutes, but now it's unmanageable.

Took me awhile to track down what was the bottleneck. When running Jekyll with --verbose I noticed that during "Generating" each line of Reading: _posts/xxxx-xx-xx-whatever.md were taking much longer, which led me to lib/jekyll/frontmatter_defaults.rb.

When I disabled all of my Front Matter defaults in _config.yml everything sped right up to pre 3.7.0 speeds. But obviously my pages don't build because I'm assigning layouts using defaults.

@ashmaroli

This comment has been minimized.

Member

ashmaroli commented Jan 10, 2018

Related to #6268 as it introduced Dir.glob which is a known Ruby bottleneck

@ashmaroli

This comment has been minimized.

Member

ashmaroli commented Jan 10, 2018

@parkr

This comment has been minimized.

Member

parkr commented Jan 10, 2018

Good catch! Let’s get this fixed up.

@Crunch09

This comment has been minimized.

Member

Crunch09 commented Jan 10, 2018

@mmistakes Thanks for flagging! This certainly doesn't make it any better and we need to fix it but may i ask what ruby version you were using?
I tried it with 2.4.2 (bump from 12.1 to 16.5 seconds) and 2.5.0 (bump from 11.7 to 16.5) so i'm wondering why you are experiencing a much larger regression?

@mmistakes

This comment has been minimized.

mmistakes commented Jan 11, 2018

@Crunch09 The original numbers above were 2.4.1 on Windows 7.

Windows 7 3.6.2 3.7.0
15s 40s

I just tested on macOS ruby 2.4.1 and the numbers are better, which in my experience is usually the case. Windows generally lags behind for some reason.

macOS 3.6.2 3.7.0
10.9s 16.1s

40s isn't that bad, but on my real repo where I have considerably more posts and use jekyll-paginate-v2 (which gets hit pretty hard from this regression), my build time got so bad that I gave up after 20 minutes.

@mattr-

This comment has been minimized.

Member

mattr- commented Jan 11, 2018

@ashmaroli i'm going to assign this to you since you wrote the PR so that I can take it out of my queue, since I probably won't get to my jekyll queue for several weeks.

@mattr- mattr- assigned ashmaroli and unassigned mattr- Jan 11, 2018

@ashmaroli

This comment has been minimized.

Member

ashmaroli commented Sep 19, 2018

👋 @mmistakes the master branch has just been updated with a patch to optimize the handling of Front Matter Defaults.
Since you were the first to report the performance issue and provide a test-repo, I felt that it's only right that you should know about a resolution, first-hand..

Will you be able to test the latest master and provide feedback..? If not, no worries..
Thanks in advance. 😄

@mmistakes

This comment has been minimized.

mmistakes commented Sep 19, 2018

Thanks @ashmaroli. Updated to master and built my large personal site and your optimizations trimmed a nice chunk of time off of the build.

Windows 7 3.8.4 master
Cold start 182.449s 143.359s
On regeneration 180.326s 144.468s
@ashmaroli

This comment has been minimized.

Member

ashmaroli commented Sep 19, 2018

Yay!! 🎉
Thank you very much for responding. I may be able to squeeze in some more optimizations on the FMD by the time 4.0 ships.. 🤞

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment