Optimize Scope Globbing in Front Matter Defaults

[ashmaroli]

Stubbing this out to a dedicated ticket in order to give this issue some priority for Jekyll 4.0

Globbing a scope path is currently unusable on moderate-large collections due to the steep increase in build times.

PR(s) that will have a positive impact on resolving this issue:

• Cache matched defaults sets for given parameters #6888 - Merged
• Check if scope applies to type before given path #7263 - Merged
• Cache globbed paths in front matter defaults #7345 - Merged

/cc @cameronmcefee

[ashmaroli]

@cameronmcefee Please check if your site builds satisfactorily with the latest master branch.
If it doesn't, we'll try and optimize a bit more.. 😃

[cameronmcefee]

Thanks @ashmaroli! I'll try to figure this out on my own, but I'm not familiar with how to build a jekyll site from the jekyll source. If there is documentation for that please send it my way.

[ashmaroli]

Its pretty simple actually..

1. Create a Gemfile if there isn't one in your site already..
2. Point to the jekyll repository via the Gemfile.

# Gemfile
source "https://rubygems.org"

gem "jekyll", :git =>"https://github.com/jekyll/jekyll.git"

# any other gems your site may need during the build process
[...]

3. Activate Jekyll via Bundler

$ bundle exec jekyll build --trace

[cameronmcefee]

Nice, that's a pretty solid improvement. On master I don't see much of a difference in glob vs non-glob.

A simple test with a glob that only matches one file:

3.8.2 (whole site/incremental)
Globbing: 38.655/3.94
No Glob: 27.382/1.18

Master (whole site/incremental)
Globbing: 27.36/1.17
No Glob: 27.07/1.2

A complex test with a glob that matches multiple depths of files/folders:

3.8.2 Globbing: 57.7
Master Globbing: 21.515

I'd call this quite satisfactory.

[ashmaroli]

Thank you for getting back @cameronmcefee
I wasn't aiming for much of a difference between "globbing" and "non-globbing", but I see that's open for optimization as well..

But I'm happy with the results from globbing with v3.8.2 vs globbing with master..

[cameronmcefee]

Ah, sorry, I was mostly including that for myself, as the difference was significant enough that I was weighing whether to make a separate collection that I'd join to the other rather than keep it all together.

[ashmaroli]

No worries, @cameronmcefee
There should be a perceivable difference when you glob vs when you don't glob at all.. So that means, the gain is being swallowed somewhere..

[rubs hands]

[cameronmcefee]

If you'd like to test with our repo, I can have a branch with the complex test mentioned above later this morning.

[cameronmcefee]

Actually, for the glob/non-glob test, you can work from master here (currently non-globbing) and revert this commit to enable the globbing test I did above

[ashmaroli]

Thank you very much Cameron. I'll use it to do some tests. 👍

[ashmaroli]

Optimized satisfactorily..