Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Recompile site time got worse between 4.1.6 and 4.2.3 #901

Closed
agross opened this Issue Jul 6, 2016 · 11 comments

Comments

Projects
None yet
2 participants
@agross
Copy link
Contributor

agross commented Jul 6, 2016

Hi,

our Nanoc site takes about 90 seconds to compile from scratch with Nanoc 4.2.3. Rebuilding the site without changing any files on disk performance changed rather dramatically between the latest versions of Nanoc. Here are a couple of timings created on a Ubuntu VM using Ruby 2.3.1. The only thing that changed during these runs is the Nanoc version.

Version Compile Recompile
4.1.6 56.54s 6.01s
4.2.0 57.62s 19.64s
4.2.1 57.87s 79.55s
4.2.2 58.46s 79.44s
4.2.3 86.40s 80.25s

I wonder how we should go about this. If you want I can get you access to the code, or I can create kcachegrind outputs using ruby-prof while running the recompile. I previsouly had some success with it stripping off 30 seconds by changing a single line of code.

Let me know what you think!

@agross agross changed the title Recompile site time got worse Recompile site time got worse between 4.1.6 and 4.2.3 Jul 6, 2016

@ddfreyne

This comment has been minimized.

Copy link
Member

ddfreyne commented Jul 9, 2016

I’d definitely be interested in profile information! This comment details an approach with Stackprof, which I found to be quite useful.

@ddfreyne

This comment has been minimized.

Copy link
Member

ddfreyne commented Jul 9, 2016

(Access to the site source would also be quite useful.)

@ddfreyne

This comment has been minimized.

Copy link
Member

ddfreyne commented Jul 9, 2016

Can you see whether #902 improves compilation times?

@agross

This comment has been minimized.

Copy link
Contributor Author

agross commented Jul 10, 2016

And we're back at 6 seconds recompile time :-)

@ddfreyne ddfreyne added this to the 4.2.4 milestone Jul 10, 2016

@ddfreyne

This comment has been minimized.

Copy link
Member

ddfreyne commented Jul 10, 2016

Excellent. Weak refs are far less useful than soft refs, as it appears!

@agross

This comment has been minimized.

Copy link
Contributor Author

agross commented Jul 10, 2016

👍 Thanks for finding the right fix :-)

@ddfreyne

This comment has been minimized.

Copy link
Member

ddfreyne commented Jul 10, 2016

FYI, Nanoc 4.2.4 will have a hidden feature, enabled by setting NANOC_FEATURE_PROFILER=1 in the env, which adds a --profile option to the compile command, writing a profile to tmp/stackprof_profile using StackProf. This’ll make it easier to debug slowdowns.

@agross

This comment has been minimized.

Copy link
Contributor Author

agross commented Jul 10, 2016

Excellent, that sounds great! I haven't looked at stackprof, so just FYI: ruby-prof's cachegrind output is really useable. You can load it into kcachegrind and browse call stacks visually.

@ddfreyne

This comment has been minimized.

Copy link
Member

ddfreyne commented Jul 10, 2016

I haven’t looked into ruby-prof in a while. Last time I looked, it wasn’t compatible with recent Rubies.

@agross

This comment has been minimized.

Copy link
Contributor Author

agross commented Jul 10, 2016

I only started using ruby-prof two weeks ago so things might have changed. While text reporting is nice for things like GH issues, I really like that there is a visual frontend with drilldown functionality: https://kcachegrind.github.io/html/Screenshots.html

Just as a suggestion when text/graphviz doesn't cut it. I at last was only able to find the issue using a UI.

In case you're interested, this diff cut 30 secs off our build:

        def nesting_level(item = @item)
-         item.identifier.components.count
+         item.identifier.to_s.count('/')
        end
@agross

This comment has been minimized.

Copy link
Contributor Author

agross commented Jul 25, 2016

FYI, Nanoc 4.2.4 will have a hidden feature

FYI, stackprof unfortunately doesn't work on Windows. tmm1/stackprof#45

It's not an issue for me as I have a Linux VM handy, but Windows-only users might suffer from it in the future in case you tell them to submit profiling information.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.