Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(redux): cache truePath lookups for performance #12693

Conversation

bennetthardwick
Copy link
Contributor

Description

When creating 50k pages, Gatsby spends a lot of time inside the truePath method (which down the stack reads every directory on each of the patch segments). In an application with a small number of components (2 or so), this method doesn't need to be run 50k times.

To fix this, a small Map was added to cache these calls, instead of running them every time.

Using this patch with gatsby-intense-benchmark, the createPages step of the build goes from about 60 seconds to 20 - 30 seconds.

@bennetthardwick bennetthardwick requested a review from a team as a code owner March 20, 2019 13:01
Copy link
Contributor

@sidharthachatterjee sidharthachatterjee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @bennetthardwick

Looks good overall, left a small comment!

@@ -86,6 +86,7 @@ const pascalCase = _.flow(
)
const hasWarnedForPageComponentInvalidContext = new Set()
const hasWarnedForPageComponentInvalidCasing = new Set()
const pageComponentCache = new Map()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Want to replace this with a WeakMap? WeakMaps can be cleaned up by GC so great for caches

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great idea! We should also see if this approach improves performance in other areas where we're using Map (where it makes sense to use a WeakMap!).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately according to MDN primitive values can't be as keys for weak maps, only object references.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This map won't ever be huge (as this just map template paths), so I don't worry about memory/gc here

Copy link
Contributor

@DSchau DSchau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick nit: re a typo, otherwise this looks good.

With your comment on WeakMap only supporting objects, we could just use an object here--or we can just keep the original Map.

I'm not sure there'd be a hugely meaningful difference.

packages/gatsby/src/redux/actions.js Outdated Show resolved Hide resolved
packages/gatsby/src/redux/actions.js Outdated Show resolved Hide resolved
DSchau and others added 2 commits March 22, 2019 08:03
Co-Authored-By: bennetthardwick <me@bennetthardwick.com>
Co-Authored-By: bennetthardwick <me@bennetthardwick.com>
@bennetthardwick
Copy link
Contributor Author

Quick nit: re a typo, otherwise this looks good.

Thanks @DSchau! I can't believe I missed those.

With your comment on WeakMap only supporting objects, we could just use an object here--or we can just keep the original Map.

I'm not sure there'd be a hugely meaningful difference.

Reading more in to Maps, it seems the difference between them and objects is that the keys are kept in the order they were inserted. Since this isn't needed it's probably unneeded overhead (albeit marginal). I'll update it to use an object instead. 👍

@sidharthachatterjee sidharthachatterjee merged commit d5c3351 into gatsbyjs:master Mar 22, 2019
@gatsbot
Copy link

gatsbot bot commented Mar 22, 2019

Holy buckets, @bennetthardwick — we just merged your PR to Gatsby! 💪💜

Gatsby is built by awesome people like you. Let us say “thanks” in two ways:

  1. We’d like to send you some Gatsby swag. As a token of our appreciation, you can go to the Gatsby Swag Store and log in with your GitHub account to get a coupon code good for one free piece of swag. (Currently we’ve got a couple t-shirts available, plus some socks that are really razzing our berries right now.)
  2. We just invited you to join the Gatsby organization on GitHub. This will add you to our team of maintainers. Accept the invite by visiting https://github.com/orgs/gatsbyjs/invitation. By joining the team, you’ll be able to label issues, review pull requests, and merge approved pull requests.

If there’s anything we can do to help, please don’t hesitate to reach out to us: tweet at @gatsbyjs and we’ll come a-runnin’.

Thanks again!

@bennetthardwick bennetthardwick deleted the feature/cache-component-lookups branch March 24, 2019 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants