You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I took a good look at the build process and managed to get the build time to be significantly less. On my local machine, it went from 13 minutes to 3.5 minutes, and Netlify builds are now 10-20 minutes depending on the cache. There are some significant changes that I'll list below:
New URL structure. The most apparent change is that I had to change the URL structure for filtering topics and languages. The URL's went from .../topic:bla+lang:bla to .../topic/bla/lang/bla. The reason for this is that Gatsby will created static JSON files based on the path on the computer, and win/mac computers would have problems with the +, : and lower-upper caps in the filenames. Therefore I changed the way page generation works so I could slugify the paths, which means that .../topic:JavaScript is now .../.topic/javascript. All in all, it shouldn't be a major change except for the fact that it is not allowed to have two separate tags with the same characters in diffferent upper and lower case combinations. I went through the code and changed all the apparent examples of this (processing vs. Processing and javascript vs. Javascript vs. JavaScript).
Optimize GraphQL queries. I changed some GraphQL queries and removed some unused GraphQL queries to significantly cut down the build time.
Remove duplicated pages. The path naming problems also resulted in duplicated pages, which means that the build process will now produce 1500 pages less.
There's a few more things that I'll tackle in upcoming PR's:
Turn homepage.svg file into a PNG since it's 1.6MB raw SVG with base64 in it
Check resizing of images to make sure everything happens correctly
Redesign the topic and language selector
Also @shiffman: Can you check whether this PR solves #367? I have the feeling that some of the page duplication logic could have resulted in this problem, so let me know if it looks better with this PR.
@shiffman After a few hours debugging the GraphQL queries, I've managed to bring the Netlify build time down to 20 minutes without cache and 12 minutes with cache! I'm still working through a few other things that might bring this down even more, so I'll report here when I have an update.
@shiffman Updated the PR message above and removed the WIP tag. Please give it a good test to make sure everything works and merge if it looks alright. The build on Netlify is now 13 minutes without cache 🥳
Wow! This is amazing @runemadsen! I've tested the build locally and checked the preview and all looks good. I'm seeing all the passenger showcase projects but I don't think we'll know if #367 is fully resolved until we have some activity over the course of the week of new submissions and multiple builds. @klinegareth let's keep a close eye and see! I'm going to go ahead and merge this so we can see how it performs. Thank you @runemadsen!!!!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I took a good look at the build process and managed to get the build time to be significantly less. On my local machine, it went from 13 minutes to 3.5 minutes, and Netlify builds are now 10-20 minutes depending on the cache. There are some significant changes that I'll list below:
.../topic:bla+lang:blato.../topic/bla/lang/bla. The reason for this is that Gatsby will created static JSON files based on the path on the computer, and win/mac computers would have problems with the+,:and lower-upper caps in the filenames. Therefore I changed the way page generation works so I could slugify the paths, which means that.../topic:JavaScriptis now.../.topic/javascript. All in all, it shouldn't be a major change except for the fact that it is not allowed to have two separate tags with the same characters in diffferent upper and lower case combinations. I went through the code and changed all the apparent examples of this (processingvs.Processingandjavascriptvs.Javascriptvs.JavaScript).There's a few more things that I'll tackle in upcoming PR's:
Also @shiffman: Can you check whether this PR solves #367? I have the feeling that some of the page duplication logic could have resulted in this problem, so let me know if it looks better with this PR.
This fixes #264