-
Notifications
You must be signed in to change notification settings - Fork 10.3k
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Worse performance results with Lighthouse v6 (?) #24332
Comments
It looks like Lighthouse 6 introduces some new metrics and removes some others from v5 so a change in score is certainly likely. This article explains what has changed: https://web.dev/lighthouse-whats-new-6.0/ There's also a link at the end to a score calculator which is really useful for breaking down the score and understanding what factors are contributing the most. https://googlechrome.github.io/lighthouse/scorecalc I get the impression there's more focus on main thread interactivity in v6 so if your site includes large JS bundles thats probably a significant factor. |
Yes @shanekenney , I'm aware, but don't really know how to reduce it apart from removing parts of the site to see what parts are provoking this Do you also see the impact on gaysbyjs and theme-ui sites? I'm curious / would love to know what optimizations on their site they may be thinking about, or if they have spotted some specific cause |
This issue is great so we can discuss overall Lighthouse / PageSpeed insights scores and the possible regressions we're seeing with v6. @kuworking one thing worth noting is that lighthouse-metrics.com seems to use "Emulated Nexus 5X" for 5.6 and "Emulated Moto G4" for 6.0 which could also add some noise to the comparison. This benchmark over 922 sites claims in v5 the median Performance score for a Gatsby site is 75. I'll try to do a quick view using hosted solutions to prevent my local network from being yet another variability factor. Currently (with Lighthouse v5.6 / PageSpeed Insights)PSI runs on a Moto G4 on "Fast 3G". Source Certain "flag" sites built using Gatsby are not really performing great on PageSpeed Insights (which is still using Lighthouse v5.6 I assume – subject to standard variability on every run, possibly running 3x or 5x and averaging would weight in more reliable metrics).
However some other sites (and most starters) are performing very well on PageSpeed Insights:
The average TTI is noticeable – and while v6 changes the overall weight of TTI from 33% to 15% and dropped First CPU Idle, it does add TBT with a weight of 25% which could possibly explain a regression of scores generally speaking just due to overall JS parsing and execution. Lighthouse v6 (with WebPageTest.org)
Here are the results, you can see the Lighthouse report by clicking on its number. I'm extracting the values from that report.
However, notice the regression on the following two sites:
Some of the open questions I have.
(I'm using the links above just as examples – if anyone would like a certain link removed I can gladly edit the message) |
My site (https://matthewmiller.dev/) appears to have gotten better (~92 to ~95), but some of the new tests reveal a few things that could probably be improved. The unnecessary JavaScript test for example, |
To me I'm having big difficulties in understanding I can see that something similar seems to happen in the starter https://parmsang.github.io/gatsby-starter-ecommerce/ |
As an update – seems that PageSpeed Insights has soft launched the update to run Lighthouse v6 (it may not be in all regions yet). Link to test https://gatsbyjs.org/. Currently getting results varying from low 60s to mid 80s, mainly depending on the values of TBT and TTI. |
@kuworking there might be an issue with Lighthouse v6 recognizing gatsby-image. According to webdev
In my case, I think Lighthouse isn't respecting the view size. And here's the image in question It might be accounting for the intrinsic size which is 3000 pixels hence the 13s LCP for me. |
@daydream05 I had similar theories and findings as well so I tested my pages without images and still had a crazy long LCP (10-12sec). I have a lot going on in my project so it could be other variables as well, but I'm curious if you've tested a page with text content and no images yet. |
Dropped from a 100 to 79~ https://dougsilkstone.com/ recently. Jumps up to 90when Google Tag Manager (and Google Analytics) are removed. Will report back on more findings as I test things. Edit: Hit 100 when removing typekit loaded font from gatsby-plugin-web-font-loader (also using preload-fonts cache). |
GTM is overall affecting my project a chunk but it isn't that drastic of a change when I remove it (5-10 points tops on sub 50s scores after LH6). I still need to do more testing but just wanted to throw that out there. |
@Jimmydalecleveland interesting! I also have another site where the is screen i just text and it’s blaming the hero text as the main cause for LCP. And LCP only accounts for whatever is in view, which doesn’t make sense. How can be a text be that big of a problem. @dougwithseismic I also use typekit and it’s def one of the major culprits for lower lighthouse scores. I wish there was a way to fix typekit since they dont support font-display I think Lighthouse v6 is really tough on JS frameworks because on how they changed weighting of the scores. (More focus on blocking time) And Gatsby sites have historically low script evaluation/main thread scores due to rehydration and other things. |
@dougwithseismic how did you link typekit font without using the stylesheet? |
I am having a similar experience, with lighthouse 5.7.1 my performance score was about 91, however lighthouse 6 has dramatically dropped my performance score to about 60. |
I don't even have these plugins installed, but my mobile score dropped from 90+ to 60 ~ 70+ |
Same here. Massive drop from 90ish to 60ish on multiple sites. |
+1 drop of about 30+ points |
Is anyone addressing this? Seems like there is no point using Gatsby over Next if it doesn't deliver better scores out-the-box. |
Do you have any numbers from Next? I am wondering whether these scores are the new normal for fast webs (that are not static, JS-free and likely also image-free) |
https://nextjs.org/ has a score of 85, with both Largest Contentful Paint (3.8) and First Contentful Paint (2.8) being the main offenders. It also has a bunch of "Unused JS". That's down from ~95 in Lighthouse 5.7.1. It's "only" a drop of around 10 points, whereas gatsby sites seem to lose twice as many points. I'm quite new to this world, but I'm following this issue after my gatsby site lost around 25 points when tested with lighthouse 6.0.0 from npm. Interestingly, if I'm using the pagespeed insights rather than npm lighthouse, my site goes back to around ~99. Whereas gatsbyjs.org gets ~70 on pagespeed insights, but ~84 with npm lighthouse. Something is probably being tweaked somewhere, I guess? All of them are getting 'Unused JS' warnings tho |
I see better scores with lighthouse-metrics https://lighthouse-metrics.com/one-time-tests/5edfbbb1cf858500080125f7 But I also don't see images there, and in my experience images seem to have a high (and legitimate IMO) impact Yet, gatsbyjs.org neither has images and its score is (relatively) bad https://lighthouse-metrics.com/one-time-tests/5edfbc58cf858500080125ff (as compared with @cbdp example) Let's see what gatsby devs think about this |
With a few tweaks, site is back to top scores.
It seems to me like a case of Google moving the goal posts to be more
strict about performance- notably FCP.
Our sites aren't slow by any means, moreso just being judged with different
criteria. I'll help out on this one ✌️
…On Tue, 9 Jun 2020, 18:48 kuworking, ***@***.***> wrote:
Is anyone addressing this? Seems like there is no point using Gatsby over
Next if it doesn't deliver better scores out-the-box.
Do you have any numbers from Next?
I am wondering whether these scores are the new normal for fast webs (that
are not static, JS-free and likely also image-free)
A Next.js website -> https://masteringnextjs.com/ 77 mobile score. A lot
of "Unused JS".
I see better scores with lighthouse-metrics
https://lighthouse-metrics.com/one-time-tests/5edfbbb1cf858500080125f7
But I also don't see images there, and in my experience images seem to
have a high (and legitimate IMO) impact
Yet, gatsbyjs.org neither has images and its score is (relatively) bad
https://lighthouse-metrics.com/one-time-tests/5edfbc58cf858500080125ff
(as compared with @cbdp <https://github.com/cbdp> example)
Let's see what gatsby devs think about this
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
<#24332 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ALSIKRH4G74CRN2FNCUO4NDRVZRVNANCNFSM4NHP7XCA>
.
|
Just wanted to drop this useful calculator for comparing results from v6 with v5: https://googlechrome.github.io/lighthouse/scorecalc/ Lighthouse scores generally vary a lot, even when using it through PageSpeed Insights. For example, for https://www.gatsbyjs.org/ I received everything from 64 to 88 mobile performance after 5 runs. Hence, for tracking down this issue the calculator might be useful to see the consequences of different weights on the same run (note: as metrics are a little different, some values like FMP must be assumed using former measurements). PS: Here I have a comparison of two runs from PageSpeed Insights for gatsbyjs.org: |
Yep, to add to @Pyrax: LCP (Largest Contentful Paint) and TBT weigh 25% each in Lighthouse v6. So we focussed our efforts on addressing those. We found: LCP
TBT
|
@Jimmydalecleveland Thanks Jimmy! Replacing
|
@abdullahe We've checked it out before and it has a dependency on canvas and node-canvas isn't super reliable. Or at least it wasn't in the past. I'll let you know if we consider it again :) @guydumais make sure to put @BenjaminSnoha & @davidpaulsson I'll share an example in a bit. The biggest issue with art-direction if the aspect ratio between images change, like horizontal to vertical. |
@wardpeet how would one use |
while this might not have anything to do with lighthouse, I am wondering why gatsby page data JSON files do not support content hashing, just like js files. I know that the content hashing for js files is performed by Webpack, but gatsby can also do the same for page data JSON files. this can save a lot cdn network requests |
@teclone page-data.json files shouldn't be downloaded over and over if your caching is setup correctly. They'll load once and then the browser revalidates them. The problem with content hashing for page data (vs JS/CSS files) is just that there's so many of them. With content hashing, before you can load a file, you need to load a manifest that translates from If you do see page-data.json files downloaded over and over — check you haven't disabled caching in your devtools & check your caching headers with https://www.npmjs.com/package/check-gatsby-caching |
@KyleAMathews , thanks for clarifying that. That is a very sensible approach |
@wardpeet is it true that image-nextgen does not support It works a lot better though, went from ~80 to ~95 |
@MelleNi it does not, I don't think it's necessary but we're happy to consider it. you can have a look at #27950 to see what we're shipping.
We're going to move remark to this plugin as well :) |
Great to hear about remark, as I saw a lot of improvement in speed. However, I noticed I could not get 99-100 without disabling Gatsby's javascript (and re-integrating particular functionality manually). I can get the old Gatsby image to work without javascript, using I understand that disabling javascript kind of defeats the idea of Gatsby, but oh well. |
Interestingly, I saw an improvement on mobile performance score (~70 to ~90) when I stopped using self-hosted fonts (fontsource) and switchted to "system" fonts. |
@wardpeet Any chance you can share an example of how to build a composable image component with art direction? I'm in the same boat as @BenjaminSnoha & @davidpaulsson, and I don't mind creating the composable component in my own project. The biggest issue I see is dealing with media queries and SSR. Libraries such as fresnel work, but suffer in performance because they load all components, then clean up the DOM after the |
On my website it seems that all pages created with createPage have the source code (markdown and markdown react components inside markdown) in the pagespeed heavy javascript (Remove unused JavaScript) |
I've just launched Plaiceholder which can be used to create pure CSS blurred placeholders. Perhaps this would be of interest? More than happy to chat with any of the core team about options forwards |
This comment has been minimized.
This comment has been minimized.
@styxlab I get slightly lower results in web.dev/measure but better in post results, definitively very good values in any case and markedly different from gatsby version https://demo.jamify.org/ I will also post that in one site I've changed (gatsby)(different design, essentially the same content, 11ty)Or in a similar page, this time with an image (gatsby)(different design, essentially the same content, 11ty)I will say that the 11ty developer experience is very nice (you can also --experimentally use While I was using 11ty, I was also thinking how nice would it be that gatsby enabled some sort of 11ty render strategy so that one could deploy mixed react and reactless static pages in one framework ... |
any updates on this? I dont have any images and I get 76 on performance because of Total Blocking Time |
@devquestionz site url ? |
@Kvintus we removed all images and svg in homepage https://deriv-com-git-fork-mahdiaryayi-fs-test-imageless.binary.sx/, still get 65 in lighthouse score |
@mustofa-binary first and foremost, you should start by getting your Cumulative Layout Shift score as close to zero as you possibly can before you work on any other optimizations. Layout shifts can dramatically increase load times and contribute to bad user experience. I've seen scores jump 20-30 points from CLS fixes alone. You can use the Layout Shift GIF Generator to help you visualize layout shifts above the fold. I see Gatsby is preloading critical links for you already but I could not find any preloads for your fonts. This can significantly speed up contentful paint scores. You can add something to the effect of this to
One of the quickest ways you can reduce total blocking time is by reducing your bundle size. I know this is not a convenient suggestion but it could help lower that 3,877 ms spent on Script Evaluation which would reduce total blocking time. You can either use chrome's network tab or other tools to analyze your bundle size. For example, Gatsby has a really cool visual tool that helps you narrow down where to focus your attention and cut unnecessary weight. Make sure you're not accidentally/indirectly importing libraries like lodash if you can use cheaper and lighter alternatives. On that note, for anyone that feels like they've exhausted their options, I suggest running your app in dev mode and see what you can find in the React Profiler. This could help identify any performance anomalies and unnecessary rerenders. I would also suggest cutting weight when you're testing performance bottlenecks. Removing images and svg's is a good start. From there, I would suggest migrating any external scripts, tracking and analytics into something like the Google Tag Manager. This will help you assess their impact. If you notice a significant difference there, I would suggest delaying tracking or analytics by an amount that fits your needs. You can do this in
You can read more about it here: Delay loading of Google Analytics / Google Tag Manager script for better PageSpeed score and initial load. When all of these things start to compound on one another, it can get a little overwhelming. The way I see it, performance is a moving target. Just like with security, it's not something we can set up once and forget about, no matter how much we may want to. I've gotten perfect scores on v6.0-v6.3 on multiple Gatsby projects so it is entirely possible with what we have available at the moment. Google is setting a new standard and it's going to take some investigative work on our part if we want to earn that "perfect score." At least for the time being. |
I am also a bit lost with my site: Any suggestions here? I really already tried out a lot with minor to zero improvements. |
We had the same issue and stopped hosting our own fonts and went with a hero pattern instead of a massive image.
First off...great site and it is fast. The Video hero is instant and the navigation is snappy so from a user's perspective it's a great website ,so maybe, if it's not killing your SEO, you shouldn't care what the Lighthouse score is. Focus more on the user. We are certain that your site is in the top 1% for speed....despite what lighthouse says. Who knows what they use to measure this stuff. Any way to answer your question, and you won't want to hear this...You have to lose the video hero. It's your LCP and why your TTI are yellow zone. Yep. We did it and went straight back to 100 across the board for mobile and DT. Good luck man. |
Thanks for this nice compliment - great to hear. We will think about removing the video but for now we keep it since the rest (despite the score) is working pretty well for us. |
My site https://mwskwong.com has its score stuck at around 80~ something. It seems that TBT is the biggest culprit for my site. I have already done code splitting on some of the heavy components (e.g. the radar chart and the contact form). I can't really think of any solution besides removing some of the features on my site. Do you guys have any suggestions? |
Since this issue turned more into a discussion and the image issues were addressed with the new |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Just wondering whether there is some information that could be of use here, since I've found in my sites a significant worsening of lighthouse results when comparing lighthouse v5.6 vs the new 6.0 (https://lighthouse-metrics.com/)
In a complex site (of mine) it goes (performance-wise) from a ~90 to ~50
In a simple starter (of mine) it lowers from ~98 to ~80
This doesn't happen in starters such as https://gatsby-starter-default-demo.netlify.app/ or https://gatsby-v2-perf.netlify.app/
But it does happen to www.gatsbyjs.org (from ~98 to ~73) or to https://theme-ui.com (from ~90 to ~75)
Since I spent some time achieving 98-100 punctuations in my code (which made me very happy), I kind of feel I don't have a lot of room for improvement (probably I do have), so I've thought I might ask here if there's something going on
Thanks
The text was updated successfully, but these errors were encountered: