-
Notifications
You must be signed in to change notification settings - Fork 113
Figure out perceived performance benchmarks #2005
Comments
https://www.instartlogic.com/blog/perceptual-speed-index-psi-measuring-above-fold-visual-performance-web-pages has relevant info, including a link to the VisualMetrics github repo, which has code to compute PSI from a video. @digitarald might also have thoughts here... |
webpagetest (we have a private instance) gives you Speed Index but also can show user timings. The gold standard for timings depends on you knowing how important each aspect of the content is. I would recommend adding timings for important content manually. Steve Souders has some good reference for this. Combine this with keeping the site interactive while streaming in the elements (webpagetest shows CPU and starts to add interactivity), as @Mardak said. Happy to chat more about specifics if needed :) |
In addition to the useful reading that @digitarald suggested, I also got a bunch out of reading LinkedIn's blog post. After a bunch of research, and ruminating, and poking around and discussion with Kate, I'm going to propose a set of user timings that I think are highest priority. In an ideal world, we'd do a bunch of user research around our existing implementation, but the SDK one is probably not the right starting point for performance stuff like this. So these measurements are in large part informed by what's easy to bootstrap/measure. In addition to replacing about:newtab, activity stream also replaces about:home. about:newtab always preloaded in a hidden context, and then made visible in a fresh tab. about:home: by default, is the first page that comes up at browser start in a new tab. This is effectively a normal render. about:home from the toolbar button (not shown by default): will make the current tab load activity stream. Tab-blanking followed by a normal render. After that, I'll also include an outline of stuff that we are likely to want to look at as time goes on. I'll break down the user-timings session into graduation issues soon, and then resolve this bug. Here are the first user timings that I propose we include; they should be useful in both development/profiling, synthetic regression testing, and Real User Monitoring (aka telemetry).
There is some perceptual stuff that I hope that we can get out of WebPageTest/Talos as well as performance.timing eventually, but I don't think that's the highest priority, since so much of that is focused on loading stuff over the network, and that's a set of issues we just don't have. maybe/later
|
I'm going to edit the above list in place as a checklist. |
I am unsure about what the aforementioned about:newtab/about:home measure, is it delay to first paint after input (probably tab opening or blank, depending on interaction). Maybe those can be tracked as what kind of response is expected:
For page load metrics, the good old
This assumes the page is usable after Hero Element, meaning that no lazy loaded component causes noticeable jank (rule of thumb is frames longer than 50ms):
|
In addition to replacing about:newtab, activity stream also replaces about:home. about:newtab always preloaded in a hidden context, and then made visible in a fresh tab. about:home: by default, is the first page that comes up at browser start in a new tab. This is effectively a normal render. about:home from the toolbar button (not shown by default): will make the current tab load activity stream. Tab-blanking followed by a normal render. The first two cases are clearly the most important. One thing all of these cases is that none of them (will, in the system-addon implementation) load any content directly from the network.
Nothing, that was a header indicating that I want simple page-load proxy measures for each of two loading contexts (I just realized the third one while writing this up, so I'll modify the plan :-).
Agreed; I mostly wanted to collect that to see where it shows up in the sequence compared to other events. That's probably not a good enough reason to collect it, especially given that it'll show up in the profiler when we profile.
As currently written, I was intending to just look at when the top-level React component had rendered and painted (equivalent to Display Done). My suspicion is that given that none of this stuff will be network loaded, and we're offloading more things to other threads, we'll be fast enough for most users that Display Done may show us that we don't need to break it down further.
We'll add that if we end up needing a placeholder in the system add-on version.
Yeah, I just checked our data, and topsites is the one. I'm thinking the best way to do it is using http://stackoverflow.com/a/34999925 .
We don't specifically know exactly what content is going to be above the fold. For that reason, this seems like enough work that I'm inclined to put it off until we see the numbers we get back from Display Done and Hero Element (Top Sites Painted). @digitarald I presumably it's hard to quantitatively monitor jank until we implement PerformanceFrameTiming. Or is there some other way to do that?
So it sounds like we want all event handles to have performance.mark bookending? |
Double rfa might be also an option here and is used by FB et al.
The workaround is having a "heartbeat" (continues 50ms timer, not having much perf impact). Using the skew of expected time and actual time you can tell if the main thread was blocked. Make sure to also include page visibility as factor as that can throttle timers. Somes sites started using this technique + meaningful paint to guess TTI.
That would be ideal. For more complex interactions you could even break it down into a R(A)IL measure to know how fast the first response paint happens and when the final state is painted. |
I've spun off a new bug to make plans for synthetic perf tests which references the discussion in this bug. I believe all the critical stuff from this bug has now been filed (and annotated in #2005 (comment) ), so I'm closing this one. |
Split from #1977. Determine "perceived performance" benchmarks (talk to philipp sackl). Mainly page loading to interactability of highlights/topsites. Probably good to have something around search.
The text was updated successfully, but these errors were encountered: