Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification about some metrics #1229

Open
sburnicki opened this issue Dec 12, 2018 · 3 comments

Comments

@sburnicki
Copy link
Contributor

commented Dec 12, 2018

Hey,

recently a bunch of questions came up about some metrics that I wasn't able to explain from any documentation or code, so I wanted to ask them here.

  1. I observed a big difference between document complete (or docTime in the XML/JSON export) and the actual load event start/end time on Android. In this example where document complete is 8.3s and load event is at 2.8s. On iPhone and Desktop these metrics match much better.

    Is this a bug? If not, what does docTime technically express?

  2. A few months back I looked where to get the interactivity metrics and I am pretty sure that I went for FirstInteractive and ConsistentlyInteractive in the JSON/XML data.
    Now most tests only have LastInteractive and TTIMeasurementEnd (like this or this), and I'm not quite sure how they relate to the old ones.

    I know that in Lighthouse First Interactive became CPU Idle (which I don't find either in the data) and Consistently Interactive basically became TTI. The WPT code defines First Interactive in the UI as FirstInteractive or LastInteractive.
    So what do these metrics in WPT really express and how do they relate to the terms that Loghthouse specify?

I'd be happy to document these things anywhere. Which is the desired format? Google Sites (is it even possible to contribute there?) or markdown in this repo?

Thanks!

@pmeenan

This comment has been minimized.

Copy link
Contributor

commented Dec 12, 2018

1 - Not necessarily a bug (usually not). docTime is measured from outside the page context from the start of navigation until the last onload is fired for the top-level frame. The "load event" metrics are measured from inside the page and is relative to the current context's navigation. The biggest reason they will diverge is if the page does something like a reload or javascript redirect. That time will not be included in the "load event" but docTime will include the full sequence.

2 - Missing FirstInteractive and ConsistentlyInteractive are usually when there isn't a 5-second window of idle in the main thread before the recording stopped so neither could be calculated. LastInteractive is the last time the thread went idle for > 50ms and is the earliest that either of the interactive metrics could possibly be assuming the main thread would have stayed idle for 5 seconds after the test stopped. TTIMeasurementEnd is basically the end time of the timeline data that we have recorded for calculating the interactive metrics.

The WPT UI will show "> xxx" when FirstInteractive isn't available and will use LastInteractive in it's place basically saying that it is at least that long.

Relative to lighthouse:

WPT's FirstInteractive === Lighthouse's First CPU Idle
WPT's ConsistentlyInteractive === Lighthouse's Time To Interactive

@omares

This comment has been minimized.

Copy link

commented May 29, 2019

Hey @pmeenan,

i hope that my question fits this issue.

Suddenly some of our WebPagetest runs that are trigged via the API are missing the Interactive metrics like FirstInteractive (e.g. a run from 29/05/2019, 05:03:37). Is that due to the mentioned lack of idle time? Unfortunately i am out of ideas on how to debunk the problem and would be really grateful on any tips.

Is there maybe a setting that we could apply to increase the wait time?

@nilskuhn

This comment has been minimized.

Copy link
Contributor

commented Jul 9, 2019

Hi Pat,

I have some further questions about interactive metrics in WPT:

WPT's FirstInteractive === Lighthouse's First CPU Idle
WPT's ConsistentlyInteractive === Lighthouse's Time To Interactive

Wouldn't it make sense to use the same names as in lighthouse to avoid confusion (at least in the GUI)?

Why first interactive / first CPU idle is shown in UI, but last interactive / TTI is not? Wouldn't it make sense to always show first CPU idle column (following new naming scheme) like it is now (with the FirstInteractive value if available and "> LastInteractive value" if FirstInteractive not available)? And additionally show a column TTI with LastInteractive if available.

Sometimes, in addition to FirstInteractive and LastInteractive there is a metric TimeToInteractive in WPT's result xml/json (see third step here, where its value is the same as FirstInteractive and LastInteractive). Is this just a duplicate / relic or another metric? Do we need that one anymore?

Thanks in advance, Nils

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.