New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log if a page was ever visible #29

Open
n8schloss opened this Issue Mar 24, 2017 · 16 comments

Comments

Projects
None yet
4 participants
@n8schloss

n8schloss commented Mar 24, 2017

A common issue that we're seeing at Facebook is that a good number of pages are never visible while they are loading. Due to background tab throttling these loads are often slower, but that's fine because no one is looking at them. To ensure that we're correctly monitoring this, we'd love to track if pages are ever visible. Unfortunately there's a brief period of time before our JavaScript runs on the page and during this time we have no clue as to the page's visibility state.

To solve this, we'd love if currently hidden pages had a way to tell us if they were ever visible and if yes how long was it visible for. Thoughts?

@igrigorik

This comment has been minimized.

Show comment
Hide comment
@igrigorik

igrigorik Mar 31, 2017

Member

Trying to wrap my head around the use case..

  1. User clicks on a link -> stares at spinner -> switches to another tab -> ... -> comes back to page
  2. User cmd-clicks link -> page loads in background -> switches to page

You want to distinguish between 1 and 2? Practically speaking, if you execute visibility test as first thing on the page (i.e. in the head) with existing API.. Doesn't it give you all the information you need -- the page is hidden hence load is/was throttled. Why does it matter if the user was looking at the page earlier or not?

Member

igrigorik commented Mar 31, 2017

Trying to wrap my head around the use case..

  1. User clicks on a link -> stares at spinner -> switches to another tab -> ... -> comes back to page
  2. User cmd-clicks link -> page loads in background -> switches to page

You want to distinguish between 1 and 2? Practically speaking, if you execute visibility test as first thing on the page (i.e. in the head) with existing API.. Doesn't it give you all the information you need -- the page is hidden hence load is/was throttled. Why does it matter if the user was looking at the page earlier or not?

@n8schloss

This comment has been minimized.

Show comment
Hide comment
@n8schloss

n8schloss Apr 6, 2017

Yeah, that's exactly what we're trying to distinguish between. In case 1 we could have shown something quicker and in case 2 if the user isn't actively looking at the page we care about it less. We can totally put something in the head, but if the user is blocked on the network and stared at a spinner we'd still like to know that.

n8schloss commented Apr 6, 2017

Yeah, that's exactly what we're trying to distinguish between. In case 1 we could have shown something quicker and in case 2 if the user isn't actively looking at the page we care about it less. We can totally put something in the head, but if the user is blocked on the network and stared at a spinner we'd still like to know that.

@igrigorik

This comment has been minimized.

Show comment
Hide comment
@igrigorik

igrigorik Apr 10, 2017

Member

In case 1 we could have shown something quicker and in case 2 if the user isn't actively looking at the page we care about it less.

How so? Practically speaking you're stuck on the response bytes from the server.. I understand that it may be nice to know if the user was "waiting" for those bytes, but practically speaking there is nothing you can do to improve the speed here. If your first bytes query visibility you know your answer, right?

Member

igrigorik commented Apr 10, 2017

In case 1 we could have shown something quicker and in case 2 if the user isn't actively looking at the page we care about it less.

How so? Practically speaking you're stuck on the response bytes from the server.. I understand that it may be nice to know if the user was "waiting" for those bytes, but practically speaking there is nothing you can do to improve the speed here. If your first bytes query visibility you know your answer, right?

@n8schloss

This comment has been minimized.

Show comment
Hide comment
@n8schloss

n8schloss Apr 17, 2017

but practically speaking there is nothing you can do to improve the speed here

That's not true, it'd be a good indication that there are network issues and that we need to make improvements to CDN, network architecture, etc.

n8schloss commented Apr 17, 2017

but practically speaking there is nothing you can do to improve the speed here

That's not true, it'd be a good indication that there are network issues and that we need to make improvements to CDN, network architecture, etc.

@igrigorik

This comment has been minimized.

Show comment
Hide comment
@igrigorik

igrigorik Jul 23, 2017

Member

@n8schloss this came up in a different discussion today. Question for you...

One solution could be to resurrect ~prerenderSwitch concept in Nav Timing. Something like:

The visibilitySwitch attribute must return a DOMHighResTimeStamp with a time value equal to the first time when document visibility changes from "hidden".

Would that address your use case?

Member

igrigorik commented Jul 23, 2017

@n8schloss this came up in a different discussion today. Question for you...

One solution could be to resurrect ~prerenderSwitch concept in Nav Timing. Something like:

The visibilitySwitch attribute must return a DOMHighResTimeStamp with a time value equal to the first time when document visibility changes from "hidden".

Would that address your use case?

@n8schloss

This comment has been minimized.

Show comment
Hide comment
@n8schloss

n8schloss Jul 24, 2017

yep, that would totally address our use case :)

n8schloss commented Jul 24, 2017

yep, that would totally address our use case :)

@spanicker

This comment has been minimized.

Show comment
Hide comment
@spanicker

spanicker Jul 24, 2017

[copy-pasting my response from internal thread]

Re: what can be made available on Navigation Timing: we could expose a field indicating whether the page was initially hidden or visible (background nav vs foreground).
Beyond that not sure it makes sense to wait and see when the visibility changed:

  • how long would we wait - some finite length of time right? then it's not exactly "when" visibility changed but rather "whether visibility changed in the first M seconds of load" or something.
  • queueing NT entry should not be blocked on the above.

Given all this I'm not sure on the value of making this a timestamp, maybe a boolean on whether initial visibility was hidden or visible (background or foreground load)?

spanicker commented Jul 24, 2017

[copy-pasting my response from internal thread]

Re: what can be made available on Navigation Timing: we could expose a field indicating whether the page was initially hidden or visible (background nav vs foreground).
Beyond that not sure it makes sense to wait and see when the visibility changed:

  • how long would we wait - some finite length of time right? then it's not exactly "when" visibility changed but rather "whether visibility changed in the first M seconds of load" or something.
  • queueing NT entry should not be blocked on the above.

Given all this I'm not sure on the value of making this a timestamp, maybe a boolean on whether initial visibility was hidden or visible (background or foreground load)?

@igrigorik

This comment has been minimized.

Show comment
Hide comment
@igrigorik

igrigorik Jul 25, 2017

Member

Is initial state sufficient to cover the original use case?

It would solve the question for "was the page in background", but it wouldn't capture the case where user clicks on a link.. waits a while.. switches to a different page while navigation is loading, right? In this case the initial state is "visible" but then switches to "hidden", and if the page or network (or both) are slow enough, the user may even do this a few times -- I've certainly done it myself, where I toggle to check if the page has loaded a few times before the answer is yes.

If we wanted to solve the full case, it seems that we'd need to log all the visibilitychange events and make them available? E.g. one plausible solution could be: prior to end onload, emit a new ~VisibilityPerformanceEntry logging the state change with timestamp. The page can harvest these later to figure out if the user was toggling visibility and how long the page was in background? </brainstorm>

Member

igrigorik commented Jul 25, 2017

Is initial state sufficient to cover the original use case?

It would solve the question for "was the page in background", but it wouldn't capture the case where user clicks on a link.. waits a while.. switches to a different page while navigation is loading, right? In this case the initial state is "visible" but then switches to "hidden", and if the page or network (or both) are slow enough, the user may even do this a few times -- I've certainly done it myself, where I toggle to check if the page has loaded a few times before the answer is yes.

If we wanted to solve the full case, it seems that we'd need to log all the visibilitychange events and make them available? E.g. one plausible solution could be: prior to end onload, emit a new ~VisibilityPerformanceEntry logging the state change with timestamp. The page can harvest these later to figure out if the user was toggling visibility and how long the page was in background? </brainstorm>

@n8schloss

This comment has been minimized.

Show comment
Hide comment
@n8schloss

n8schloss Jul 25, 2017

It would solve the question for "was the page in background", but it wouldn't capture the case where user clicks on a link.. waits a while.. switches to a different page while navigation is loading, right?

@igrigorik, that's correct. I think we'd prefer to be able to answer the question about when the page was visible or not instead of just knowing about it's initial state. Given the current page visibility API though our main gap is around the initial state before our JS loads. If we're open to a bigger change, I think VisibilityPerformanceEntry we definitely be the ideal solution here. I could see it growing large though, so perhaps there should be some size limit with a default of like 100 entries or something.

n8schloss commented Jul 25, 2017

It would solve the question for "was the page in background", but it wouldn't capture the case where user clicks on a link.. waits a while.. switches to a different page while navigation is loading, right?

@igrigorik, that's correct. I think we'd prefer to be able to answer the question about when the page was visible or not instead of just knowing about it's initial state. Given the current page visibility API though our main gap is around the initial state before our JS loads. If we're open to a bigger change, I think VisibilityPerformanceEntry we definitely be the ideal solution here. I could see it growing large though, so perhaps there should be some size limit with a default of like 100 entries or something.

@igrigorik

This comment has been minimized.

Show comment
Hide comment
@igrigorik

igrigorik Jul 25, 2017

Member

@igrigorik, that's correct. I think we'd prefer to be able to answer the question about when the page was visible or not instead of just knowing about it's initial state.

We've received feedback before that asking "inline this in the head" is a tough sell for all but a few hand-tuned sites. For example, if you have existing RUM analytics.. it's usually loaded async (as it should) and "in the head" requirement is not compatible with current deployments. Hence my thought around buffering events to allow them to harvest such data.

Member

igrigorik commented Jul 25, 2017

@igrigorik, that's correct. I think we'd prefer to be able to answer the question about when the page was visible or not instead of just knowing about it's initial state.

We've received feedback before that asking "inline this in the head" is a tough sell for all but a few hand-tuned sites. For example, if you have existing RUM analytics.. it's usually loaded async (as it should) and "in the head" requirement is not compatible with current deployments. Hence my thought around buffering events to allow them to harvest such data.

@spanicker

This comment has been minimized.

Show comment
Hide comment
@spanicker

spanicker Jul 25, 2017

Buffering up to max(N entries, time limit) seems reasonable given the inlining in head constraint.
This requires a new Entry type though.

spanicker commented Jul 25, 2017

Buffering up to max(N entries, time limit) seems reasonable given the inlining in head constraint.
This requires a new Entry type though.

@igrigorik

This comment has been minimized.

Show comment
Hide comment
@igrigorik

igrigorik Jul 25, 2017

Member

Right. Following the pattern we converged at F2F — see w3c/performance-timeline#81 (comment) — I think we'd be looking at following changes in PV:

  1. Define new ~VisibilityPerformanceEntry interface
  2. Define an associated buffer + buffer size
  3. Update processing section to queue new perf entries defined in (1)

Conceptually, nothing hard.. but it does add some new non-trivial surface to the PV spec.

@toddreifsteck would love to hear your thoughts on this one.

Member

igrigorik commented Jul 25, 2017

Right. Following the pattern we converged at F2F — see w3c/performance-timeline#81 (comment) — I think we'd be looking at following changes in PV:

  1. Define new ~VisibilityPerformanceEntry interface
  2. Define an associated buffer + buffer size
  3. Update processing section to queue new perf entries defined in (1)

Conceptually, nothing hard.. but it does add some new non-trivial surface to the PV spec.

@toddreifsteck would love to hear your thoughts on this one.

@toddreifsteck

This comment has been minimized.

Show comment
Hide comment
@toddreifsteck

toddreifsteck Jul 26, 2017

Member

We have seen the need for this use-case in discussions with developers. I agree that we should tackle this use-case, but believe it should be in L3.

Member

toddreifsteck commented Jul 26, 2017

We have seen the need for this use-case in discussions with developers. I agree that we should tackle this use-case, but believe it should be in L3.

@spanicker

This comment has been minimized.

Show comment
Hide comment
@spanicker

spanicker Jul 26, 2017

Todd's proposal from public-web-perf meeting: for near term expose "wasHidden" and "wasVisible" to indicate whether the Document was "ever hidden" or "ever visible".
Nate, that would address this usecase?

spanicker commented Jul 26, 2017

Todd's proposal from public-web-perf meeting: for near term expose "wasHidden" and "wasVisible" to indicate whether the Document was "ever hidden" or "ever visible".
Nate, that would address this usecase?

@n8schloss

This comment has been minimized.

Show comment
Hide comment
@n8schloss

n8schloss Jul 28, 2017

@spanicker, yeah for the short term that would be sufficient. Tackling the full visibility history in L3 sounds good to me.

n8schloss commented Jul 28, 2017

@spanicker, yeah for the short term that would be sufficient. Tackling the full visibility history in L3 sounds good to me.

@igrigorik

This comment has been minimized.

Show comment
Hide comment
@igrigorik

igrigorik Jul 28, 2017

Member

Hmm, so I'm less keen on 'hack a solution for L2, solve it in L3' approach.. If we think that exposing the booleans is not sufficient to address all the underlying use cases, then we should work on the full solution and then figure out if it belongs in L2 or L3.

Member

igrigorik commented Jul 28, 2017

Hmm, so I'm less keen on 'hack a solution for L2, solve it in L3' approach.. If we think that exposing the booleans is not sufficient to address all the underlying use cases, then we should work on the full solution and then figure out if it belongs in L2 or L3.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment