A mechanism to track if a hint was used?

[igrigorik]

Having a signal on whether the resource hint was "used" would be helpful to help optimize which hints are emitted - e.g. the server could gather stats and adjust the types of hints it provides.

1. A preconnect is "used" if the opened socket is matched with at least one request.
2. A preload hint is "used" if it is matched with a request in the current navigation context.
3. A prerender hint is "used" if it is matched with a request of the next navigation context.

Note: "matched" is not (yet) well defined

Assuming some such mechanism exists to retrieve this status, for (1) and (2) the application should delay gathering this signal until page unload - the matching request can be initiated at any time and is not restricted to initial page load sequence. Alternatively, there could be an event emitted on the hint that could indicate that the hint was just "matched". For (3), things get more complicated, since we no longer have access to the prerender link element which initiated the fetch.

• Perhaps the initiator attribute on ResourceTiming/NavTiming should instead indicate link, or the type of hint, if the request was matched against a hinted object/socket? E.g. initiator -> prerender? However, preconnect value for initiator doesn't seem right....
• Alternative route is to expose a different flag on RT/NT. e.g. speculative -> {preconnect, preload, prerender}, which would be set if the request was matched against a hinted socket/request.

The last option seems most straight-forward and works well for both same and next navigation contexts.

(original discussion: http://lists.w3.org/Archives/Public/public-web-perf/2014Aug/0013.html)

[bizzbyster]

I agree that we need a way to indicate that a hint was used. Can we just add boolean attributes to http://www.w3.org/TR/resource-timing/#performanceresourcetiming that indicate whether a preconnect, preload, or prerender were used in the loading of the resource.

For resources initiated by a LINK tag we need to know destinationContext as well. So, for instance

link rel="preload" href="/some/image.jpg"
params="{destinationContext: 'image', headers: {'Accept': 'image/jpeg'}}"

needs to indicate link, preload, and destinationContext=image. Not sure the best syntax for that.

[igrigorik]

Thinking about this some more, I think we already have all the necessary pieces in place:

Same resource can have multiple requests associated with it, meaning that both the hint-initiated fetch and all other requests for it are logged by Resource Timing API. For example, in above example, the first entry is the timing object for link[preload], and second object is the img request for the same URL.

• The link[preload] object contains all the timing data for the hint-initiated fetch
• The img object would contain timing data to fulfill the image request, which, if matched with the preload response, would show 0's for some or all of the timestamps - e.g. if preload completed fully by the time the img request was made then most fields are 0; if preload is still in flight, then some of the img fields are zero (for the parts that preload has already completed), but the remaining timestamps are same as preload object.

In other words, while this sort of processing does require a bit of smarts to line up the timestamps, it does allow you to answer all the important questions: was the hint request initiated; what was the timing data for hint-initiated request; were there other, non hint-initiated requests for same resource; did hint-initiated request help later request, and so on. Also, if you wanted, you could infer how the hint-initiated request was used by looking at initiatorType of other requests for the same URL - i.e. in above example, it would be the img object.

---

For next-navigation fetches, we have two cases:

• The HTML document of next navigation context: we already have document.visibilityState -> prerender which indicates that page is being prerendered and this can be easily logged by the application.
• The resources used by the next navigation context: we wouldn't have access to the ResourceTiming object that initiated the fetch prior to the navigation, but if we have a request in the current navigation context that was matched with the hint-initiated request in previous navigation, the timing data would reflect that by showing 0's for some or all of the timestamps. The application could gather this data and compare the timing data with and without the use of hints...

---

Finally, for rel[preconnect] we wouldn't see a request object in ResourceTiming or NavTiming, but we can still look at all the timestamps reported by NavTiming+ResourceTiming prior to requestStart to see if those have improved when the hint is used -- similar logic to next-navigation subresource fetches.

In summary, it seems like there is no need to modify or extend RT/NT specs: all the pieces are in place, we just need to test the implementations to ensure that the timing data is being reported correctly.

[bizzbyster]

"The img object would contain timing data to fulfill the image request, which, if matched with the preload response, would show 0's for some or all of the timestamps - e.g. if preload completed fully by the time the img request was made then most fields are 0; if preload is still in flight, then some of the img fields are zero (for the parts that preload has already completed), but the remaining timestamps are same as preload object."

Why not instead use -1 for these timestamps when a hint was used both in the case where the preload had completed fully and in the case where the preload was still in flight? This eliminates confusion for the case when RT events actual take 0 milliseconds and also when timestamps happens to be the same exact value as the hint, even though the hint was not used.

"Also, if you wanted, you could infer how the hint-initiated request was used by looking at initiatorType of other requests for the same URL - i.e. in above example, it would be the img object."

You are right about this case -- thanks for this clarification.

[igrigorik]

Why not instead use -1 for these timestamps when a hint was used both in the case where the preload had completed fully and in the case where the preload was still in flight? This eliminates confusion for the case when RT events actual take 0 milliseconds and also when timestamps happens to be the same exact value as the hint, even though the hint was not used.

Note that the 0's case is how the current browsers already behave... and we can't and wouldn't change that. The browser can initiate its own optimizations (e.g. preconnect) based on past navigation data, or other heuristics. Then, if a request is matched against that socket, the timing for DNS/TCP/TLS handshakes may be set to 0 -- depending on the state of the socket, etc. Similar logic applies for other types of hints, and the more general case of reusing an open socket, etc.

Further, under the hood, the browser shouldn't care if the hint was initiated by user or through some other means, and as a result, we should treat all these cases in a consistent manner -- both for sanity of implementers and users of RT/NT API's.

Besides, I don't believe DOMHighResTimeStamp allows for negative values, and semantically it doesn't seem right either. Also, given that we're talking about microsecond granularity, I think the false positive rate would be very low.

[bizzbyster]

Okay I'm fine with this approach. I still wish there could be a mechanism to allow explicitly tying together a hint and the object that used the hint. But I suppose in many cases this will be a tricky thing to determine.

[igrigorik]

Great, closing this. We can revisit in the future as needed.

[bizzbyster]

Thinking more about this, there is one thing I don't like about this: "if preload is still in flight, then some of the img fields are zero (for the parts that preload has already completed), but the remaining timestamps are same as preload object."

By making the timestamps the same as the preload object, we will lose the ability to know how much we are benefiting from the preload as we will lose the start time of the non-preload object. If a preload is still inflight, I'd prefer that the fetchStart timestamp of the non-preload object be the current time when that non-preload is first instantiated. This keeps the PerformanceEntry objects of the preload and the non-preload independent from each other. The non-preload will just complete faster than it otherwise would have.

Instead of setting the non-preload object's timings to the preload object's to indicate that an inflight preload was used, I prefer the idea of creating a "fetchID". This ID would be unique across all fetches performed by the browser for a given period. In the case of a preload "hit", meaning that the preloaded object was subsequently used to render the page, both entries would have the same fetchID but they could have different timings.

Thoughts?

[igrigorik]

If a preload is still inflight, I'd prefer that the fetchStart timestamp of the non-preload object be the current time when that non-preload is first instantiated.

Yes, that's exactly how it would work. The two objects are independent of each other. Note that when we say "set non-preload object's timings to be the same", there is no magic here.. It's simply stating that the same timing values will be visible in both objects because they are tied to the same socket.

Say we're fetching an object over TLS and we have a preload request, followed by an img request asking for it... The preload object would have fetchStart set as time when the hint was processed, followed by full set of timestamps for TCP, TLS, etc. Then, an img request is dispatched, which happens to be after the preload has already done the TCP+TLS handshakes.. So, for this request, the timing object would return fetchStart as the time when img request was dispatched, 0's for TCP and TLS timestamps (because those were already complete), and appropriate timestamps for the remaining variables.

I think the above is sufficient to answer what we're after here.

[bizzbyster]

Let's say fetchStart_img = fetchStart_preload + 1 millisecond. How would you distinguish between A) the preloaded fetch was used by the img request, and B) the img request issued its own fetch on an existing TLS connection to the host?

[igrigorik]

Try initiating a few XHR's for the same resource in a loop.. if they end up using different sockets, I think you'll find that the timing data is different: a) we're talking about microsecond granularity, b) packets arrive at different intervals. The chance of having exact same timestamps, at microsecond granularity, for two different sockets are very, very low.

Also, I'd argue that if the intent here is to asses whether a hint was useful or not, just observing the fact that the actual resource request is being dispatched a millisecond or so later is a good indicator that the hint is probably not that useful, and you should focus on something else instead.

[bizzbyster]

When analyzing data for many users that successfully used a preloaded resource, it's possible for the majority of users to benefit significantly from a preloaded resource while a minority receives only a little benefit. We do not want the minority that did not benefit (for some other reason) to mess up our statistics on the question of whether or not the hint was needed by the page.

Any approach that requires this type of inexact inference to determine whether or not a hint was used feels hackish to me. Can you think of other ways where RT API feedback can tell us with 100% accuracy whether or not a hint was used?

[igrigorik]

I don't see how what I've described precludes you from answering those questions. As I said, the granularity of timestamps (microsecond!) should be sufficient to even distinguish cases where requests are being dispatched back to back. Also, note that 100% accuracy is probably a bit of a misnomer given that the browser may invoke own optimizations, downgrade your hints, and so on.

Practically speaking, I think we need to get this out the door, gather some real-world data, and then evaluate it and see if it meets our needs. As is, the above behavior is what we should get by default without any extra spec modifications, and I think it should be sufficient. That said, if we get data that proves this otherwise, we can always revisit.

[bizzbyster]

I'm definitely all for getting this out the door. And I do understand your point about the matching of the microsecond timestamps as a solution that will get the correct result most of the time. So I'm fine with this approach for now.

[bizzbyster]

@igrigorik As a somewhat related thing, I wanted to report that in Chrome today, performance.getentries() returns only a single record for a LINK rel=subresource fetched object that is used by the page. Is there another forum I should be using to report this bug?

[igrigorik]

@bizzbyster sounds like a bug for crbug.com

[bizzbyster]

I just realized that it’s hard to report the bug while LINK rel=subresource is totally broken though. Meaning, in the current version of Chromium, if a preload hint is needed, there are still two requests. And therefore two RT entries. I’ll wait til the bug is fixed to report it.

Thanks,

Peter
On Sep 10, 2014, at 12:06 PM, Ilya Grigorik notifications@github.com wrote:

@bizzbyster sounds like a bug for crbug.com

—
Reply to this email directly or view it on GitHub.

[bizzbyster]

See https://github.com/bizzbyster/ResourceHintsDocs/blob/master/Preload_Used_Waterfall.adoc for illustration of how this looks in a waterfall.

[igrigorik]

Peter, taking another pass over the doc... Example 1: lgtm.

For example 2, I'd expect to see something slightly different.

For request fetch, shouldn't the column be:

• fetchStart: 153.25
• domainLookupStart: 153.25 (?) - we should check how this is handled today when we issues a dns-prefetch and then try to connect to same host while that's in flight.
• connectStart: 170.693
• secureConnectionStart: 265.021
• ...

In other words, the timestamps for preload and prefetch should align to same values. The way to think about this is to decouple request from the socket. The "preload request" initiates DNS/TCP/TLS work at the socket layer, and the HTTP request is bound to it.. A later fetch request for same resource is then bound to same socket and hence gets the same timestamps (modulo difference for start time).

[bizzbyster]

The only thing I don't like about this approach is then it doesn't make sense to use a different color in waterfalls for preload used objects, which really highlights them and informs the waterfall user that they really are a different sort of thing. But that's not a big issue for me either so I'm fine with what you propose.

[igrigorik]

Right, the visual treatment is a separate concern... There is nothing stopping different tools from providing different visualizations of this data. The important bit is that there is a consistent way to correlate the two requests as preload vs subsequent fetch.