Chrome having difficulties with h2o push - erratic behaviour

[utrenkner]

I assume it is a Chrome issue - but I would like to know if others seeing this, too:
Chrome unpredictably reloads resources already pushed to the client. I ran tests on several WebPagetest instances and sometimes it works flawlessly, but sometimes Chrome re-downloads some or all of the resources. Tests with Firefox went always ok.

Examples of problems with Chrome and push:
https://www.webpagetest.org/result/170516_GX_TER/
https://www.webpagetest.org/result/170516_QH_TPV/

No problems with Firefox:
https://www.webpagetest.org/result/170516_AS_T1X/

If others can confirm, I would try configure h2o to not push to Chrome until the issue is resolved.

$ h2o --version
h2o version 2.2.2
OpenSSL: LibreSSL 2.4.5
mruby: YES
$ freebsd-version
10.3-RELEASE-p19

[utrenkner]

Further info: In order to ensure that TLSv1.3 was not the reason for the problems I ran some of the tests with maximum-version: TLSv1.2. It showed no difference.

[utrenkner]

Really looks like a regression in Chrome: I downgraded h2o to 2.1.0 on a test instance and saw the same erratic behaviour. I have never seen this in the past - with any browser.

[deweerdt]

FWIW, I'm seeing this too on Chrome 57 and 58. Here's the relevant extract from chrome://net-internals:

t=471980 [st=  168]    HTTP2_SESSION_RECV_PUSH_PROMISE
                       --> :method: GET
                           :scheme: https
                           :authority: mygreatjobs.de
                           :path: /jobs/Greenjobs/Greenjobs.css
                           user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
                           accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
                           accept-encoding: gzip, deflate, sdch, br
                           accept-language: en-US,en;q=0.8,fr;q=0.6,es;q=0.4
                       --> id = 1
                       --> promised_stream_id = 2

t=471980 [st=  168]    HTTP2_SESSION_RECV_HEADERS
                       --> fin = false
                       --> :status: 200
                           server: h2o/2.2.2
                           date: Tue, 16 May 2017 21:09:07 GMT
                           cache-control: max-age=31536000
                           content-type: text/css
                           last-modified: Thu, 17 Mar 2016 12:09:31 GMT
                           vary: accept-encoding
                           content-encoding: br
                           accept-ranges: bytes
                           strict-transport-security: max-age=31536000; includeSubDomains; preload
                           content-security-policy: default-src 'self' 'unsafe-inline' 'unsafe-eval' https://fonts.googleapis.com data:; font-src 'self' https://fonts.gstatic.com
                           x-xss-protection: 1; mode=block
                           x-frame-options: SAMEORIGIN
                           x-content-type-options: nosniff
                           x-http2-push: pushed
                           content-length: 27369
                       --> stream_id = 2

t=471982 [st=  170]    HTTP2_SESSION_RECV_DATA
                       --> fin = false
                       --> size = 4959
                       --> stream_id = 2
t=471982 [st=  170]    HTTP2_SESSION_RECV_DATA
                       --> fin = false
                       --> size = 5598
                       --> stream_id = 2
t=472126 [st=  314]    HTTP2_SESSION_RECV_DATA
                       --> fin = false
                       --> size = 5827
                       --> stream_id = 2
t=472417 [st=  605]    HTTP2_SESSION_RECV_DATA
                       --> fin = false
                       --> size = 3840
                       --> stream_id = 2
t=472417 [st=  605]    HTTP2_SESSION_RECV_DATA
                       --> fin = false
                       --> size = 7145
                       --> stream_id = 2
t=472418 [st=  606]    HTTP2_SESSION_RECV_DATA
                       --> fin = true
                       --> size = 0
                       --> stream_id = 2


t=472707 [st=  895]    HTTP2_STREAM_ADOPTED_PUSH_STREAM
                       --> stream_id = 2
                       --> url = "https://mygreatjobs.de/jobs/Greenjobs/Greenjobs.css"

t=472930 [st= 1118]    HTTP2_SESSION_SEND_HEADERS
                       --> exclusive = true
                       --> fin = true
                       --> has_priority = true
                       --> :method: GET
                           :authority: mygreatjobs.de
                           :scheme: https
                           :path: /jobs/Greenjobs/Greenjobs.css
                           pragma: no-cache
                           cache-control: no-cache
                           user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36
                           accept: text/css,*/*;q=0.1
                           referer: https://mygreatjobs.de/jobs/en/jobboard
                           accept-encoding: gzip, deflate, sdch, br
                           accept-language: en-US,en;q=0.8,fr;q=0.6,es;q=0.4
                       --> parent_stream_id = 0
                       --> stream_id = 9
                       --> weight = 256

We clearly see Chrome getting the full pushed stream, associating it with the request in HTTP2_STREAM_ADOPTED_PUSH_STREAM, but still issuing a request for it. I've also tried disabling cookies, in case that made Chrome invalidate the entry in the cache, but that didn't solve the issue either.

[worenga]

Cannot confirm for chrome 59 and tls1.2, seems to be a chrome regression. Link preload header used to be buggy in past versions without an as="style" attribute.

Also tried Chrome 57.0.2987.133 and i cannot observe the behaviour (now).

[utrenkner]

As suggested by Pat Meenan at WebPagetest.org and @worenga I tried it with the relative "as" attribute, and this seems to solve the issue!

I will submit another issue regarding the h2o documentation, which (wrongly) leaves out this attribute.

[worenga]

Why is there
pragma: no-cache
and
cache-control: no-cache

in the request? Can you also send us the pushed stream headers please?
It might be that your CSS is not cacheable. In this case the push is not usable since the standard mandates that a pushed request must be cacheable.

[utrenkner]

The server does not send the no-cache headers. If you look above, the server answers with cache-control: max-age=31536000. It is the client that re-requests the same resource with the no-cache directive.

If you could tell me, how to record the pushed stream headers, I will gladly post them here.

But again, the problem seems to have been solved by adding the relative "as" attribute to the preload links. I am 99% sure that the problem was due to some changes in Chrome, as the exact same setup used to work flawlessly with Chrome. After having tested an earlier version of h2o I was able to confirm that the problem - without the "as" - was there, too. So really, it looks like it has to do with the "as"-attribute and some changes in Chrome.

[deweerdt]

If you could tell me, how to record the pushed stream headers, I will gladly post them here.

They're available under chrome://net-internals/#http2 . I made a screen cast demo'ing how to get them: https://yay.im/get-headers.mp4

[deweerdt]

@utrenkner Do you mind if I re-use the WPT links in a Chrome bug report? I would like to get to the bottom of why Chrome is re-fetching the resources.

[worenga]

@deweerdt i still think it is a regression since i cannot reproduce it in newer chrome versions.
if we would have a live example that could help

[deweerdt]

@worenga a regression of Chrome or H2O? I'm going to setup a server that uses link: headers with missing as= attributes to try to repro.

[utrenkner]

@deweerdt Please feel free to use the WPT-results in the bug report. Thank you for asking. I would be interested in following that bug report - may I ask you to post the link, once you have filed the bug report?

[deweerdt]

I ended up not filing a chromium bug, because https://bugs.chromium.org/p/chromium/issues/detail?id=655698 is already tracking the issue on their side. I think that we're good with a change of docs in h2o.