network.requestWillBeSent triggered to late

[johnerikhalse]

When issuing a page.navigate, the network.requestWillBeSent event is triggered after request has been sent and response has been received, but just before the network.responseReceived event is triggered.

For embedded resources loaded by the request, the events are triggered as expected. It is just this first request which is triggered to late.

[sns45]

I am also having the exact same problem. network.requestwillbesent is sometimes too slow. It's drastically increasing the runtime of my tests. I have to wait for 3 to 5 sec to ensure that the request is captured. Although I can see in UI that request has been sent and the response has been successfully received but still request is not captured through network.requestwillbesent.

[allada]

This is likely because the navigation request is not owned before it 'commits' to a renderer, so we wait until the header is received before we know if it belongs to the same target. A way to work around this is to use request interception: https://chromedevtools.github.io/devtools-protocol/tot/Network#method-setRequestInterception

[route]

I have the same issue but with clicks (navigate works pretty stable) the chances that event comes earlier than the response are 50/50:

>>> {"method":"Input.dispatchMouseEvent","params":{"type":"mouseReleased","modifiers":0,"button":"left","x":631.421875,"y":347.46875,"clickCount":1},"id":27}
    <<< {"id":27,"result":{}}
    <<< {"method":"Network.requestWillBeSent","params":{"requestId":"87281F5DA7D32BD51B703F974866C260","loaderId":"87281F5DA7D32BD51B703F974866C260","documentURL":"http://127.0.0.1:35421/with_simple_html","request":{"url":"http://127.0.0.1:35421/with_simple_html","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/73.0.3641.0 Safari/537.36","Referer":"http://127.0.0.1:35421/with_html"},"mixedContentType":"none","initialPriority":"VeryHigh","referrerPolicy":"no-referrer-when-downgrade"},"timestamp":541443.238731,"wallTime":1545237507.262082,"initiator":{"type":"other"},"type":"Document","frameId":"4A72653511E5A7B2D51F5831B4493B21","hasUserGesture":true}}
    <<< {"method":"Page.frameStartedLoading","params":{"frameId":"4A72653511E5A7B2D51F5831B4493B21"}}
    <<< {"method":"Network.requestIntercepted","params":{"interceptionId":"id-2","request":{"url":"http://127.0.0.1:35421/with_simple_html","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/73.0.3641.0 Safari/537.36","Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","Referer":"http://127.0.0.1:35421/with_html"},"initialPriority":"VeryHigh","referrerPolicy":"no-referrer-when-downgrade"},"frameId":"4A72653511E5A7B2D51F5831B4493B21","resourceType":"Document","isNavigationRequest":true}}

So currently there's no way for sure to know if click happened to be reloading the whole frame or just a plain click w/o explicitly checking subsequent events with smth like sleep...

This is also somewhat awkward when you get Network.requestWillBeSent before Network.requestIntercepted:

    <<< {"method":"Network.requestWillBeSent","params":{"requestId":"65C55C1D077DFED3D02979143E0AD039","loaderId":"65C55C1D077DFED3D02979143E0AD039","documentURL":"http://127.0.0.1:46289/cuprite/unwanted","request":{"url":"http://127.0.0.1:46289/cuprite/unwanted","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/73.0.3641.0 Safari/537.36","Referer":"http://127.0.0.1:46289/cuprite/url_blacklist"},"mixedContentType":"none","initialPriority":"VeryHigh","referrerPolicy":"no-referrer-when-downgrade"},"timestamp":594386.725868,"wallTime":1545290450.749218,"initiator":{"type":"parser","url":"http://127.0.0.1:46289/cuprite/url_blacklist","lineNumber":4},"type":"Document","frameId":"DB43E81BEF3984484A86193625F4B496","hasUserGesture":false}}
    <<< {"method":"Network.requestIntercepted","params":{"interceptionId":"id-3","request":{"url":"http://127.0.0.1:46289/cuprite/unwanted","method":"GET","headers":{"Upgrade-Insecure-Requests":"1","User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/73.0.3641.0 Safari/537.36","Accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","Referer":"http://127.0.0.1:46289/cuprite/url_blacklist"},"initialPriority":"VeryHigh","referrerPolicy":"no-referrer-when-downgrade"},"frameId":"DB43E81BEF3984484A86193625F4B496","resourceType":"Document","isNavigationRequest":true}}
    <<< {"method":"Page.frameStartedLoading","params":{"frameId":"DB43E81BEF3984484A86193625F4B496"}}

[TimvdLippe]

This repository is related to Chrome DevTools Protocol, but does not track issues regarding its definition or implementation. If you want to file an issue for the Chrome DevTools Protocol, please open an issue on https://crbug.com under component: Platform>DevTools>Platform. Thanks in advance!