core(main-resource): adjust main resource identification logic

[kdzwinel]

this PR aligns main-resource gatherer logic with final URL logic from driver.js

lighthouse/lighthouse-core/gather/driver.js

Lines 611 to 622 in 3030b4f

	// Update startingUrl if it's ever redirected.
	this._monitoredUrl = startingUrl;
	this._networkStatusMonitor.on('requestloaded', redirectRequest => {
	// Ignore if this is not a redirected request.
	if (!redirectRequest.redirectSource) {
	return;
	}
	const earlierRequest = redirectRequest.redirectSource;
	if (earlierRequest.url === this._monitoredUrl) {
	this._monitoredUrl = redirectRequest.url;
	}
	});

Fixes #4454
Fixes #4440

[kdzwinel]

@patrickhulce @paulirish PTAL

[patrickhulce]

Sorry @kdzwinel I had my review sitting here and never hit submit!! Thoughts below

I think ideally we'd have a single source of truth for the final URL rather than duplicate copies of the logic, even if it is pretty small

Which of the following sounds best to folks

• Change the computed artifact to accept the final URL/requestId as input, and find the correct network record that way
• Change the driver code to just call this function to get the final URL instead of keeping track as we go
• Add an assert to all usages of requestMainResource that it matches the URL artifact

[kdzwinel]

@patrickhulce no worries! Reusing URL.finalURL makes much more sense, not sure how I missed that. I went ahead and updated the code. PTAL

[kdzwinel]

that's the correct URL for the ../fixtures/traces/progressive-app.json returned by the fake-driver

[kdzwinel]

critical-request-chains now depends on URL via mainResource

[patrickhulce]

LGTM thanks @kdzwinel!

[patrickhulce]

the appveyor failure is hopefully unrelated @kdzwinel ?

[kdzwinel]

@patrickhulce fun bug alert!

Hm… It fails on AppVeyor but passes on Travis? Yesterday I couldn't reproduce it locally, but today I can?? It fails on master???

Mystery! So I started debugging and it looks like mainResource.responseHeaders suddenly misses one of the Link headers 🤔Someone messed with the static-server and commited a broken PR? Nope! It's Chrome:

DevTools in Chrome 63 (stable):

DevTools in Chrome 66 (one used for testing):

https://crbug.com/812592 ❤️

[patrickhulce]

awesome sleuthing and thanks for filing the crbug @kdzwinel!