Add server request count assertions to smoke tests

[patrickhulce]

Summary
In the past we've had some issues with Lighthouse making unnecessary/excessive requests to the server (#11465 #11244) when running afterPass gatherers. We don't really have a great mechanism to test this in our smokes right now, so this issue proposes adding an assertion to smoketests for the number of network requests that Lighthouse makes during its analysis.

[connorjclark]

This breaks down for headless (LR) + favicon/pwa icons. Is there some other way to do this? If not, I'd also want some way to mark an assertion as headless so it can be skipped in envs where it will differ.

[patrickhulce]

This breaks down for headless (LR) + favicon/pwa icons.

Only if we assert exact counts and don't filter out favicons :)

Is there some other way to do this?

I haven't really proposed a specific way to do this yet. I'm just saying we need to find a way to assert that certain requests don't get made to the server and that it's not much higher than expected.

I'd also want some way to mark an assertion as headless so it can be skipped in envs where it will differ.

SG, you're talking in general for smoke improvements though right? seems relatively unrelated to server assertions

[adamraine]

I've been poking around in this area.

One issue I ran into was with smoke tests run in parallel. It's difficult to discern which requests were made by a specific test in this case.

Would it make sense to look at the devtools log from lighthouse run to get a request count?

[patrickhulce]

Ah yeah that's tricky. The devtools log stops recording before many of these requests are made (intentionally, we only want to measure what the page does and not the cost of us fetching robots.txt or sourcemaps for example). This might be bigger task than I was thinking (spinning up separate servers per smoke?). Your PR a great MVP of this already :)