Add WPT dashboards

[jpvincent]

The goal is to have almost the same dashboards for WebPageTest results as there is for the original sitespeed.

I've :

• copied some of the original dashboards, prefixed them with "WebPagetest" (file name and dashboard title) and adapted the graphite path to those of Webpagetest, namely
  • PageSummary
  • CompareBetweenPages
  • 3rdvs1stparty (<= this one is simplified)
• modified the WPT Page Summary

Be careful to configure the WPT test to send more metrics to graphite, here is my current configuration :

	filterRegistry.registerFilterForType([
		'data.median.firstView.SpeedIndex',
		'data.median.firstView.render',
		'data.median.firstView.TTFB',
		'data.median.firstView.loadTime',
		'data.median.firstView.fullyLoaded',
		'data.median.firstView.userTimes.*',
		'data.median.firstView.bytesIn',
		'data.median.firstView.breakdown.*.requests',
		'data.median.firstView.breakdown.*.bytes',
		'data.median.firstView.requestsFull',

		'data.median.firstView.custom.*',
		'data.median.firstView.domContentLoadedEventEnd',
		'data.median.firstView.fullyLoadedCPUms',
		'data.median.firstView.docCPUms',
		'data.median.firstView.score_cache',
		'data.median.firstView.score_gzip',
		'data.median.firstView.score_combine',
		'data.median.firstView.score_minify',
		'data.median.firstView.domains.*.bytes',
		'data.median.firstView.domains.*.requests',

		'data.median.firstView.domElements',
		'data.median.firstView.lastVisualChange',
		'data.median.firstView.visualComplete85',
		'data.median.firstView.visualComplete90',
		'data.median.firstView.visualComplete95',
		'data.median.firstView.visualComplete99',
		'data.median.firstView.FirstInteractive',
		'data.median.firstView.TimeToInteractive',
		'data.median.firstView.cpuTimes.*',
		'data.median.firstView.chromeUserTiming.*'
	], 'webpagetest.pageSummary')

• for chromeUserTiming to work, the WPT test must have been set with chrome and the --timeline option

[jpvincent]

Known problems :

• can not display the "Runs" now : currently sitespeed.io does not send the required annotations to graphite in order to retrieve the test
• "3rd party vs 1st party" graphes assumes all assets are on the main tested domain : that can't work with all sites, but the information about the whitelisted domains is not sent by sitespeed.io for the WPT results

[soulgalore]

@jpvincent

can not display the "Runs" now : currently sitespeed.io does not send the required annotations to graphite in order to retrieve the test

That's because of you disabled browsertime or? We can fix that so that if "only" run WebPageTest, we can still send the annotations, right now we only do it on the browsertime.pageSummary, let us fix that in sitespeedio/sitespeed.io#1867

[soulgalore]

@jpvincent I wouldn't mind changing the default config for WebPageTest to send metrics.

I'll try this out, best case it would be great to release it with grafana/grafana#1611 so that it's separated, or we should change the install script so you could turn on WebPageTest dashboards. People seems to get confused why they don't see any data in those dashboards when they only run Browsertime.

[jpvincent]

@soulgalore

That's because of you disabled browsertime or?

In my tests, as a matter of fact I only use Webpagetest but in the case of WPT dashboards we should ideally show the link to the WPT result itself. I struggle with anotations so I'm not even sure it's possible at all to have one anotation for the sitespeed result page and another for the WPT tests, displayed on 2 different dashboards for the same test. Anotations seems to be global, but maybe we can add a 'webpagetest' tag to filter it ?

@jpvincent I wouldn't mind changing the default config for WebPageTest to send metrics.
Yes that would be nice in order to populate the dashboards :)

If overcharging the disk by default is a problem, we could add in the documentation and in the dashboard comments how to enable more measures, with an associated warning.
From the above list, we could have those by default :

		'data.median.firstView.SpeedIndex',
		'data.median.firstView.render',
		'data.median.firstView.TTFB',
		'data.median.firstView.loadTime',
		'data.median.firstView.fullyLoaded',
		'data.median.firstView.bytesIn',
		'data.median.firstView.requestsFull',
		'data.median.firstView.custom.*',
		'data.median.firstView.domContentLoadedEventEnd',
		'data.median.firstView.fullyLoadedCPUms',
		'data.median.firstView.docCPUms',
		'data.median.firstView.score_cache',
		'data.median.firstView.score_gzip',
		'data.median.firstView.score_combine',
		'data.median.firstView.score_minify',
		'data.median.firstView.userTimes.*',
		'data.median.firstView.domElements',
		'data.median.firstView.lastVisualChange',
		'data.median.firstView.visualComplete85',
		'data.median.firstView.visualComplete90',
		'data.median.firstView.visualComplete95',
		'data.median.firstView.visualComplete99',
		'data.median.firstView.FirstInteractive',
		'data.median.firstView.TimeToInteractive',
		'data.median.firstView.chromeUserTiming.*'

the chromeUserTiming contains metrics like the First meaningful paint but it looks like it's not available in each WPT server release (currently works on the .org but no more on the latest private instance, I will investigate).

cpuTimes.* contains 24 metrics, I start to monitor to see if I can have something useful from it, but maybe we should have it off by default

So those are the metrics that could eat up space or that are a bit obscure :

		'data.median.firstView.domains.*.bytes',
		'data.median.firstView.domains.*.requests',
		'data.median.firstView.cpuTimes.*',

I can PR that into sitespeed itself if you want

[soulgalore]

@jpvincent if you have time that would be great (a PR with better default for WPT), thanks!

[jpvincent]

the companion PR is here : sitespeedio/sitespeed.io#1871

[soulgalore]

I'll have a go and push these to the dashboard.sitespeed.io this weekend (the docker tag is latest) and go through them and verify, the we can tag a 6.3 version and update the docker compose file.