Update Dashboards and Docker Compose setup

[ppcano]

Update the existing test result dashboard, it is now called k6 Prometheus.

It also creates a new similar dashboard, k6 Prometheus - Native Histograms, to be used when storing trend metrics in native histograms.

These dashboards are an advancement from the existing one and #142. A big thanks to @jwcastillo for their exceptional work and contributions.

Docker compose changes

• Update Grafana and Prometheus versions
• Remove k6 service from the docker compose setup. This extension is built in core, so user can follow docs instructions.

Other changes

• Remove examples folder
• Remove dashboards: Apdex, custom metrics.
• Remove the test list dashboard. You can now select tests from a dropdown menu (dynamic variable).

k6 Prometheus - Native Histograms

Dynamic variables to configure the dashboard queries:

• Change Prometheus datasource
• Filter TestID when testid tag is present
• Quantile calculation for Trend metrics
• AnalyzeMetric: select a k6 metric to analyze on the panel at the bottom
• AdhocFilter to filter all the mtrics

k6 Prometheus

This dashboard is similar than the previous one. Instead of Quantile, it has Trend Metrics Query to select the Stat depending on the values from the K6_PROMETHEUS_RW_TREND_STATS option.

[codebien]

Thanks for the effort @ppcano! 🙇

I added some comments.

[codebien]

1. When I open the dashboard the first time, it should keep open the Performance Overview section. It looks better to me.
2. Why the default quantile is 99? Why do we call it here quantile and p9x in the other? We should pick only one name to have a consistent glossary.
3. The AnalyzeMetric on top creates confusion, it applies only to one chart and not to all of them as you expect from something on top. Also, in the end, having a panel for doing what the Explore functionality in Grafana already does seems not optimal to me. For these two reasons, I'm for us dropping completely the Analyze panel.
4. I think HTTP section should stay before Checks section.
5. I don't know if there is a way to do it, but if I select a testID then I would like to get the time range that centers the test and not use a relative time range.
6. The Adhoc filters adds confusion, it pulls series not relevant for the k6 context. We should drop it, because if the user needs to add custom queries then the Explorer is the correct tool.

[ppcano]

@codebien

When I open the dashboard the first time, it should keep open the Performance Overview section. It looks better to me.

✅ All the sections are open now.

Why the default quantile is 99? Why do we call it here quantile and p9x in the other? We should pick only one name to have a consistent glossary.

The format of both options cannot be the same.

1. For native histograms, it uses the histogram_quantile() function, requiring a value between 0 and 1.

2. Without native histograms, the available options depends on the values of K6_PROMETHEUS_RW_TREND_STATS. Stats functions: count, sum, min, max, avg, med, p(x).

Although the situation is not ideal, exposing this info to the end-user could guide them to learn how this output and the k6 metric format work.

The default for NH is 0.99 because is the default of K6_PROMETHEUS_RW_TREND_STATS.

For consistency, I have now renamed the variable name.

 

The AnalyzeMetric on top creates confusion, it applies only to one chart and not to all of them as you expect from something on top. Also, in the end, having a panel for doing what the Explore functionality in Grafana already does seems not optimal to me. For these two reasons, I'm for us dropping completely the Analyze panel.

✅ ok - I agree is redundant and adds an unnecessary overhead.

My intention here was to "guide" users how to query k6 metrics. Wdyt about adding an informative panel like:

 

I think HTTP section should stay before Checks section.

✅

I don't know if there is a way to do it, but if I select a testID then I would like to get the time range that centers the test and not use a relative time range.

This option is not available.

 

The Adhoc filters adds confusion, it pulls series not relevant for the k6 context. We should drop it, because if the user needs to add custom queries then the Explorer is the correct tool.

Agree, it adds confusion. To improve this, I have opened a feature request in the grafana repo.

I'd preserve the AdhocFilter because:

1. The majority of the dashboard users might not know how k6 metrics and tags(labels) work, so this menu would give visibility that they can filter results by k6 tags. It is a hook to be more proactive and start querying k6 metrics, not only visualizing the current panels.

2. Without this option, filters could not be applied to the existing visualizations.

[codebien]

LGTM 🙇 🎉

[oleiade]

Great work 😎 Overall I find this to be an improvement indeed 👍🏻

I've also spotted some small UI inconsistencies: some lines are full / some are dotted, some graphs have a "filling" / some don't. But I assumed this was made on purpose.

I have however spotted that some graphs stayed empty ("no data") in the native histograms dashboard (they have data in the normal one). Is this expected?

[ppcano]

@oleiade, thanks for sharing!

I've also spotted some small UI inconsistencies: some lines are full / some are dotted, some graphs have a "filling" / some don't. But I assumed this was made on purpose.

Good that you have mentioned them. I know there are still a few inconsistencies like "filling" charts. I'll give a last review to fix them.

Dotted lines are for req_per_second metrics.

I have however spotted that some graphs stayed empty ("no data") in the native histograms dashboard (they have data in the normal one). Is this expected?

Yes.

I guess the reason is that for NH, the k6 run command must pass the K6_PROMETHEUS_RW_TREND_AS_NATIVE_HISTOGRAM=true option.

This is why we need two different dashboards.

This is documented on the docs but I am not sure if it is very clear. As part of the changes associated to new dashboard, I'll try to update that part of the docs.