Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Prometheus Remote Write as an Optional Results Visualization #1761

joe-elliott opened this issue Dec 8, 2020 · 5 comments · Fixed by #2784

Support Prometheus Remote Write as an Optional Results Visualization #1761

joe-elliott opened this issue Dec 8, 2020 · 5 comments · Fixed by #2784


Copy link

While browsing currently supported results visualizations I noticed that Prometheus is missing. Prometheus is a commonly used and powerful time series aggregation and querying solution that we would like to leverage to visualize output from k6.

We have recently just merged a few k6 benchmark tests in our repo and enabled them in CI. We (and I think others as well) could benefit from storing output metrics from these applications in Prometheus.

Feature Description

I can't say I'm familiar enough with the various visualization options and what data is available to give a good description here. I am quite familiar with the prometheus metrics format/options and can provide insight/advice from that side.

Suggested Solution (optional)

I believe the best path to supporting Prometheus would be to implement the remote write protocol.
Also note by doing so you would immediately be adding support for a large number of tsdbs and backends: that others may be interested in using.

The actual Prometheus RW client is exposed and usable directly in a go application:

Copy link

na-- commented Dec 9, 2020

Thanks for making this issue! I can't say that I am familiar with Prometheus remote write, but from skimming the docs, it seems like a much better fit for k6 than the previous attempt at integrating Prometheus (#478). And yes, I'd say that we want to have Prometheus support in the core of k6...

As I mentioned in a PR for another potential output yesterday (#1755 (comment)), we plan to heavily refactor the current k6 output interface in the next few months, hopefully in time for k6 v0.31.0. The main reason for that is that the current output (or Collector, as it's known in the code) API has many issues (#1606, #1423 and the many linked in #1075) and is barely fit for purpose... And we also want to to enable xk6 output extensions!

That said, even if someone contributes a quality Prometheus remote write PR with the current output interface, I think we'd be willing to merge it and port it to the new API ourselves... 😅 Or, after we refactor everything, making such a PR ourselves will probably be very high on the priority list.

Somewhat connected issues: #858, #343, #1064

Copy link

dgzlopes commented Mar 2, 2021

Prometheus folks added experimental support for remote_write requests on the latest release.

That means, that apart from shipping results to a wide array of remote write backends, this would work with "vanilla" Prometheus too!

Copy link

simskij commented Jul 12, 2021

From a glance, this look as it could be as easy as just forwarding the existing samples through the prometheus remote write client That's of course not the case though. From some playing, I think the way we'll need to go about doing this is by:

  1. Transform all k6 samples to the corresponding Prometheus metric, including timestamps
  2. Somehow construct a time series out of these metrics (this is where I'm currently at)
  3. Convert the time series into a protobuf stream
  4. Send it using the WriteClient

Copy link

yorugac commented Sep 14, 2021

Confirming that @simskij's draft extension works 👍
Apparently, the problem with snappy compression on remote write was that Encode must be used only as a return value:

encoded := snappy.Encode(nil, buf)

And not like this:

var encoded []byte
snappy.Encode(encoded, buf)


@na-- na-- mentioned this issue Oct 5, 2021
Copy link

yorugac commented Oct 7, 2021

To track overall progress for the remote write extension see here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet

Successfully merging a pull request may close this issue.

6 participants