Allow exporting graphed data (as CSV?)

[juliusv]

I've talked to people introducing Prometheus in a more traditional big company - when trying to convince deciders there that Prometheus is the right tool for them, the more Excel-type of people have been asking for simple ways to export graph data as CSV. Grafana supports some kind of graph export, but it's not ideal for their needs.

We could think about adding a query/graph data export button to the Prometheus web UI. Questions are:

• do we want that?
• what formats should it support? (I can imagine JSON and CSV being the obvious ones)
• if we do support CSV, do we want to drag in some JS dependency to be sure to get all the quoting right?

[brian-brazil]

My thoughts on this have been that there should be a tool that can run a query and output in csv. I'm not sure about it being part of the core.

[beorn7]

Yeah, I would also see this in some tool.

Put it into promtool? There is still #1753 . Once that is in, adding a CSV output would be straight forward.

[juliusv]

Makes sense, but wouldn't really help with the original goal: to help the Excel-type people in a more traditional company who want to have a simple UI and probably don't even know how to use the command line. Maybe that's a somewhat isolated use case though, as that's the first time I've heard about it like that too. So if we decide we don't deem that important enough to the UI, we could close this issue (or change it to add that to promtool).

[beorn7]

So you are more thinking about a "download as ..." menu on the /graph page, so that you can retrieve the result of a query you have crafted in the UI in a format of your choice?

This might be useful even for other purposes.

[juliusv]

@beorn7 Precisely.

[beorn7]

Random concern: Prometheus queries are not necessarily reproducible. To generally get exactly the data displayed, you needed to export from the data already in the browser instead of re-run the query. So this might be a concern of the JS code running in the browser.

[juliusv]

@beorn7 That shouldn't be a problem, as we're already remembering the exact graph JSON for every graph in the JS and would have access to it.

[juliusv]

So just in case someone built such a button - would people be ok with having it in Prometheus?

[beorn7]

I'm pretty easy with it if it is really just something living in the UI.

[brian-brazil]

Yeah, I'm fine with it being just JS (and I can see it being handy for support). We'd probably also want to request it in Grafana.

[fabxc]

Same, as a pure UI feature, totally fine.

Generally, @grobie brought up the point, that it would be cool to have UI
as a separately developed tool, that can point to any Prometheus server. It
would just get resynced into the prometheus repo for each release as we
still want each instance to have a UI in case of emergency. This requires
full REST API coverage of course so the UI can entirely rely on it rather
than server-side generation.
This would provide clearer separation when questions come up whether we
want X to be a feature of Prometheus. We ultimately will need it anyway to
allow users/vendors to fully integrate Prometheus into their own management
panels.

On Fri, Sep 30, 2016 at 10:56 AM Brian Brazil notifications@github.com
wrote:

Yeah, I'm fine with it being just JS (and I can see it being handy for
support). We'd probably also want to request it in Grafana.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#2039 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEuA8vXKXwx2ZFoRq7A9nnDkw_Ne5iDoks5qvM7ZgaJpZM4KJuRY
.

[juliusv]

@fabxc Agreed to that as a mostly orthogonal thing. The REST API features are already all there, or are you talking about not just the graphing/querying interface, but all the other metadata pages as well? Anyways, probably off-topic here.

[fabxc]

Yes, we need it for status, targets, and alerts as well.

On Fri, Sep 30, 2016 at 11:22 AM Julius Volz notifications@github.com
wrote:

@fabxc https://github.com/fabxc Agreed to that as a mostly orthogonal
thing. The REST API features are already all there, or are you talking
about not just the graphing/querying interface, but all the other metadata
pages as well? Anyways, probably off-topic here.

—
You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub
#2039 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEuA8ptD7cjgQF1poZBPvzh0FX0GHKDmks5qvNTUgaJpZM4KJuRY
.

[roidelapluie]

Hi,

I have made a console template:

{{ $t := query "time()" | first | value }}{{ $i := 0 }}{{ range query .Params.query -}}
{{ if eq $i 0 }}time{{ range $v, $x := .Labels}}{{ if not (eq $v "__name__") }},{{$v}}{{ end }}{{ end }},value{{$i = 1}}{{ end }}
{{$t|humanizeTimestamp|safeHtml}}{{ range $v, $x := .Labels}}{{ if not (eq $v "__name__") }},{{$x}}{{ end }}{{ end }},{{.Value}}
{{- end }

Maybe we can add it as an example console template?

usage:

curl http://127.0.0.1:9090/consoles/query.csv?query=up

[roidelapluie]

v2 (error handling)

{{ $error := false }}{{ $t := query "time()" | first | value }}{{ $i := 0 }}{{ $samples := query .Params.query -}}                        
{{ $labels := ($samples | first).Labels -}}
{{ $ln := $labels | len -}}
{{ range $samples -}}
{{ $samplelabels := .Labels -}}
{{ if not (eq $ln ($samplelabels | len)) -}}{{$error = true}}# ERROR: different number of labels {{ $ln }} vs {{$samplelabels | len}}
{{end -}}
{{ range $lname, $lvalue := $labels -}}
{{ if not (index $samplelabels $lname) }}{{$error = true}}# ERROR: Extra label {{ $lname}} ({{$labels}} vs {{$samplelabels}})
{{ end -}}
{{ end -}}
{{ range $lname, $lvalue := $samplelabels -}}
{{ if not (index $labels $lname) }}{{$error = true}}# ERROR: Extra label {{ $lname}} ({{$samplelabels}} vs {{$labels}})
{{end -}}
{{ end -}}
{{ end -}}
{{ if not $error -}}
time{{ range $k, $v := $labels}}{{ if not (eq $k "__name__") }},{{$k}}{{ end }}{{ end }},value
{{- range $sample := $samples}}
{{$t|humanizeTimestamp|safeHtml}}{{ range $k, $v := $labels}}{{ if not (eq $k "__name__") }},{{index $sample.Labels $k}}{{ end }}{{ end }},{{.Value}}
{{- end }}
{{- end }}

v3 (add debug=true support)

{{ $error := false }}{{ $t := query "time()" | first | value }}{{ $i := 0 }}{{ $samples := query .Params.query -}}
{{ $labels := ($samples | first).Labels -}}
{{ $ln := $labels | len -}}
{{ range $samples -}}
{{ $samplelabels := .Labels -}}
{{ if not (eq $ln ($samplelabels | len)) -}}{{$error = true}}# ERROR: different number of labels {{ $ln }} vs {{$samplelabels | len}}
{{end -}}
{{ range $lname, $lvalue := $labels -}}
{{ if not (index $samplelabels $lname) }}{{$error = true}}# ERROR: Extra label {{ $lname}} ({{$labels}} vs {{$samplelabels}})
{{ end -}}
{{ end -}}
{{ range $lname, $lvalue := $samplelabels -}}
{{ if not (index $labels $lname) }}{{$error = true}}# ERROR: Extra label {{ $lname}} ({{$samplelabels}} vs {{$labels}})
{{end -}}
{{ end -}}
{{ end -}}
{{ if not $error -}}
time{{ range $k, $v := $labels}}{{ if not (eq $k "__name__") }},{{$k}}{{ end }}{{ end }},value
{{- range $sample := $samples}}
{{$t|humanizeTimestamp|safeHtml}}{{ range $k, $v := $labels}}{{ if not (eq $k "__name__") }},{{index $sample.Labels $k}}{{ end }}{{ end }},{{.Value}}
{{- end }}
{{else -}}
{{if not .Params.debug }}{{.Failure}}{{ end -}}
{{- end }}