Loki Datasource: Make the "Downloads" button more prominent

[09jvilla]

Problem
Today figuring out how to download logs from the Explore view of a Loki datasource is difficult.

First the user has to (1) click on 'Inspector', then they have to (2) click on 'Data', and then they can (3) click one of two buttons (download logs or download csv).

This makes discoverability challenging for what is a pretty common use case: downloading your logs as a file so you can use them in some other context. For example, some users want to download logs to attach them to a bug report. Others want to download them because they'd like to manipulate or search them in some other tool (e.g., one of the self serve users that we recently spoke to wanted to load his into a jupyter notebook for further processing).

Proposal
We'd like to put the download button on the main screen to make it more discoverable. Furthermore, we'd like to have a single button which then gives you a choice of what format to download the logs in. Right now, we support downloading as a csv or as a txt file. In the future, we might support other formats.

Here's a mockup of what it would look like after this change:

For full design, we have a Figma link as well:
https://www.figma.com/file/7ZHtMoaK4RxBz3b6m6U0cW/Show-more-logs-%2B-Sorting?node-id=1709%3A9228

[niat22]

Our new proposal for the location of the download button is to place it closer to the log lines to indicate the relation between the action button and the logs.

[gabor]

there might be a complication there. the current download-csv button only downloads a single "dataframe" (a package of data), not all the data. so, when you use the current version of the download-csv button, you can click on "Data options" to the left of the button, and choose which dataframe you are interested in. i'm not sure how to solve it here, one option would be to change the behavior and somehow download all the data... not sure.

sorry to point this out so late, i've just realized it yesterday, when looking at a github-issue 😄 (details at #53742)

[09jvilla]

NOTE: starting with grafana9, Loki uses a single-dataframe for logs always, so it's not possible to reproduce the problem there.

Given this though, wouldn't this cease to be a problem after a user is on Grafana v9 or greater?

[gabor]

NOTE: starting with grafana9, Loki uses a single-dataframe for logs always, so it's not possible to reproduce the problem there.

Given this though, wouldn't this cease to be a problem after a user is on Grafana v9 or greater?

for Loki yes, but other logs-datasources may still use multiple dataframes.

[09jvilla]

How much data is in a single data frame vs. all the data? I'm guessing it varies by logging datasource?

In Loki world, is 'all the data' the 1000 line limit and single data frame (before the change) was a fraction of that 1000 lines?

Asking because I was trying to think through what seem like the two options (for other non Loki datasources) if we change the button location. They seem to be:

1. both buttons just download the active data frame
2. both buttons download all data

Agreed that having 'download as csv' vs 'download logs' behave differently would be strange.

[gabor]

the single-dataframe vs multiple-dataframe is usually not because how much data you have but more because some datasources just split the data based on the labels.

for example:

• in Prometheus, if you query some data, and you get back "4 lines on the graph", that comes in 4 dataframes. every "line on the graph" is a separate dataframe

in grafana8 it worked the same for Loki, we took the Loki-data coming from the Loki-database, and split into dataframes, where log-lines with the same labels would go into the same dataframe.

and, i also don't know what to do with the buttons, what i know:

• the current situation is confusing
• any change is a breaking change 😄

regarding your "1." idea: "both buttons just download the active data frame" .. the problem here is that at the button's current location, the user can see what the 'active' dataframe is.
but if we move them to the "new place", there is no info about what the active dataframe is, or how to change it.

[09jvilla]

Discussed on Aug 25 call -
We'll bring the Downloads button to the front of the page for now just supporting 'txt' format (not csv).

[09jvilla]

In a multi page scenario, this downloads button should duplicate the current functionality of the 'download logs' button, where we only download the 1000 lines of the selected time range.

Only the lines in the red would be downloaded (not the previous page)

[09jvilla]

posting for visibility:

we are going to merge the Download logs button with the text-option into the current main branch. In the meantime we are working on a possible JSON format and try to add that before the 9.3.0 release.