[Lens] CSV export

[flash1293]

The current tables behind the rendered chart should be downloadable as CSV similar to the ingration in Visualize.

This includes a way to do so from the editor as well as from the context menu on Dashboards.

The biggest part of this can probably be solved by #55902

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[flash1293]

Answered questions:

• How does the UI look for this?
  • There is a "Download CSV" button in the top nav. On dashboard, there is a "Download CSV" entry in the context menu
• We have the inspector framework which also supports CSV, would it make sense to simply have an "Inspect" button in the top nav?
  • No, we will ship with our own adapter in the first phase and don't support a UI-based inspector in the first iteration (see proposal below)
• How does it work in the multi layer case? Are there multiple files to download? How does it look in the UI? (Input Alona: Multiple layers based on best effort, up to you it can be two files or any other solution you have in mind. I don't want to spend a lot of time solving this, it is still relatively rare )
  • This will simply download multiple csv files at once

[flash1293]

Proposal:

To use the currently rendered data in the chart, we need to lift it out of the expression. This can be done using adapters. Adapters are meant to be a serializable data structure to make it possible to pass them between execution contexts. The current data adapter doesn't provide this, so we should implement a Lens specific solution for this for now and unify later with legacy adapters.

• Write an expression function lens_multi_table_spy which will be inserted between the datasource expression and the visualization expression. This function takes a lens_multi_table context and passes it through, adding the map of all tables to the tables adapter. This adapter is defined by Record<string, KibanaDatatable>. Once we got rid of the multi table and store all tables in individual variables, the spy can be rewritten to a table_spy taking a name argument which defines which table in the tables adapter map to set.
• Extend the ExpressionRendererComponent by an onData$: (renderData: any, inspectData: any) => void callback. If set, it will be called each time the data$ observable of the expression loader is emitting a new value, passing the result of handler.inspect() along as well. This will contain the tables adapter as well
• In the editor frame, add currentTables to the global state. in the workspace panel, onData$ updates this state (make sure this is not resulting in an update loop) via a new action which is dispatched in the callback.
• The current table is also passed up to the app level via the callback like the state itself
• In the embeddable, the callback is simply storing this information in a member of the embeddable instance
• In a later step, we can pass the currentTables into the various functions of visualization and datasource (e.g. to use it to build better suggestions)
• Implement a generic datatableToCsv function as part of the data plugin to turn a data table into a csv file. It will use the formatting hints from the table to format cells to build a blob (or something similar which can be turned into a downloadable file easily). It can take a few parameters (raw/formatted, separator symbols).
• In the frame, the app offers the "CSV export" button if there are currentTables. It calls the datatableToCsv function if clicked and downloads the resulting blobs as individual files. The parameters of the function are set by advanced settings specifically for Lens which default to a formatted csv using comma and newline separators
• For the embeddable, a new action CsvExport is implemented using the EmbeddableContext. It will check the member on the embeddable instance and use the same datatableToCsv function logic to create downloadable files.

Open questions:

• How are the files named? I would go for just the title of the current visualization if there's a single table and append Layer <x> by entry order of the tables map if there are multiple. We can extend that using named layers if it becomes necessary later
• Should datatableToCsv be in the data plugin? We can also implement it within Lens for now, but it seems like such a generic helper

[legrego]

Should datatableToCsv be in the data plugin? We can also implement it within Lens for now, but it seems like such a generic helper

This is probably a good idea. There's a relevant discussion in #56081 around escaping output and/or warning about potentially dangerous output that could lead to CSV command injection. The reporting team implemented protections in #63645, but looking at #56081, it seems that the data table & inspector don't have similar controls in place yet.

If datatableToCsv was part of the data plugin, then we could implement those protections once, and then everyone could take advantage of this.

[flash1293]

Some discussed questions:

Will this data inspector view use the tables adapter data?

Yes, in a second step we can do this (and also have support for esdocs, essql, esdsl, ... out of the box)

Can we use this in alerting as well?

Not sure, maybe? We would need more meta information on the table though. Peter will keep it in mind in alerting planning meetings

How do adapter actually work? How should they work?

Clarify API (maybe enforce some hooks to make it clear how one should use it)

[flash1293]

Fixed by #83430