Allow multiple plots from one plot file

[mattlbeck]

Consider a mock dvc.yaml such as:

stages:
    A:
        cmd: ...
        plots:
            - metrics.csv:
                x: x
                y: y1
            - metrics.csv:
                x: x
                y: y2

When using dvc plots show, DVC will only show one (the last) plot. The fix is to split the one metrics file into multiple metrics files with the same x but different y variables. This feels pretty inconvenient for metrics that are related in their independent variable, and so can be stored in the same table and file.

On the surface I don't see any reason why a specified plot artifact can only be used for one plot.

[pared]

Related #5980

[shcheklein]

Relevant to iterative/studio-support#23.

[shcheklein]

@dberenbaum should we/can we prioritize this?

[dberenbaum]

See a proposed spec for this in #5980 (reply in thread).

@shcheklein Yup, we will be working on it soon. @pared What do you think? I think we had planned to address #6603 first to get image support looking a little better, but this is probably more important given the number of requests for it.

[daavoo]

By "combining" we have been usually referring to get multiple (y) columns from either the same file or multiple files, which seems reasonable given how we usually recommend formatting the data in linear plots.

However, I think that the user in iterative/studio-support#23 (comment) showcases a new scenario that we haven't discussed so far in #5980 (reply in thread) . It is basically using the values of a third column (z) to group the values of a single column (y) and the template looks very clean.

IMO this use case could be easily covered on the DVC side by extending the vega template and adding a new placeholder
DVC_METRIC_COLOR_LABEL and we could encourage this format as a quick workaround for combining y values. On the DVCLive side, we could add some logic to facilitate the optional usage of this format (i.e. adding a new stage column for grouping based on the metric name).

[daavoo]

I think that #6603 still needs to be addressed first because the diff operation for this kind of combined plots would require implementing some sort of facet grid in the HTML layout (looks like Studio would need to implement something similar as well), as @dberenbaum mentioned in #5980 (comment) .

[dberenbaum]

However, I think that the user in iterative/studio-support#23 (comment) showcases a new scenario that we haven't discussed so far in #5980 (reply in thread) . It is basically using the values of a third column (z) to group the values of a single column (y) and the template looks very clean.

I think this could even be a separate issue to add grouping by a categorical column. We could add a template like linear_categories and add the DVC_METRIC_COLOR_LABEL. We could also add a similar template for the scatter plot (and maybe others like bar plots in the future).

[pared]

While grouping by categorical values sounds like a valid use case, I would not focus on it right away. My reasoning for this is the user had to come up with a workaround for the currently not existing feature. This needs to be addressed but I think we should not focus on it because it is about adjusting existing plots implementation to unsupported use case. I think we should focus on #5980 and come up with a process that will allow us to handle that. Main research should focus on two things:

1. How to structure Repo.plots data so that we can meaningfully group both images and vega datapoints.
2. How to implement rendering for Vega - ideally we should limit number of plots to as small as possible, if its possible to do "multiple grouping" ex revision -> filename/stage_name we might not require new template for that.

[dberenbaum]

I agree that it seems better to support multi-series plots natively in a more generic way. Users should not have to create custom templates or adjust their data format to make it work. The train/test example is often going to have data in different files, and we shouldn't ask users to combine them to plot them together. However, we could point users to that workaround in the meantime.

@pared What about handling the proposal in #5980 (reply in thread) and parsing data from that syntax? Do we need more research there?

[pared]

Yes, we need to define how diff will behave, and consider how more advanced use cases (like parallel coordinates) plots will be defined using the proposed approach.

[dberenbaum]

I think it would be great if parallel coordinates plots could be made via dvc plots, but I think there are alternatives. If it's only part of dvc exp show --html or similar, it doesn't necessarily need to fit cleanly into the dvc plots syntax, right?

[pared]

@dberenbaum at this time it does not correspond at all, I just need to make a POC and research how hard it will be to generalize its behavior.

[pared]

But besides that I believe we can start implementation right away.