exp show: Add parallel coordinates plot.

[daavoo]

• ❗ I have followed the Contributing to DVC checklist.

• 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.

Closes #4455

Introduces the following new arguments / behavior to dvc exp show:

• --html 🤷
• --sort-by {foo} Defines the column to create colorscale from (Uses Experiment as default)
• --out {foo}
• --open

Examples

Using https://github.com/iterative/demo-fashion-mnist

• Interactivity

Grabacion.de.pantalla.2021-11-30.a.las.13.28.45.mp4

• Default: dvc exp show -T --html

• Customize color: dvc exp show -T --html --sort-by accuracy

• Interaction with other options: dvc exp show -T --html -sort-by accuracy --exclude-metrics loss

[dberenbaum]

My biggest question is whether this belongs in dvc exp show or might be better as a separate command, like dvc exp plot? A dvc exp plot command could also potentially show other things, like dvc plots diff between experiments, or checkpoint trends (like dvclive plots).

[dberenbaum]

Looks great!

One other comment: behavior seems kind of odd where a value is missing, like the points show up in unpredictable patterns. Would it make sense to have a set value for missing data so it all gets grouped together? Not sure if this is possible for numeric fields.

[casperdcl]

just to check - the colourbar missing a label in the video/images :)

[daavoo]

just to check - the colourbar missing a label in the video/images :)

Fixed and pushed. Nothing like a good Q/A demo in All Hands

[jorgeorpinel]

--html 🤷

Why something as generic and conceptually disconnected as "HTML"? How about --parallel-coordinates-plot/--pcp? Or a separate command as @dberenbaum mentioned

[jorgeorpinel]

--out {foo}

I understand the default here is dvc_plots/ but dvc plots (show) uses . by default. Seems inconsistent?

[daavoo]

It is the same for plots. At least in current version, DVC pots are saved in dvc_plots

[jorgeorpinel]

According to https://dvc.org/doc/command-reference/plots/show#options dvc plots show uses the same path as the source image or data series file.

It would be more consistent to just use the cwd . I think, as we may be in a subdirectory or something. dvc_plots/ seems to assume we're in the root of the repo.

Or are the docs outdated?

[jorgeorpinel]

Also, the choice of dvc_plots/ by default reinforces the argument that this feature should be in dvc plots exp --pcp or maybe dvc exp plots --pcp (exp. if we plan to have more exp-specific plots) IMO. But maybe I'm missing a discussion that already happened (lots of collapsed comments above).

[shcheklein]

To be honest --html for me would mean an HTML table, not plot. I would consider renaming this to be explicit about its meaning. I think --plot is a bit more explicit. But can limit us later if we decide to introduce more types of plots. Probably --pcp is the best unfortunately.

[daavoo]

Or are the docs outdated?

Yes they are treeverse/dvc.org#3110