dvc metrics diff for summon/catalog artifacts

[dmpetrov]

For projects with summon/catalog artifacts (#2719), we need the same metrics diff command as a regular one #2995. Scenario - if I updated my artifact I'd like to see a metrics diff with an old version and the latest merged version (for CD4ML case).

We agreed to separate metrics files and summon-yaml files to avoid data duplication #2719 (comment). So, some additional implementation might be needed for the summon-artifact.

I suggest to implement it as a part of the dvc metrics diff command as a separate option:

$ dvc metrics diff --artifact resnet50  --artifact resnet18 HEAD^
resnet50: top1-error  -0.00385187
resnet50: top5-error -0.00048518
resnet18: top1-error  -0.00001939
resnet18: top5-error -0.00000719

[efiop]

Blocked by #2995 and #2719

[dberenbaum]

Closed as irrelevant. Please feel free to reopen if I misunderstood.