`dvc exp show`'s table: shifts params values to metrics columns after renaming metrics file

[agrizzli]

Bug Report

dvc exp show -A or dvc show -n N shows values of params in wrong columns.

Description

Phase 1. I had a simple training stage in pipelines/example/dvc.yaml without metrics tracking, but with params tracking.

dummy_train:
    wdir: ../..
    cmd: python scripts/example/dummy_train.py
    deps:
    - scripts/example/dummy_train.py
    params:
    - pipelines/example/params.yaml:
      - test.value
    - pipelines/example/custom_params.yaml:
      - custom.value

Phase 2. I ran some experiments and then introduced metrics file pipelines/example/scores.json.

dummy_train:
    wdir: ../..
    cmd: python scripts/example/dummy_train.py
    deps:
    - scripts/example/dummy_train.py
    params:
    - pipelines/example/params.yaml:
      - test.value
    - pipelines/example/custom_params.yaml:
      - custom.value
    metrics:
    - pipelines/example/scores.json:
        cache: false

Phase 3. I used this stage to run some more experiments and at some point I removed pipelines/example/scores.json and added pipelines/example/metrics.json instead and the ran further experiments.

Results. Now, when I use dvc exp show -A I see values from params' columns in the columns for metrics pipelines/example/scores.json beginning from the timepoint when I introduced metrics in experiments (phase 2). Params test.value and custom.value are shown in columns pipelines/example/scores.json:mean_acc and pipelines/example/scores.json:mean_loss.

This happens only if the list of experiments is "long enough". As soon as I use, e.g dvc exp show -n 8 everything is fine. This matches only the experiments from phase 3, where the obsolete pipelines/example/scores.json was already removed from the stage and Git repository.

Reproduce

1. Create a stage in pipelines/example/dvc.yaml as described in phase 2.
2. dvc repro pipelines/example/dvc.yaml
3. git add pipelines/example/ and git commit
4. Amend dvc.yaml as said in phase 3 by renaming scores.json to metrics.json
5. dvc repro pipelines/example/dvc.yaml
6. git rm pipelines/example/scores.json
7. git add pipelines/example/
8. dvc exp show -A

Expected

Expected to see values of params in columns for params as show in the 2nd image above.

Environment information

Output of dvc doctor:

DVC version: 2.8.3 (pip)
---------------------------------
Platform: Python 3.8.10 on Linux-5.8.0-63-generic-x86_64-with-glibc2.32
Supports:
	hdfs (fsspec = 2021.11.0, pyarrow = 5.0.0),
	webhdfs (fsspec = 2021.11.0),
	http (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.6),
	https (aiohttp = 3.7.4.post0, aiohttp-retry = 2.4.6),
	ssh (sshfs = 2021.11.1)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p3
Caches: local
Remotes: ssh, ssh
Workspace directory: ext4 on /dev/nvme0n1p3
Repo: dvc, git

[agrizzli]

I have now experienced that the described shift in the table does not only happen when renaming files, but even when introducing new metrics. For runs with less metrics, there will be a shift in column values, making it completely unreadable.

[daavoo]

Hi @agrizzli ! I'm going to try to reproduce and get back to you

[daavoo]

Able to reproduce. The bug also affects a few other scenarios