-
Notifications
You must be signed in to change notification settings - Fork 1.3k
Closed
Labels
A: experimentsRelated to dvc expRelated to dvc expfeature requestRequesting a new featureRequesting a new featurep3-nice-to-haveIt should be done this or next sprintIt should be done this or next sprintuiuser interface / interactionuser interface / interaction
Description
If I had run an experiment with multiple checkpoints and I run dvc exp apply {some_checkpoint}, there is no indicator in the table generated by dvc exp show to quickly identify the applied checkpoint.
I think it would be nice to somehow highlight the applied checkpoint.
One option would be to add some indicator similar to how queued experiments have an * indicator.
I.e, given the following table:
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Experiment ┃ Created ┃ step ┃ loss ┃ acc ┃ seed ┃ lr ┃ weight_decay ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━┩
│ workspace │ - │ 9 │ 0.086935 │ 0.9733 │ 473987 │ 0.001 │ 0 │
│ full_pipeline │ May 13, 2021 │ - │ - │ - │ 473987 │ 0.001 │ 0 │
│ │ ╓ 8c16052 [exp-8ee4f] │ 09:14 AM │ 9 │ 0.086935 │ 0.9733 │ 473987 │ 0.001 │ 0 │
│ │ ╟ c47871d │ 09:14 AM │ 8 │ 0.080751 │ 0.9759 │ 473987 │ 0.001 │ 0 │
│ │ ╟ a68df6f │ 09:14 AM │ 7 │ 0.077116 │ 0.9752 │ 473987 │ 0.001 │ 0 │
│ │ ╟ abe7304 │ 09:13 AM │ 6 │ 0.093463 │ 0.9716 │ 473987 │ 0.001 │ 0 │
│ │ ╟ f9ab8a9 │ 09:13 AM │ 5 │ 0.12796 │ 0.9619 │ 473987 │ 0.001 │ 0 │
│ │ ╟ 2b88312 │ 09:13 AM │ 4 │ 0.15237 │ 0.954 │ 473987 │ 0.001 │ 0 │
│ │ ╟ ad90693 │ 09:12 AM │ 3 │ 0.19057 │ 0.9405 │ 473987 │ 0.001 │ 0 │
│ │ ╟ 4140811 │ 09:12 AM │ 2 │ 0.25379 │ 0.9255 │ 473987 │ 0.001 │ 0 │
│ │ ╟ ef176d1 │ 09:12 AM │ 1 │ 0.3144 │ 0.9035 │ 473987 │ 0.001 │ 0 │
│ ├─╨ 76a651d │ 09:11 AM │ 0 │ 0.49538 │ 0.8428 │ 473987 │ 0.001 │ 0 │
└─────────────────────────┴──────────────┴──────┴──────────┴────────┴────────┴───────┴──────────────┘Running dvc exp apply c47871d would result in the following (being <- the indicator):
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Experiment ┃ Created ┃ step ┃ loss ┃ acc ┃ seed ┃ lr ┃ weight_decay ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━┩
│ workspace │ - │ 8 │ 0.080751 │ 0.9759 │ 473987 │ 0.001 │ 0 │
│ full_pipeline │ May 13, 2021 │ - │ - │ - │ 473987 │ 0.001 │ 0 │
│ │ ╓ 8c16052 [exp-8ee4f] │ 09:14 AM │ 9 │ 0.086935 │ 0.9733 │ 473987 │ 0.001 │ 0 │
│ │ ╟ c47871d <- │ 09:14 AM │ 8 │ 0.080751 │ 0.9759 │ 473987 │ 0.001 │ 0 │
│ │ ╟ a68df6f │ 09:14 AM │ 7 │ 0.077116 │ 0.9752 │ 473987 │ 0.001 │ 0 │
│ │ ╟ abe7304 │ 09:13 AM │ 6 │ 0.093463 │ 0.9716 │ 473987 │ 0.001 │ 0 │
│ │ ╟ f9ab8a9 │ 09:13 AM │ 5 │ 0.12796 │ 0.9619 │ 473987 │ 0.001 │ 0 │
│ │ ╟ 2b88312 │ 09:13 AM │ 4 │ 0.15237 │ 0.954 │ 473987 │ 0.001 │ 0 │
│ │ ╟ ad90693 │ 09:12 AM │ 3 │ 0.19057 │ 0.9405 │ 473987 │ 0.001 │ 0 │
│ │ ╟ 4140811 │ 09:12 AM │ 2 │ 0.25379 │ 0.9255 │ 473987 │ 0.001 │ 0 │
│ │ ╟ ef176d1 │ 09:12 AM │ 1 │ 0.3144 │ 0.9035 │ 473987 │ 0.001 │ 0 │
│ ├─╨ 76a651d │ 09:11 AM │ 0 │ 0.49538 │ 0.8428 │ 473987 │ 0.001 │ 0 │
└─────────────────────────┴──────────────┴──────┴──────────┴────────┴────────┴───────┴──────────────┘Alternatively/additionally, the workspace row name could be altered to reflect the applied checkpoint:
┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Experiment ┃ Created ┃ step ┃ loss ┃ acc ┃ seed ┃ lr ┃ weight_decay ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━┩
│ workspace [c47871d] │ - │ 8 │ 0.080751 │ 0.9759 │ 473987 │ 0.001 │ 0 │
│ full_pipeline │ May 13, 2021 │ - │ - │ - │ 473987 │ 0.001 │ 0 │
│ │ ╓ 8c16052 [exp-8ee4f] │ 09:14 AM │ 9 │ 0.086935 │ 0.9733 │ 473987 │ 0.001 │ 0 │
│ │ ╟ c47871d <- │ 09:14 AM │ 8 │ 0.080751 │ 0.9759 │ 473987 │ 0.001 │ 0 │
│ │ ╟ a68df6f │ 09:14 AM │ 7 │ 0.077116 │ 0.9752 │ 473987 │ 0.001 │ 0 │
│ │ ╟ abe7304 │ 09:13 AM │ 6 │ 0.093463 │ 0.9716 │ 473987 │ 0.001 │ 0 │
│ │ ╟ f9ab8a9 │ 09:13 AM │ 5 │ 0.12796 │ 0.9619 │ 473987 │ 0.001 │ 0 │
│ │ ╟ 2b88312 │ 09:13 AM │ 4 │ 0.15237 │ 0.954 │ 473987 │ 0.001 │ 0 │
│ │ ╟ ad90693 │ 09:12 AM │ 3 │ 0.19057 │ 0.9405 │ 473987 │ 0.001 │ 0 │
│ │ ╟ 4140811 │ 09:12 AM │ 2 │ 0.25379 │ 0.9255 │ 473987 │ 0.001 │ 0 │
│ │ ╟ ef176d1 │ 09:12 AM │ 1 │ 0.3144 │ 0.9035 │ 473987 │ 0.001 │ 0 │
│ ├─╨ 76a651d │ 09:11 AM │ 0 │ 0.49538 │ 0.8428 │ 473987 │ 0.001 │ 0 │
└─────────────────────────┴──────────────┴──────┴──────────┴────────┴────────┴───────┴──────────────┘Metadata
Metadata
Assignees
Labels
A: experimentsRelated to dvc expRelated to dvc expfeature requestRequesting a new featureRequesting a new featurep3-nice-to-haveIt should be done this or next sprintIt should be done this or next sprintuiuser interface / interactionuser interface / interaction