Re-run previous experiments at current HEAD

[edwardwbarber]

During my use of dvc exp I have repeatedly run into situations where I would like to adjust post-processing methods based on the results of previous experiments (for example, adding extra metrics to my output dataset). Ideally, I would like to be able to commit those changes, then re-run previous experiments against those changes. To put it another way, the ability to rebase existing experiments against my current HEAD, then re-run them.

@pmrowla suggested a good manual approach to it in Discord:

$ git merge --squash --no-commit <past_experiment_sha>
$ dvc exp run

It would be convenient to be able to queue up all the experiments from a previous commit instead of retrieving experiments one by one. The most common use case for me would be to queue experiments from HEAD^ into HEAD.

I will note this gets sticky if there are changes to the same files between the experiment and the most recent commit (or I suppose anywhere you would have an issue calling git merge). Seems like an easy trap to fall into if you aren't careful to keep input parameter changes separate from pre/post-processing codes, although seems like that should be considered best practice anyway.

Cheers,
-Edward

[dberenbaum]

Might be related to the suggestion of dvc exp queue --save/--restore in #5615.