exp run: option to append unique id to duplicate exp names

[dberenbaum]

For example, right now the behavior is:

$ dvc exp run -n alexnet
Reproduced experiment(s): alexnet
$ dvc exp run -n alexnet
ERROR: Reproduced experiment conflicts with existing experiment alexnet

Desired behavior would be:

$ dvc exp run -n alexnet
Reproduced experiment(s): alexnet
$ dvc exp run -n alexnet
WARNING: Reproduced experiment conflicts with existing experiment alexnet, so unique ID was appended to the experiment name.
Reproduced experiment(s): alexnet-fds23

There might need to be some option to either strictly fail or use a unique ID.

Extracted from #6071 (comment).

For anyone interested, please note the workaround mentioned by @iesahin:

dvc exp run -n myprefix-${RANDOM}

[iesahin]

I didn't mention there but I think this feature might be some p5-too-little-to-consider There are many workarounds like

dvc exp run -n myprefix-$(date +%s | md5sum | cut -c 1-5)

which guarantees unique names more than $RANDOM

TBH I believe it doesn't deserve an option, but can be considered as a part of a broader feature to add formatting to names, like

dvc exp run --format "my-prefix-param1={param1}-param2={param2}-{hash}"

that will also fix the problem we have in #6071

[dberenbaum]

Similarly, there are workarounds for #6071, right? Like:

lr=0.5
dvc exp run -n lr=$lr -S lr=$lr

There could be all kinds of shortcuts to make this easier. Maybe it's better to focus on a doc or blog post showing off how these kinds of tricks could make for a more robust way to organize experiments?

[iesahin]

dvc exp run --format "exp-param1={param1}-metric1={metric1}-{sha}" is a general purpose idea that can be used for #6071 and this. This option can automatically rename the experiment based on runtime parameters and metrics, e.g.,

$ dvc exp list 
exp-param1=0.1-metric1=0.9-ab123a
exp-param1=0.2-metric1=0.8-ce432b
exp-param1=0.3-metric1=0.95-ee1212

I think the team can direct their efforts for this general purpose option instead of fixing experiment naming case by case.

[karajan1001]

@dberenbaum
As in #6674, currently duplicated exp names are forbidden before creating. Do we still need this?

[dberenbaum]

Thanks @karajan1001, that makes it less problematic, although the original feature request from the user in #6071 was to alleviate the burden on users to be responsible for making unique names. It's really just one implementation of #6071. I don't see it as a likely feature to implement anytime soon, so we could set as low priority or move it to a discussion.