exp init: introduce --with={default,live} flag - #6716
Conversation
skshetry
added
ui
There was a problem hiding this comment.
Will need to iterate on prompts separately.
There was a problem hiding this comment.
Okay, we don't need to block this PR, but how about, "Do you want to log checkpoints with dvclive"? We should probably make clear that this will generate checkpoints. Also, we will maybe want to link to dvclive docs.
There was a problem hiding this comment.
Not so sure about this flag, can change to anything better.
There was a problem hiding this comment.
--type? --format? --stage?
There was a problem hiding this comment.
I prefer --type, --stage has a different meaning. Compared to --format, --type is more simpler.
This flag will select the different mode of outputs of stage. With `--with=live`, it'll enable `live` setting in the stage. On interactive mode, it'll always ask user if they want to use live (unless `--with=live` is specified), and then it'll skip metrics/plots question and moves on to ask for path to dvclive path. Also, if live setting is enabled in the stage, it'll set the `models` to be a checkpoint output.
skshetry
changed the title
--with={default,live} flag--with={default,live} flag
dberenbaum
left a comment
dberenbaum
left a comment
There was a problem hiding this comment.
It feels a little clunky to me to ask about dvclive in the middle of entering paths. I think it's better to require --with non-interactively (--name, --run, etc. aren't covered). It also fits with the description of --interactive: Prompt for values that are not provided.
Minor note: should --template be removed at this point?
There was a problem hiding this comment.
--type? --format? --stage?
There was a problem hiding this comment.
Okay, we don't need to block this PR, but how about, "Do you want to log checkpoints with dvclive"? We should probably make clear that this will generate checkpoints. Also, we will maybe want to link to dvclive docs.
| { | ||
| "cmd": "[b]Command[/b] to execute", | ||
| "code": "Path to a [b]code[/b] file/directory", | ||
| "data": "Path to a [b]data[/b] file/directory", |
There was a problem hiding this comment.
Why not include params here?
There was a problem hiding this comment.
I wanted to separate dependencies and then pipeline artifacts, but I don't need to. 😄
| if not live: | ||
| ui.error_write() | ||
| live = Confirm.ask( | ||
| "Do you want to log training steps iteratively with " |
There was a problem hiding this comment.
I don't think this makes sense for dvc exp init -i --with=default.
I felt the same way, I thought it was just me. I'll make it a requirement then. :) |
dberenbaum
left a comment
dberenbaum
left a comment
There was a problem hiding this comment.
LGTM!
My only concern is that we may need to come back to the idea of custom templates again in the future. For example, users who mainly plot image files might want plots with cache: true. We don't want to turn this into dvc stage add, but it seems useful to enable a consistent stage structure, especially if it could be shared across projects. This seems like it might require some redesign if we ever need it, but for now it's just a note to keep in mind.
skshetry
changed the title
--with={default,live} flag--with={default,live} flag
2 participants
This flag will select the different mode of outputs of stage.
With
--with=live, it'll enablelivesetting in the stage.On interactive mode, it'll always ask user if they want to
use live (unless
--with=liveis specified), and then it'llskip metrics/plots question and moves on to ask for path to
dvclive path.
Also, if live setting is enabled in the stage, it'll set the
modelsto be a checkpoint output.