guide: Managing Experiments section(s)

[dashohoxha]

UPDATE: Skip to #816 (comment)

Add a User Guide page that explains how to manage experiments:

• by tags
• by branches
• by folders (check some details and ideas in user-guide: add "folders" way of experimentation #159)
• hybrid (mixing tags/branches/folders)

Documenting first the existing methods may also help the discussions about adding new methods for experiment management (iterative/dvc#2799), because my impression is that most of what is being discussed there can already be accomplished with the existing tools.

Another related issue: #159

[dashohoxha]

I started the PR #828 to close this issue, but actually I don't have a very clear idea about how to best explain it, so I would appreciate some help from the @iterative/engineering team.

• https://dvc-org-pr-828.herokuapp.com/doc/user-guide/experiments/tags
• https://dvc-org-pr-828.herokuapp.com/doc/user-guide/experiments/branches
• https://dvc-org-pr-828.herokuapp.com/doc/user-guide/experiments/dirs
• https://dvc-org-pr-828.herokuapp.com/doc/user-guide/experiments/mixed

The first two cases are almost clear, and also there are some examples in the Get Started and the Tutorials, which might be summarized on these pages.

For the case of managing experiments by directories, there is a very simple example at the end of this interactive tutorial: https://katacoda.com/dvc/courses/basics/pipelines
But it does not look like a realistic example. You have much more experience than me with DVC and with helping users to use it in their ML scenarios. If you know some more realistic example (or you can make up one), please let me know.

The last case, where we can mix different kinds of experiment management techniques, is even more difficult for me. Conceptually I know how it should work, but it is not so easy to come up with a convincing example, which is both simple and looks kind of realistic. I have no experience with ML, so it is too difficult for me to make up something. If you can think up something suitable please let me know.

[shcheklein]

I think the original intention was more or less correct in this ticket - User Guide page like this makes sense to me.

[jorgeorpinel]

OK but in #72 (comment) you said

section like Managing Experiments should be within Get started, Use Cases, and User Guide at the top level

but I guess it depends on what/how much content we will have for this topic. Could be in UG at first.