Skip to content

Conversation

@pmrowla
Copy link
Contributor

@pmrowla pmrowla commented Aug 31, 2020

Thank you for the contribution - we'll try to review it as soon as possible. πŸ™

Will close #4346.

  • When using --include-metrics/--include-params, columns will be displayed left-to-right in the order specified by the --include option.
  • --sort-by <column_name> and --sort-order asc|desc can be used to sort related experiments on the specified metric or param column.
    • Only nested experiments will be sorted, baseline (formal) commits will still be displayed in the git history order.
    • --sort-order defaults to ascending
    • Only sorting by a single column is supported
❯ dvc exp show --include-params=train.seed,featurize --sort-by=featurize.max_features --sort-order desc --all-commits --no-pager
┏━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓
┃ Experiment            ┃ Created      ┃     auc ┃ train.seed ┃ featurize.max_features ┃ featurize.ngrams ┃
┑━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩
β”‚ workspace             β”‚ -            β”‚ 0.58164 β”‚ 20170428   β”‚ 1500                   β”‚ 3                β”‚
β”‚ executor-tree         β”‚ Aug 28, 2020 β”‚ 0.50822 β”‚ 20170428   β”‚ 1500                   β”‚ 4                β”‚
β”‚ └── *41a9f4b          β”‚ Aug 28, 2020 β”‚       - β”‚ 20170428   β”‚ 456                    β”‚ 5                β”‚
β”‚ 6d18fc1               β”‚ Aug 24, 2020 β”‚ 0.58164 β”‚ 20170428   β”‚ 1500                   β”‚ 3                β”‚
β”‚ β”œβ”€β”€ 5840202           β”‚ Aug 24, 2020 β”‚ 0.50822 β”‚ 20170428   β”‚ 1500                   β”‚ 4                β”‚
β”‚ β”œβ”€β”€ 3c8db11           β”‚ Aug 24, 2020 β”‚ 0.50298 β”‚ 20170428   β”‚ 500                    β”‚ 4                β”‚
β”‚ └── *3b6e442          β”‚ Aug 25, 2020 β”‚       - β”‚ 20170428   β”‚ 123                    β”‚ 5                β”‚
β”‚ 0562956               β”‚ Aug 24, 2020 β”‚ 0.61314 β”‚ 20170428   β”‚ 1500                   β”‚ 2                β”‚
β”‚ β”œβ”€β”€ e6d1716           β”‚ Aug 24, 2020 β”‚ 0.58164 β”‚ 20170428   β”‚ 1500                   β”‚ 3                β”‚
β”‚ └── *c34fd7e          β”‚ Aug 25, 2020 β”‚       - β”‚ 20170428   β”‚ 456                    β”‚ 5                β”‚
β”‚ acd59eb               β”‚ Jul 22, 2020 β”‚ 0.61314 β”‚ 20170428   β”‚ 1500                   β”‚ 2                β”‚
...
β”‚ 0-git-init            β”‚ Jun 20, 2020 β”‚       - β”‚ 20170428   β”‚ 1500                   β”‚ 3                β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

@pmrowla pmrowla added enhancement Enhances DVC A: experiments Related to dvc exp labels Aug 31, 2020
@pmrowla pmrowla self-assigned this Aug 31, 2020
@pmrowla pmrowla requested review from efiop, pared and skshetry August 31, 2020 08:13

if include:
ret = set()
ret = OrderedDict()
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use OrderedDict() with None values to get set-like behavior with preserved ordering

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, use a dict?

Copy link
Contributor Author

@pmrowla pmrowla Aug 31, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the preserved key ordering for regular dict technically isn't guaranteed before 3.7 (it's a cpython implementation behavior in 3.6 that was added to the formal spec in 3.7), so I thought it would be preferred to just use OrderedDict?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, you are right that it is only on >3.6. My point was that, as CPython and Python are closely tied, no one is going to change that behavior in 3.6, so should be safe to just use dict for convenience. Saying that I have no issues with OrderedDict, it was just a suggestion.

Copy link
Collaborator

@skshetry skshetry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Just one suggestion.


if include:
ret = set()
ret = OrderedDict()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, use a dict?

@pared pared added the ui user interface / interaction label Aug 31, 2020
@pmrowla pmrowla merged commit 4f84328 into treeverse:master Aug 31, 2020
@pmrowla pmrowla deleted the experiments-sort branch August 31, 2020 14:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A: experiments Related to dvc exp enhancement Enhances DVC ui user interface / interaction

Projects

None yet

Development

Successfully merging this pull request may close these issues.

experiments show: allow filtering/sorting specific metrics/params

3 participants