Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

View experiment via CLI #10

Closed
eponvert opened this issue Sep 2, 2015 · 0 comments
Closed

View experiment via CLI #10

eponvert opened this issue Sep 2, 2015 · 0 comments
Assignees
Milestone

Comments

@eponvert
Copy link
Contributor

eponvert commented Sep 2, 2015

Requirements

View an experiment with high level results analysis with CLI. Terminal output must fit within 80 chars horizontal.

For classification experiments, include overall accuracy and per-label + overall precision, recall, f-score. For less than 10 classes also include a confusion matrix.

Wireframes

Classification experiments:

$ levar-cli view experiment adult exp-0101
Experiment:  exp-0101
Dataset:     adult
Type:        classify
Predictions: 32563 (full dataset coverage)

Results:

Overall accuracy: .931

 Label         | Prec | Rec  | F1
---------------|------|------|------
 <=50K         | .952 | .917 | .934
 >50K          | .871 | .970 | .917
 Macro-Avg     | .912 | .944 | .926

Confusion matrix (columns give predicted counts; rows give gold counts)

Labels        |    <=50K |       >50K |     Totals
--------------|----------|------------|-----------
<=50K         |    22690 |       1142 |      23832
>50K          |     1118 |       7611 |       8729
Totals        |    23808 |       8753 |      32561

Regression experiments:

$ levar-cli view experiment winequality-red exp-234
Experiment:  exp-234
Dataset:     winequality-red
Type:        regression
Predictions: 1599 (full dataset coverage)

Results:
- RMSE:              4.2
- Mean abs error:    4.8
- Median abs error:  3.9
- 10-percentile err: 1.0
- 90-percentile err: 7.1
@eponvert eponvert added this to the v0.1 milestone Sep 2, 2015
@eponvert eponvert self-assigned this Sep 2, 2015
eponvert added a commit that referenced this issue Oct 6, 2015
Basic experiment upload, list, view, and DB/API plumbing in
place. Also:
- Revamped API routes
- Revamped experiment <=> dataset relationship
- Revamped CLI help docs

Progress on #9 #10 #11
eponvert added a commit that referenced this issue Oct 7, 2015
- Revised CLI to be like `lever-cli list datasets`
- Support for renaming datasets
- Support for deleting experiments
- Scaladoc for DB experiments interface
- DB/data transfer support for classification results

Fixes #21
Fixes #9
Progress on #10 and #11
eponvert added a commit that referenced this issue Oct 8, 2015
@eponvert eponvert closed this as completed Oct 8, 2015
eponvert added a commit that referenced this issue Oct 8, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant