Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

supress value error when label contains one label #157

Merged
merged 3 commits into from
May 19, 2018

Conversation

corochann
Copy link
Member

No description provided.

@codecov-io
Copy link

codecov-io commented Apr 24, 2018

Codecov Report

Merging #157 into master will increase coverage by 0.65%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master     #157      +/-   ##
==========================================
+ Coverage   77.92%   78.57%   +0.65%     
==========================================
  Files          79       80       +1     
  Lines        3388     3468      +80     
==========================================
+ Hits         2640     2725      +85     
+ Misses        748      743       -5

@corochann corochann mentioned this pull request Apr 24, 2018
@@ -73,6 +74,10 @@ class ROCAUCEvaluator(Evaluator):
are considered as negative.
ignore_labels (int or list or None): labels to be ignored.
`None` is used to not ignore all labels.
raise_value_error (bool): If False, `ValueError` caused by
`roc_auc_score` calculation is suppressed and ignored with warning
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

with a warning message.

@@ -73,6 +74,10 @@ class ROCAUCEvaluator(Evaluator):
are considered as negative.
ignore_labels (int or list or None): labels to be ignored.
`None` is used to not ignore all labels.
raise_value_error (bool): If False, `ValueError` caused by
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

enclose False with backquotes.

else:
# This is usually caused by the following
# Only one class present in y_true.
# ROC AUC score is not defined in that case
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It describe when roc_auc_score is raised. So I think we should move to l.145.

@corochann
Copy link
Member Author

updated

try:
roc_auc = metrics.roc_auc_score(t_total, y_total)
except ValueError as e:
# `ValueError` is usually caused by the following
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems you list only one situation where ValueError is raised. So, by the following seems strange.

roc_auc = metrics.roc_auc_score(t_total, y_total)
except ValueError as e:
# `ValueError` is usually caused by the following
# Only one class present in y_true.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not capitalize here because the previous sentence is not completed.

@corochann
Copy link
Member Author

ROCAUCEvaluator will raise ValueError when only one label is contained in the dataset.
This will stop the whole script, but this is evaluation script and not affect to training.
I added a option to ignore this error so that we can keep running the training.

It has another benefit that we can make test example faster.

@corochann
Copy link
Member Author

Could you review @delta2323 @mottodora ?

@corochann corochann requested a review from mottodora May 13, 2018 23:20
@delta2323
Copy link
Member

LGTM

@delta2323 delta2323 merged commit 7ac45b1 into chainer:master May 19, 2018
@corochann corochann deleted the rocaucevaluator_supress_error branch May 19, 2018 12:39
@mottodora mottodora added this to the 0.4.0 milestone Jul 3, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants