Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

Support single array input for metric #9930

Merged
merged 7 commits into from
Mar 13, 2018
Merged

Support single array input for metric #9930

merged 7 commits into from
Mar 13, 2018

Conversation

hetong007
Copy link
Contributor

Description

Fix #9865 by reshaping both label and pred.

Checklist

Essentials

  • Passed code style checking (make lint)
  • Changes are complete (i.e. I finished coding on this PR)
  • All changes have test coverage:
  • Unit tests are added for small changes to verify correctness (e.g. adding a new operator)
  • Nightly tests are added for complicated/long-running ones (e.g. changing distributed kvstore)
  • Build tests will be added for build configuration changes (e.g. adding a new build option with NCCL)
  • Code is well-documented:
  • For user-facing API changes, API doc string has been updated.
  • For new C++ functions in header files, their functionalities and arguments are documented.
  • For new examples, README.md is added to explain the what the example does, the source of the dataset, expected performance on test set and reference to the original paper if applicable
  • To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes

  • Feature1, tests, (and when applicable, API doc)
  • Feature2, tests, (and when applicable, API doc)

Comments

  • If this change is a backward incompatible change, why must this change be made.
  • Interesting edge cases to note here

@hetong007 hetong007 requested a review from szha as a code owner February 28, 2018 23:59
sxjscience
sxjscience reviewed Mar 1, 2018
View reviewed changes
python/mxnet/metric.py
@@ -793,6 +793,8 @@ def update(self, labels, preds):

if len(label.shape) == 1:
label = label.reshape(label.shape[0], 1)
if len(pred.shape) == 1:
pred = pred.reshape(pred.shape[0], 1)

Copy link
Member

@sxjscience sxjscience Mar 1, 2018 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also revise the check_label_shapes above to be check_label_shapes(labels, preds, 1) to force the labels and preds have the same shape. Currently the default behavior is to just match the ndim.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to set shape=0 by default?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure. I think shape=1 is more suitable to be the default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@piiswrong can we have your input?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the check_label_shapes function should be redesigned.

  1. it should test if labels is a single NDArray instead of a list, and wrap it with a list labels=[labels]
  2. 0/1 is not pythonic. Use boolean switches

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we check both length and shape? I assume they shoud always match.

@szha szha added this to To Do in Metric via automation Mar 1, 2018
@piiswrong
Copy link
Contributor

when labels and preds are NDArray instead of list NDArray, the for loop is superfluous

@hetong007
Copy link
Contributor Author

@piiswrong The for loop is fixed.

@CodingCat
Copy link
Contributor

Hi, the community has passed to vote about associating the code changes with JIRA (https://lists.apache.org/thread.html/ab22cf0e35f1bce2c3bf3bec2bc5b85a9583a3fe7fd56ba1bbade55f@%3Cdev.mxnet.apache.org%3E)

We have updated the guidelines for contributors in https://cwiki.apache.org/confluence/display/MXNET/Development+Process, please ensure that you have created a JIRA at https://issues.apache.org/jira/projects/MXNET/issues/ to describe your work in this pull request and include the JIRA title in your PR as [MXNET-xxxx] your title where MXNET-xxxx is the JIRA id

Thanks!

@szha szha merged commit dd1f21b into apache:master Mar 13, 2018
Metric automation moved this from To Do to Done Mar 13, 2018
jinhuang415 pushed a commit to jinhuang415/incubator-mxnet that referenced this pull request Mar 30, 2018
@hetong007
Support single array input for metric (apache#9930)
fbe2c6c 
* fix apache#9865

* add unittest

* fix format

* fix format

* fix superfluous loop in metric

* fix lint
rahul003 pushed a commit to rahul003/mxnet that referenced this pull request Jun 4, 2018
@hetong007 @szha
Support single array input for metric (apache#9930)
8e9cdb3 
* fix apache#9865

* add unittest

* fix format

* fix format

* fix superfluous loop in metric

* fix lint
zheng-da pushed a commit to zheng-da/incubator-mxnet that referenced this pull request Jun 28, 2018
@hetong007 @zheng-da
Support single array input for metric (apache#9930)
f1a09c3 
* fix apache#9865

* add unittest

* fix format

* fix format

* fix superfluous loop in metric

* fix lint
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@piiswrong piiswrong piiswrong left review comments

@sxjscience sxjscience sxjscience left review comments

@marcoabreu marcoabreu marcoabreu approved these changes

@szha szha Awaiting requested review from szha szha is a code owner

Assignees
No one assigned
Labels
None yet
Projects
No open projects
Metric
  
Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Confusing behavior of some evaluation metrics
6 participants
@hetong007 @piiswrong @CodingCat @sxjscience @marcoabreu @szha