[MRG + 1] Fix ValueError in LabelEncoder when using inverse_transform on unseen labels #9816

newey01c · Sep 21, 2017

Reference Issue

Fix #9812

amueller · Sep 21, 2017

amueller reviewed Sep 21, 2017

View changes

looks good to me.

amueller · Sep 21, 2017

sklearn/preprocessing/label.py

@@ -148,8 +149,9 @@ def inverse_transform(self, y):
        check_is_fitted(self, 'classes_')

        diff = np.setdiff1d(y, np.arange(len(self.classes_)))
-        if diff:
-            raise ValueError("y contains new labels: %s" % str(diff))
+        if len(diff):


This len is the fix, right?

Correct - that's all that was needed to produce a sensible error 😄

amueller · Sep 21, 2017

sklearn/preprocessing/tests/test_label.py

-    assert_raises(ValueError, le.inverse_transform, [-1])
+    le.fit([1, 2, 3, -1, 1])
+    msg = "contains previously unseen labels"
+    assert_raise_message(ValueError, msg, le.inverse_transform, [-2])


I find the organization of the tests a bit weird but not your fault. The test that it actually works if they are present is way at the top of the file.

Happy to reorganise tomorrow if you are able to give me some pointers - I'm not very familiar with the testing structure of sklearn as this is my first issue.

it's fine, I think.

LGTM, merging, thanks a lot @newey01c!

lesteve · Sep 21, 2017

LGTM, merging, thanks a lot @newey01c!

This is missing a whats_new entry. I'll pull it into my 0.19.1 branch and write an entry there

jnothman · Sep 24, 2017

This is missing a whats_new entry. I'll pull it into my 0.19.1 branch and write an entry there

The issue appears to be persistent - I am using LabelEncoder. Here is my stack trace:

 File "ann.py", line 71, in <module>
    X_train, X_test, y_train, y_test = get_dataset("Churn_Modelling.csv", 3, 13, 13)
  File "ann.py", line 28, in get_dataset
    encoder.fit(labels)
  File "/home/yolopc/.local/lib/python3.5/site-packages/sklearn/preprocessing/label.py", line 96, in fit
    self.classes_ = np.unique(y)
  File "/home/yolopc/.local/lib/python3.5/site-packages/numpy/lib/arraysetops.py", line 210, in unique
    return _unique1d(ar, return_index, return_inverse, return_counts)
  File "/home/yolopc/.local/lib/python3.5/site-packages/numpy/lib/arraysetops.py", line 277, in _unique1d
    ar.sort()
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Do you have any suggestions?

vdaita · Feb 3, 2018

The issue appears to be persistent - I am using LabelEncoder. Here is my stack trace:

 File "ann.py", line 71, in <module>
    X_train, X_test, y_train, y_test = get_dataset("Churn_Modelling.csv", 3, 13, 13)
  File "ann.py", line 28, in get_dataset
    encoder.fit(labels)
  File "/home/yolopc/.local/lib/python3.5/site-packages/sklearn/preprocessing/label.py", line 96, in fit
    self.classes_ = np.unique(y)
  File "/home/yolopc/.local/lib/python3.5/site-packages/numpy/lib/arraysetops.py", line 210, in unique
    return _unique1d(ar, return_index, return_inverse, return_counts)
  File "/home/yolopc/.local/lib/python3.5/site-packages/numpy/lib/arraysetops.py", line 277, in _unique1d
    ar.sort()
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Do you have any suggestions?

As noted in #10552, this was accidentally not included in the 0.19.1 release. Using the development version of scikit-learn will make it work.

jnothman · Feb 3, 2018

As noted in #10552, this was accidentally not included in the 0.19.1 release. Using the development version of scikit-learn will make it work.

I tried to follow the instructions on the website, but I could not import any scikit-learn libraries. I also ran pip install -e, but I got the following error: IndexError: list index out of range. Do you have any pointers? I am using Python 2.7 on Ubuntu 16.04.

On Sat, Feb 3, 2018 at 6:41 PM Joel Nothman ***@***.***> wrote: As noted in #10552 <#10552>, this was accidentally not included in the 0.19.1 release. Using the development version of scikit-learn will make it work. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9816 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AV4hUWvS5UzS3P1532AQtZOjXe8bnh6Sks5tRO68gaJpZM4PfUrv> .

-- Vijay

vdaita · Feb 4, 2018

I tried to follow the instructions on the website, but I could not import any scikit-learn libraries. I also ran pip install -e, but I got the following error: IndexError: list index out of range. Do you have any pointers? I am using Python 2.7 on Ubuntu 16.04.

On Sat, Feb 3, 2018 at 6:41 PM Joel Nothman ***@***.***> wrote: As noted in #10552 <#10552>, this was accidentally not included in the 0.19.1 release. Using the development version of scikit-learn will make it work. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9816 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AV4hUWvS5UzS3P1532AQtZOjXe8bnh6Sks5tRO68gaJpZM4PfUrv> .

-- Vijay

not sure without a full traceback what index was out of range. you can try pip install https://g <https://hith> ithub.com/scikit-learn/scikit-learn/archive/master.zip to install the latest development version.

…

jnothman · Feb 4, 2018

not sure without a full traceback what index was out of range. you can try pip install https://g <https://hith> ithub.com/scikit-learn/scikit-learn/archive/master.zip to install the latest development version.

…

Thank you - I found an alternate solution.

On Sun, Feb 4, 2018 at 5:00 PM Joel Nothman ***@***.***> wrote: not sure without a full traceback what index was out of range. you can try pip install https://g <https://hith> ithub.com/scikit-learn/scikit-learn/archive/master.zip to install the latest development version. > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9816 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AV4hUXCZMFcf1ARIl9XGIRyNj2IdsGZDks5tRihmgaJpZM4PfUrv> .

-- Vijay

vdaita · Feb 5, 2018

Thank you - I found an alternate solution.

On Sun, Feb 4, 2018 at 5:00 PM Joel Nothman ***@***.***> wrote: not sure without a full traceback what index was out of range. you can try pip install https://g <https://hith> ithub.com/scikit-learn/scikit-learn/archive/master.zip to install the latest development version. > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#9816 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AV4hUXCZMFcf1ARIl9XGIRyNj2IdsGZDks5tRihmgaJpZM4PfUrv> .

-- Vijay

Fix incorrect ValueError, add extra test case

Loading status checks…

397a8a2

newey01c referenced this pull request Sep 21, 2017
Closed
Fix ValueError in LabelEncoder when using inverse_transform on unseen labels #9813

Wrap error message to <79 chars

Loading status checks…

effbd45

amueller changed the title from Fix ValueError in LabelEncoder when using inverse_transform on unseen labels to [MRG + 1] Fix ValueError in LabelEncoder when using inverse_transform on unseen labels Sep 21, 2017

jnothman added this to the 0.19.1 milestone Sep 24, 2017

newey01c deleted the newey01c:patch_9812 branch Sep 25, 2017

maskani-moh added a commit to maskani-moh/scikit-learn that referenced this pull request Nov 15, 2017

[MRG + 1] Fix ValueError in LabelEncoder when using inverse_transform…
… on unseen labels (#9816)

9985b80

jwjohnson314 pushed a commit to jwjohnson314/scikit-learn that referenced this pull request Dec 18, 2017

[MRG + 1] Fix ValueError in LabelEncoder when using inverse_transform…
… on unseen labels (#9816)

35f2151

newey01c referenced this pull request Dec 27, 2017
Closed
LabelEncoder: ValueError: The truth value of an array with more than one element is ambiguous #9812

jnothman referenced this pull request Jan 10, 2018
Closed
Numpy deprecation warning: sklearn/preprocessing/label.py:151 #10449

vadasri referenced this pull request Jan 30, 2018
Closed
Inverse Transform for Label Encoder fails when more than one new values are present. #10552

qinhanmin2014 referenced this pull request Jan 31, 2018
Merged
DOC move labelencoder what's new from 0.19 to 0.20 #10556

ayashjorden referenced this pull request Mar 10, 2018
Closed
empty parsed text #846

fakesecret referenced this pull request Mar 21, 2018
Closed
gives warning while running the examples #293

ianozsvald referenced this pull request Aug 12, 2018
Open
Conflict with PermutationImportance, DataFrame and XGBoost (with workaround) #256

scikit-learn/scikit-learn

Join GitHub today

[MRG + 1] Fix ValueError in LabelEncoder when using inverse_transform on unseen labels #9816

Conversation

newey01c commented Sep 21, 2017

Reference Issue

newey01c referenced this pull request Sep 21, 2017

amueller reviewed Sep 21, 2017 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

amueller changed the title from Fix ValueError in LabelEncoder when using inverse_transform on unseen labels to [MRG + 1] Fix ValueError in LabelEncoder when using inverse_transform on unseen labels Sep 21, 2017

This comment has been minimized.

lesteve commented Sep 21, 2017

Hide details View details lesteve merged commit c554aad into scikit-learn:master Sep 21, 2017 6 checks passed

6 checks passed

This comment has been minimized.

jnothman commented Sep 24, 2017

jnothman added this to the 0.19.1 milestone Sep 24, 2017

newey01c deleted the newey01c:patch_9812 branch Sep 25, 2017

maskani-moh added a commit to maskani-moh/scikit-learn that referenced this pull request Nov 15, 2017

jwjohnson314 pushed a commit to jwjohnson314/scikit-learn that referenced this pull request Dec 18, 2017

newey01c referenced this pull request Dec 27, 2017

jnothman referenced this pull request Jan 10, 2018

vadasri referenced this pull request Jan 30, 2018

qinhanmin2014 referenced this pull request Jan 31, 2018

This comment has been minimized.

vdaita commented Feb 3, 2018 • edited

This comment has been minimized.

jnothman commented Feb 3, 2018

This comment has been minimized.

vdaita commented Feb 4, 2018

This comment has been minimized.

jnothman commented Feb 4, 2018

This comment has been minimized.

vdaita commented Feb 5, 2018

ayashjorden referenced this pull request Mar 10, 2018

fakesecret referenced this pull request Mar 21, 2018

ianozsvald referenced this pull request Aug 12, 2018

amueller reviewed Sep 21, 2017

View changes

lesteve merged commit `c554aad` into scikit-learn:master Sep 21, 2017
6 checks passed

vdaita commented Feb 3, 2018 •

edited