FEA implements SMOTEN to handle nominal categorical features by glemaitre · Pull Request #802 · scikit-learn-contrib/imbalanced-learn

glemaitre · 2021-02-15T18:57:04Z

closes #565

Implements SMOTE for nominal categorical features only. It uses the VDM distance and a majority vote regarding the per feature to create new categories from the k-NN.

pep8speaks · 2021-02-15T18:57:11Z

Hello @glemaitre! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

In the file imblearn/over_sampling/_smote.py:

Line 798:89: E501 line too long (91 > 88 characters)

In the file imblearn/over_sampling/tests/test_smoten.py:

Line 43:9: W503 line break before binary operator
Line 44:9: W503 line break before binary operator
Line 45:9: W503 line break before binary operator
Line 46:9: W503 line break before binary operator

Comment last updated at 2021-02-15 22:55:02 UTC

lgtm-com · 2021-02-15T19:24:35Z

This pull request introduces 1 alert when merging 049dde9 into 6155658 - view on LGTM.com

new alerts:

1 for Signature mismatch in overriding method

lgtm-com · 2021-02-15T20:07:29Z

This pull request introduces 1 alert when merging fee46f7 into 6155658 - view on LGTM.com

new alerts:

1 for Signature mismatch in overriding method

lgtm-com · 2021-02-15T20:58:47Z

This pull request introduces 1 alert when merging 8626040 into d9ba4af - view on LGTM.com

new alerts:

1 for Signature mismatch in overriding method

codecov · 2021-02-15T22:00:07Z

Codecov Report

Merging #802 (a66bfaa) into master (b6621f9) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #802      +/-   ##
==========================================
- Coverage   98.63%   98.58%   -0.05%     
==========================================
  Files          89       90       +1     
  Lines        5934     6007      +73     
  Branches      499      503       +4     
==========================================
+ Hits         5853     5922      +69     
- Misses         80       84       +4     
  Partials        1        1

Impacted Files	Coverage Δ
imblearn/over_sampling/_random_over_sampler.py	`100.00% <ø> (ø)`
imblearn/over_sampling/__init__.py	`100.00% <100.00%> (ø)`
imblearn/over_sampling/_adasyn.py	`92.30% <100.00%> (-6.11%)`	⬇️
imblearn/over_sampling/_smote.py	`97.60% <100.00%> (+0.29%)`	⬆️
imblearn/over_sampling/tests/test_smoten.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6621f9...a66bfaa. Read the comment docs.

lgtm-com · 2021-02-15T22:27:55Z

This pull request introduces 1 alert when merging 516b33c into 326119b - view on LGTM.com

new alerts:

1 for Signature mismatch in overriding method

lgtm-com · 2021-02-15T23:22:24Z

This pull request introduces 1 alert when merging a66bfaa into b6621f9 - view on LGTM.com

new alerts:

1 for Signature mismatch in overriding method

chkoar · 2021-02-16T06:37:49Z

-    6. ADASYN - Adaptive synthetic sampling approach for imbalanced learning [15]_
-    7. KMeans-SMOTE [17]_
-    8. ROSE - Random OverSampling Examples [19]_
+    4. SMOTEN - SMMOTE for Nominal only [8]_


SMMOTE->SMOTE

chkoar · 2021-02-16T06:42:39Z

 other extra interpolation.

+However, :class:`SMOTENC` is working with data composed of categorical data
+only. WHen data are made of only nominal categorical data, one can use the


chkoar · 2021-02-16T06:44:08Z

 columns are belonging to the same categories originally presented without any
 other extra interpolation.

+However, :class:`SMOTENC` is working with data composed of categorical data


However, :class:SMOTENC is working with datasets composed of continuous and categorical features.

chkoar · 2021-02-16T06:44:29Z

+two ways:
+
+* the nearest neighbors search does not rely on the Euclidean distance. Indeed,
+  the value difference metric (VDM) also implemented in the class


Value Difference Metric

chkoar · 2021-02-16T06:46:00Z

+* the nearest neighbors search does not rely on the Euclidean distance. Indeed,
+  the value difference metric (VDM) also implemented in the class
+  :class:`~imblearn.metrics.ValueDifferenceMetric` is used.
+* the new sample generation is based on majority vote per feature to generate


on the majority?

chkoar · 2021-02-16T06:53:37Z

    --------
    SMOTE : Over-sample using SMOTE.

+    SMOTEN : Over-sample using the SMOTE variable specifically for categorical


SMOTEN : Over-sample using the SMOTE variant specifically for nominal features only.

chkoar · 2021-02-16T06:53:46Z


+    SMOTENC : Over-sample using SMOTE for continuous and categorical features.
+
+    SMOTEN : Over-sample using the SMOTE variable specifically for categorical


SMOTEN : Over-sample using the SMOTE variant specifically for nominal features only.

chkoar · 2021-02-16T06:54:30Z

+    SMOTEN : Over-sample using the SMOTE variable specifically for categorical
+        features only.
+
+    SVMSMOTE : Over-sample using SVM-SMOTE variant.


SVMSMOTE : Over-sample using the SVM-SMOTE variant.

chkoar · 2021-02-16T06:54:40Z

+
+    SVMSMOTE : Over-sample using SVM-SMOTE variant.
+
+    BorderlineSMOTE : Over-sample using Borderline-SMOTE variant.


BorderlineSMOTE : Over-sample using the Borderline-SMOTE variant.

chkoar · 2021-02-16T06:55:19Z

+
+    ADASYN : Over-sample using ADASYN.
+
+    KMeansSMOTE : Over-sample applying a clustering before to oversample using


KMeansSMOTE : Over-sample by applying a clustering before to oversample using
SMOTE.

chkoar · 2021-02-16T07:00:25Z

@@ -168,11 +168,12 @@ Below is a list of the methods currently implemented in this module.
    1. Random minority over-sampling with replacement
    2. SMOTE - Synthetic Minority Over-sampling Technique [8]_
    3. SMOTENC - SMOTE for Nominal Continuous [8]_


SMOTENC - SMOTE for Nominal and Continuous
or
SMOTENC - SMOTE Nominal Continuous

glemaitre · 2021-02-16T07:46:36Z

Thanks. I will make the changes in a new PR.

Regarding "nominal" I am not convinced what is the best way. I think that we don't make a good job in the doc to explain what are nominal values and in scikit-learn usually, we talk about categorical features. When it comes to nominal features, we are usually using the term "nominal categorical features" to oppose it to "ordinal categorical features".

@chkoar do you have any thoughts on that?

chkoar · 2021-02-16T10:29:28Z

Keeping aside the numerical features and the representation of the categorical data.

In general, nominal and ordinal variables represent distinct categories where math does not make sense. However, in ordinal variables order does matter. For instance color is nominal while priority is ordinal: L, M,H. So, you should have domain knowledge in order to encode accordingly(if you dont have already the encoding). You should not encode blindingly using the type of a variable. Depending on the context you make the encoding. Although, I suppose that priority could be coded as it was nominal in the context of feature encoding. On the other hand if you have the priority in the class probably you should encode using ordinal encoding while you are interested in ordinal classification.

To sum up, IMHO, I think that Chawla et al with word nominal are referring to categorical features. Since nominal and ordinal type are both categorical I would say that we could use the word categorical every where. Instead of the name SMOTEN and SMOTENC.

Does this make sense?

glemaitre · 2021-02-16T10:48:12Z

Yep I think that would be best to be consistent. I will make a pass on the doc.

glemaitre added 2 commits February 15, 2021 19:52

FEA implement SMOTEN

836fabd

TST add first basic test

049dde9

glemaitre added 3 commits February 15, 2021 20:26

add doc

fbac276

DOC update other docstring

795b269

iter

fee46f7

glemaitre added 3 commits February 15, 2021 21:23

Merge remote-tracking branch 'origin/master' into is/smoten_2

73c3743

iter

3f20f39

iter

8626040

glemaitre added 2 commits February 15, 2021 22:32

Merge remote-tracking branch 'origin/master' into is/smoten_2

c75123c

iter

39dd844

glemaitre added 2 commits February 15, 2021 23:02

DOC add docstring to API

516b33c

Merge remote-tracking branch 'origin/master' into is/smoten_2

f648089

glemaitre added 2 commits February 15, 2021 23:53

DOC add whats new

57f839f

DOC update whats new

a66bfaa

glemaitre merged commit e3df215 into scikit-learn-contrib:master Feb 15, 2021

chkoar reviewed Feb 16, 2021

View reviewed changes


		SMOTENC : Over-sample using SMOTE for continuous and categorical features.

		SMOTEN : Over-sample using the SMOTE variable specifically for categorical


		SVMSMOTE : Over-sample using SVM-SMOTE variant.

		BorderlineSMOTE : Over-sample using Borderline-SMOTE variant.


		ADASYN : Over-sample using ADASYN.

		KMeansSMOTE : Over-sample applying a clustering before to oversample using

Conversation

glemaitre commented Feb 15, 2021

Uh oh!

pep8speaks commented Feb 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Comment last updated at 2021-02-15 22:55:02 UTC

Uh oh!

lgtm-com Bot commented Feb 15, 2021

Uh oh!

lgtm-com Bot commented Feb 15, 2021

Uh oh!

lgtm-com Bot commented Feb 15, 2021

Uh oh!

codecov Bot commented Feb 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

lgtm-com Bot commented Feb 15, 2021

Uh oh!

lgtm-com Bot commented Feb 15, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

glemaitre commented Feb 16, 2021

Uh oh!

chkoar commented Feb 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

glemaitre commented Feb 16, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pep8speaks commented Feb 15, 2021 •

edited

Loading

codecov Bot commented Feb 15, 2021 •

edited

Loading

chkoar commented Feb 16, 2021 •

edited

Loading