Add class-distinctiveness function and its example code #215

msyamamoto · 2022-12-02T17:36:18Z

Hello everyone!

I am Maria Sayu Yamamoto.
I created a new function called class_distinctiveness that evaluates the degree of class separation between two classes of SPD matrices on the manifold.

This metric quantifies the class separability of the dataset based on the inter-class distance and the intra-class variance on the manifold, and it is originally proposed in the following reference:
Lotte, F., & Jeunet, C. (2018). Defining and quantifying users’ mental imagery-based BCI skills: a first step. Journal of neural engineering, 15(4), 046030.
https://iopscience.iop.org/article/10.1088/1741-2552/aac577/meta?casa_token=bZ_Sz-fcfXIAAAAA:_lr-H99Z4_Y-GrbxMgL6ELEOvx03b8I4Xa7gedXTciHcUdvz5TzQhGCZ1WGcKqNOOpWWRmbVKXx5

This can be used to select the optimal frequency band or time window on a manifold for enhancing the oscillatory activity classification performance, as it has been done in the following reference:
Yamamoto, M. S., Lotte, F., Yger, F., & Chevallier, S. (2022, July). Class-distinctiveness-based frequency band selection on the Riemannian manifold for oscillatory activity-based BCIs: preliminary results. In EMBC 2022-44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society.
https://hal.archives-ouvertes.fr/hal-03641137/

I also made an example code to show how this metric works by generating toy datasets with different class separability levels.
This metric can extend to more than two classes as well.

I would appreciate your feedback and suggestions:-)

Sincerely,
Maria

pyriemann/featureselection.py

…n scripts, move the function to preprocessing.py and delete featureselection.py

qbarthelemy

Thx Maria for this nice contribution!
Do not forget to add tests for your new function, for two and three classes.

pyriemann/preprocessing.py

examples/simulated/plot_class_distinctiveness.py

pyriemann/preprocessing.py

qbarthelemy · 2022-12-06T08:31:16Z

pyriemann/preprocessing.py

+
+def class_distinctiveness(X, y, nume_denomi=False):
+    """Measure class separability between classes on a manifold.
+


Can you write the formula for class distinctiveness value for k classes?

Actually, I’m not super comfortable with the current formula of class distinctiveness.
Because Fisher criterion is defined as the ratio of the variance between the classes to the variance within the classes.
IMO, I think all distances in code should be at a power of 2.

We can solve this difference, adding a parameter p being the exponent of power:
p=2 will measure class dispersions with respect means, ie Frechet variances minimized during means estimation;
p=1 will measure class dispersions with respect medians, ie standard deviations.

This is a quick update, but when I was writing the formula for the documentation, I realized that my implementation of the second commit did not correctly reflect the equation of multi-class distinctiveness (Equation 7 in the original paper). I am sorry for that. I will modify it in the third commit.

Actually, I am not sure if I fully understood your suggestion of using parameter p. That would be a different equation than the original paper's one, wouldn't it?
https://iopscience.iop.org/article/10.1088/1741-2552/aac577/meta?casa_token=bZ_Sz-fcfXIAAAAA:_lr-H99Z4_Y-GrbxMgL6ELEOvx03b8I4Xa7gedXTciHcUdvz5TzQhGCZ1WGcKqNOOpWWRmbVKXx5

In the article, a power of 2 was lost bewteen Eq(4) and Eq(5), giving a mean absolute deviation instead of a variance.

I suggest to add an argument $p$:
$$\mathrm{classDis}(A, B, p) = \frac{d \left(\bar{X}^{A}, \bar{X}^{B}\right)^p} {\frac{1}{2} \left( \sigma_{X^{A}}^p + \sigma_{X^{B}}^p \right)}$$
where
$$\sigma_{X^{K}}^p = \frac{1}{m} \sum_{i=1}^m d \left(X_i, \bar{X}^{K}\right)^p$$
When $p=1$, it gives equation from article.
When $p=2$, it really generalizes Fisher criterion, defined as the ratio of the variance between the classes to the variance within the classes.

Thank you very much for the clear explanation! I got it!
Indeed, it is good for the user that they can choose from the two ways of quantification by the parameter p.
Thank you so much for the suggestion:-)

examples/simulated/plot_class_distinctiveness.py

…he function to classification, and incorporate class_dis example into plot_toy_classification

msyamamoto · 2022-12-07T16:23:47Z

Do not forget to add tests for your new function, for two and three classes.

I added two tests at the end of test_classification.py. Please confirm them.

agramfort · 2022-12-07T20:42:59Z

@msyamamoto pushed a short commit here. Please have a look

@qbarthelemy thoughts?

msyamamoto · 2022-12-07T21:11:59Z

@agramfort Wow! I can learn so many things from your commit!! Thank you so much for the speedy revision!

qbarthelemy

Do not forget to complete whatsnew file.

pyriemann/classification.py

sylvchev

The example is really nice!

pyriemann/classification.py

agramfort

nothing more to add on my end

thx @msyamamoto 🙏

pyriemann/classification.py

qbarthelemy · 2022-12-09T11:32:17Z

pyriemann/preprocessing.py

+
+def class_distinctiveness(X, y, nume_denomi=False):
+    """Measure class separability between classes on a manifold.
+


In the article, a power of 2 was lost bewteen Eq(4) and Eq(5), giving a mean absolute deviation instead of a variance.

I suggest to add an argument $p$:
$$\mathrm{classDis}(A, B, p) = \frac{d \left(\bar{X}^{A}, \bar{X}^{B}\right)^p} {\frac{1}{2} \left( \sigma_{X^{A}}^p + \sigma_{X^{B}}^p \right)}$$
where
$$\sigma_{X^{K}}^p = \frac{1}{m} \sum_{i=1}^m d \left(X_i, \bar{X}^{K}\right)^p$$
When $p=1$, it gives equation from article.
When $p=2$, it really generalizes Fisher criterion, defined as the ratio of the variance between the classes to the variance within the classes.

pyriemann/classification.py

agramfort

just a final nitpick on my end

pyriemann/classification.py

agramfort · 2022-12-13T17:18:35Z

@qbarthelemy @sylvchev feel free to merge if happy

qbarthelemy

Thx @msyamamoto !
@sylvchev , some remarks?

sylvchev · 2022-12-13T21:45:00Z

Good for me, ok to merge
Thanks @msyamamoto and kudos for your first PR!

agramfort · 2022-12-13T22:04:47Z

👏 @msyamamoto 👏

msyamamoto · 2022-12-14T02:05:31Z

Thank you so much for approving this PR! @agramfort @qbarthelemy @sylvchev
During these 11 days, I could learn so many things from you all and my coding skill was greatly improved thanks to you!
Also, it was very enjoyable to improve something together through GitHub interactions:-)
I appreciate all your detailed feedback and look forward to working on the next PR 😄

Add class-distinctiveness function and its example code

91a4d44

agramfort reviewed Dec 2, 2022

View reviewed changes

pyriemann/featureselection.py Outdated Show resolved Hide resolved

pyriemann/featureselection.py Outdated Show resolved Hide resolved

pyriemann/featureselection.py Outdated Show resolved Hide resolved

sylvchev requested changes Dec 3, 2022

View reviewed changes

pyriemann/featureselection.py Outdated Show resolved Hide resolved

pyriemann/featureselection.py Outdated Show resolved Hide resolved

sylvchev assigned msyamamoto Dec 3, 2022

sylvchev added the enhancement label Dec 3, 2022

Make class_dis function support more than two classes, apply flake8 o…

be06868

…n scripts, move the function to preprocessing.py and delete featureselection.py

qbarthelemy requested changes Dec 6, 2022

View reviewed changes

qbarthelemy reviewed Dec 6, 2022

View reviewed changes

pyriemann/preprocessing.py Outdated Show resolved Hide resolved

qbarthelemy reviewed Dec 6, 2022

View reviewed changes

agramfort reviewed Dec 6, 2022

View reviewed changes

examples/simulated/plot_class_distinctiveness.py Outdated Show resolved Hide resolved

Add the test of class_dis function, fix the class_dis formula, move t…

d9eb91b

…he function to classification, and incorporate class_dis example into plot_toy_classification

update doc, change a bit figure example + factorize tests

84c594e

flake8

f06d261

qbarthelemy reviewed Dec 7, 2022

View reviewed changes

pyriemann/classification.py Outdated Show resolved Hide resolved

pyriemann/classification.py Outdated Show resolved Hide resolved

pyriemann/classification.py Outdated Show resolved Hide resolved

sylvchev reviewed Dec 7, 2022

View reviewed changes

qbarthelemy reviewed Dec 8, 2022

View reviewed changes

pyriemann/classification.py Outdated Show resolved Hide resolved

msyamamoto added 2 commits December 8, 2022 21:59

update whatsnew + add argument 'metric'

7f1e39c

update test

1aa2cd1

agramfort approved these changes Dec 9, 2022

View reviewed changes

improve doc

a593525

qbarthelemy reviewed Dec 10, 2022

View reviewed changes

add private func + add parameter p

1732f75

agramfort reviewed Dec 12, 2022

View reviewed changes

pyriemann/classification.py Outdated Show resolved Hide resolved

change the parameter name to exponent

c24ceb6

agramfort approved these changes Dec 13, 2022

View reviewed changes

pyriemann/classification.py Outdated Show resolved Hide resolved

improve doc

fc6a6c0

fix equation display and add last modifs

5567fc2

qbarthelemy approved these changes Dec 13, 2022

View reviewed changes

sylvchev approved these changes Dec 13, 2022

View reviewed changes

agramfort merged commit 56cf653 into pyRiemann:master Dec 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add class-distinctiveness function and its example code #215

Add class-distinctiveness function and its example code #215

msyamamoto commented Dec 2, 2022

qbarthelemy left a comment

qbarthelemy Dec 6, 2022 •

edited

qbarthelemy Dec 6, 2022 •

edited

msyamamoto Dec 7, 2022 •

edited

msyamamoto Dec 8, 2022

qbarthelemy Dec 9, 2022 •

edited

msyamamoto Dec 12, 2022

msyamamoto commented Dec 7, 2022

agramfort commented Dec 7, 2022

msyamamoto commented Dec 7, 2022

qbarthelemy left a comment

sylvchev left a comment

agramfort left a comment

qbarthelemy Dec 9, 2022 •

edited

agramfort left a comment

agramfort commented Dec 13, 2022

qbarthelemy left a comment

sylvchev commented Dec 13, 2022

agramfort commented Dec 13, 2022

msyamamoto commented Dec 14, 2022


		def class_distinctiveness(X, y, nume_denomi=False):
		"""Measure class separability between classes on a manifold.

Add class-distinctiveness function and its example code #215

Add class-distinctiveness function and its example code #215

Conversation

msyamamoto commented Dec 2, 2022

qbarthelemy left a comment

Choose a reason for hiding this comment

qbarthelemy Dec 6, 2022 • edited

Choose a reason for hiding this comment

qbarthelemy Dec 6, 2022 • edited

Choose a reason for hiding this comment

msyamamoto Dec 7, 2022 • edited

Choose a reason for hiding this comment

msyamamoto Dec 8, 2022

Choose a reason for hiding this comment

qbarthelemy Dec 9, 2022 • edited

Choose a reason for hiding this comment

msyamamoto Dec 12, 2022

Choose a reason for hiding this comment

msyamamoto commented Dec 7, 2022

agramfort commented Dec 7, 2022

msyamamoto commented Dec 7, 2022

qbarthelemy left a comment

Choose a reason for hiding this comment

sylvchev left a comment

Choose a reason for hiding this comment

agramfort left a comment

Choose a reason for hiding this comment

qbarthelemy Dec 9, 2022 • edited

Choose a reason for hiding this comment

agramfort left a comment

Choose a reason for hiding this comment

agramfort commented Dec 13, 2022

qbarthelemy left a comment

Choose a reason for hiding this comment

sylvchev commented Dec 13, 2022

agramfort commented Dec 13, 2022

msyamamoto commented Dec 14, 2022

qbarthelemy Dec 6, 2022 •

edited

qbarthelemy Dec 6, 2022 •

edited

msyamamoto Dec 7, 2022 •

edited

qbarthelemy Dec 9, 2022 •

edited

qbarthelemy Dec 9, 2022 •

edited