Leaderboard and result reports

Report for the evaluation of pipelines on MOABB datasets

Motor imagery

Left/Right hand classes

Data filtered between 7-35 Hz. Epoch length are dataset-dependent and use maximum window length. Average score is computed across subjects.

TODO:

• add sessions above chance level, using combrisson algorithm to compute chance level
• check results on collab (C) and local machine (L) and complete confirmed column

Dataset	Evaluation type	Pipeline	Avg score	Confirmed
001-2014	WithinSession	CSP + LDA	0.823360	L
		MDM	0.816943	L
		CSP + SVM	0.828946	L
		Log Variance LDA	0.779562	L
		Tangent Space LR	0.874123	L
		Log Variance SVM	0.711478	L
		Tangent Space SVM	0.864214	L
		CSP + optSVM	0.820544	L
004-2014	WithinSession	CSP + LDA	0.800957	L
		MDM	0.776555	L
		CSP + SVM	0.796910	L
		Log Variance LDA	0.785073	L
		Tangent Space LR	0.800938	L
		Log Variance SVM	0.779423	L
		Tangent Space SVM	0.798726	L
		CSP + optSVM	0.791219	L
Cho2017	WithinSession	CSP + LDA	0.711742
		MDM	0.635015
		CSP + SVM	0.712101
		Log Variance LDA	0.645397
		Tangent Space LR	0.745154
		Log Variance SVM	0.661121
		Tangent Space SVM	0.743365
		CSP + optSVM	0.706311
Grosse-Wentrup 2009	WithinSession	CSP + LDA	0.764378
		MDM	0.642911
		CSP + SVM	0.771578
		Log Variance LDA	0.787111
		Tangent Space LR	0.876000
		Log Variance SVM	0.809867
		Tangent Space SVM	0.884600
		CSP + optSVM	0.775644
Lee2019_MI	WithinSession	CSP + LDA	0.768815
		MDM	0.702296
		CSP + SVM	0.769074
		Log Variance LDA	0.662278
		Tangent Space LR	0.830926
		Log Variance SVM	0.740407
		Tangent Space SVM	0.841815
		CSP + optSVM	0.765315
Physionet Motor Imagery	WithinSession	CSP + LDA	0.657480
		MDM	0.547599
		CSP + SVM	0.651111
		Log Variance LDA	0.619386
		Tangent Space LR	0.672834
		Log Variance SVM	0.624919
		Tangent Space SVM	0.694299
		CSP + optSVM	0.648555
Schirrmeister2017	WithinSession	CSP + LDA	0.772348
		MDM	0.615261
		CSP + SVM	0.771553
		Log Variance LDA	0.784386
		Tangent Space LR	0.872200
		Log Variance SVM	0.787537
		Tangent Space SVM	0.876428
		CSP + optSVM	0.769153
Shin2017A	WithinSession	CSP + LDA	0.722989
		MDM	0.629885
		CSP + SVM	0.713218
		Log Variance LDA	0.617816
		Tangent Space LR	0.693103
		Log Variance SVM	0.615517
		Tangent Space SVM	0.721839
		CSP + optSVM	0.729310
Weibo 2014	WithinSession	CSP + LDA	0.807191	L
		MDM	0.588015	L
		CSP + SVM	0.801518	L
		Log Variance LDA	0.741323	L
		Tangent Space LR	0.836191	L
		Log Variance SVM	0.728597	L
		Tangent Space SVM	0.837777	L
		CSP + optSVM	0.798406	L
Zhou 2016	WithinSession	CSP + LDA	0.931489	L
		MDM	0.907018	L
		CSP + SVM	0.932315	L
		Log Variance LDA	0.883861	L
		Tangent Space LR	0.941601	L
		Log Variance SVM	0.854676	L
		Tangent Space SVM	0.938613	L
		CSP + optSVM	0.928818	L

Right hand/feet classes

Dataset	Pipeline	Evaluation type	Average score across subjects	Sessions above chance level

All classes of dataset

Dataset	Pipeline	Evaluation type	Avg score	#Classes	Confirmed
Alexandre Motor Imagery	WithinSession	DLCSPauto + shLDA	0.606250	3
		CSP + LDA	0.610417	3
		TS + optSVM	0.675000	3
		MDM	0.606250	3
		CSP + SVM	0.622917	3
		AM + optSVM	0.497917	3
		Tangent Space LR	0.691667	3
		CSP + optSVM	0.622917	3
Zhou 2016	WithinSession	DLCSPauto + shLDA	0.820614	3
		CSP + LDA	0.829631	3
		TS + optSVM	0.834106	3
		MDM	0.760476	3
		CSP + SVM	0.831694	3
		AM + optSVM	0.742366	3
		Tangent Space LR	0.848793	3
		CSP + optSVM	0.818829	3
001-2014	WithinSession	DLCSPauto + shLDA	0.663070	4
		CSP + LDA	0.659941	4
		TS + optSVM	0.714290	4
		MDM	0.616008	4
		CSP + SVM	0.667110	4
		AM + optSVM	0.565252	4
		Tangent Space LR	0.719735	4
		CSP + optSVM	0.660210	4
Physionet Motor Imagery	WithinSession	DLCSPauto + shLDA	0.468488	4
		CSP + LDA	0.477343	4
		TS + optSVM	0.557987	4
		MDM	0.429645	4
		CSP + SVM	0.471500	4
		AM + optSVM	0.458511	4
		Tangent Space LR	0.585544	4
		CSP + optSVM	0.479305	4
Schirrmeister2017	WithinSession	DLCSPauto + shLDA	0.728150	4
		CSP + LDA	0.729715	4
		TS + optSVM	0.841548	4
		MDM	0.520315	4
		CSP + SVM	0.733400	4
		AM + optSVM	0.726038	4
		Tangent Space LR	0.845914	4
		CSP + optSVM	0.733464	4
Zhou 2016	WithinSession	DLCSPauto + shLDA	0.820614	4
		CSP + LDA	0.829631	4
		TS + optSVM	0.834106	4
		MDM	0.760476	4
		CSP + SVM	0.831694	4
		AM + optSVM	0.742366	4
		Tangent Space LR	0.848793	4
		CSP + optSVM	0.818829	4
004-2015	WithinSession	DLCSPauto + shLDA	0.270952	5
		CSP + LDA	0.270754	5
		TS + optSVM	0.450595	5
		MDM	0.241905	5
		CSP + SVM	0.268373	5
		AM + optSVM	0.340794	5
		Tangent Space LR	0.448056	5
		CSP + optSVM	0.267540	5
Physionet Motor Imagery	WithinSession	DLCSPauto + shLDA	0.468488	5
		CSP + LDA	0.477343	5
		TS + optSVM	0.557987	5
		MDM	0.429645	5
		CSP + SVM	0.471500	5
		AM + optSVM	0.458511	5
		Tangent Space LR	0.585544	5
		CSP + optSVM	0.479305	5