Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new benchmark over many datasets #264

Merged
merged 68 commits into from
Jun 24, 2024
Merged

Conversation

toncho11
Copy link
Collaborator

@toncho11 toncho11 commented Apr 8, 2024

It does include many P300 and Motor Imagery datasets.
It adapts both the paradigms and the pipelines.
And most importantly it runs statistics to show if the results are significant (using moabb).

The initial version can be run already.

Currently it requires the latest version of MOABB from github where:
- cache_config parameter is availabe for WithinSessionEvaluation()
- this bug is fixed: NeuroTechX/moabb#514

toncho11 and others added 30 commits March 4, 2024 16:17
It gives an error: AttributeError: Pipeline has none of the following attributes: decision_function.
Added an example that runs on a small number of test samples, so that we can get results quicker.
- limite size of training set
- change to slsqp optimizer
Added second parameter that specifies the number of hulls.
Added support for transform().
Added a new pipeline [NCH+LDA]
Improvements requested by GC.
@toncho11 toncho11 marked this pull request as draft April 8, 2024 13:39
@gcattan
Copy link
Collaborator

gcattan commented Jun 22, 2024

@toncho11 Do you want to update up-merge your branch and update your code with the latest changes? I could reuse this pipeline also to benchmark the NCH.

@toncho11 toncho11 requested review from gcattan June 24, 2024 16:20
@toncho11 toncho11 marked this pull request as ready for review June 24, 2024 16:24
Copy link
Collaborator

@gcattan gcattan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx @toncho11 :)

@gcattan gcattan merged commit 74bacb6 into pyRiemann:main Jun 24, 2024
11 checks passed
@toncho11
Copy link
Collaborator Author

But this benchmark requires the latest version of MOABB from git.

@toncho11
Copy link
Collaborator Author

I mean after MOABB 1.1.0.

@toncho11
Copy link
Collaborator Author

install_requires = [
        'moabb @ git+ssh://git@github.com//NeuroTechX/moabb.git'
    ]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Evaluation silently fails on multiple datasets
2 participants