Skip to content

adding NMT dataset - #443

Merged
bruAristimunha merged 35 commits into
braindecode:masterfrom
javadbayazi:master
Apr 3, 2024
Merged

bruAristimunha merged 35 commits into
braindecode:masterfrom
javadbayazi:master

Conversation

@javadbayazi

@javadbayazi javadbayazi commented Mar 21, 2023

Copy link
Copy Markdown
Contributor

As the NMT dataset is similar to TUH abnormal, with a few changes we can add this dataset to Braindecode!
Reference paper

@codecov

codecov Bot commented Mar 21, 2023

Copy link
Copy Markdown

Codecov Report

Merging #443 (b505acd) into master (c8d7de6) will increase coverage by 0.10%.
The diff coverage is 91.46%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #443      +/-   ##
==========================================
+ Coverage   84.00%   84.11%   +0.10%     
==========================================
  Files          66       67       +1     
  Lines        5338     5419      +81     
==========================================
+ Hits         4484     4558      +74     
- Misses        854      861       +7     

@robintibor

Copy link
Copy Markdown
Contributor

Thanks for starting this!
Having NMT be a subclass of TUH seems a potentially confusing decision to me.
We also had some code here to load NMT: https://gist.github.com/gemeinl/e932169ed2eca5bf8b25432f0d54b095
Is there any difference in functionality between these two versions as far as you can see?

In order to avoid potential confusions and also that code may break for one dataset if the other is changed, I suggest we either keep them separate as in gist above or we define common functions that both use in a separate file. This might be overkill, definitely I think subclassing TUH here seems too high a risk of creating unintentional errors later

@javadbayazi

Copy link
Copy Markdown
Contributor Author

Thanks for starting this! Having NMT be a subclass of TUH seems a potentially confusing decision to me. We also had some code here to load NMT: https://gist.github.com/gemeinl/e932169ed2eca5bf8b25432f0d54b095 Is there any difference in functionality between these two versions as far as you can see?

No, they are the same! I haven’t seen that gist and it worked for me with a few changes!

In order to avoid potential confusions and also that code may break for one dataset if the other is changed, I suggest we either keep them separate as in gist above or we define common functions that both use in a separate file. This might be overkill, definitely I think subclassing TUH here seems too high a risk of creating unintentional errors later

Sure! Having a separate file would be a better option.

@robintibor

Copy link
Copy Markdown
Contributor

Thanks for starting this! Having NMT be a subclass of TUH seems a potentially confusing decision to me. We also had some code here to load NMT: https://gist.github.com/gemeinl/e932169ed2eca5bf8b25432f0d54b095 Is there any difference in functionality between these two versions as far as you can see?

No, they are the same! I haven’t seen that gist and it worked for me with a few changes!

In order to avoid potential confusions and also that code may break for one dataset if the other is changed, I suggest we either keep them separate as in gist above or we define common functions that both use in a separate file. This might be overkill, definitely I think subclassing TUH here seems too high a risk of creating unintentional errors later

Sure! Having a separate file would be a better option.

Great would you mind having a try to create this separate file then based on the gist?

@bruAristimunha

Copy link
Copy Markdown
Collaborator

Hi @MohammadJavadD,

do you need any help?

@javadbayazi

Copy link
Copy Markdown
Contributor Author

@bruAristimunha I made the discussed change to create a separate class for the NMT dataset, Please take a look.

@bruAristimunha bruAristimunha changed the title 'adding NMT dataset' adding NMT dataset Jan 8, 2024
Comment thread braindecode/datasets/nmt.py Outdated
Comment thread braindecode/datasets/nmt.py Outdated
@bruAristimunha

Copy link
Copy Markdown
Collaborator

We need a fetch for the dataset. I am doing something @MohammadJavadD

@bruAristimunha

Copy link
Copy Markdown
Collaborator

Can you make the tests in the meantime, @MohammadJavadD?

There are plenty of tests for Temple University on braindecode, and you can follow the same type of mock test.

@javadbayazi

Copy link
Copy Markdown
Contributor Author

We need a fetch for the dataset. I am doing something @MohammadJavadD

This might help

@bruAristimunha

Copy link
Copy Markdown
Collaborator

Thank you for the suggestion @MohammadJavadD, I saw your code, but I will use mne to fetch the data. I am uploading the dataset in a better place than google drive.

@bruAristimunha

bruAristimunha commented Mar 28, 2024

Copy link
Copy Markdown
Collaborator

If you need any help, please let me know, on the test side.

@javadbayazi

Copy link
Copy Markdown
Contributor Author

@bruAristimunha I added a mock dataset for the NMT and created some unit tests based on that.

@bruAristimunha

Copy link
Copy Markdown
Collaborator

Hey @apmellot, @jpaillard and @dengemann!

FYI.

We are adding this potential brain-age dataset, smaller than the Temple University Dataset, as the community uses the dataset, cc. @MohammadJavadD.

You may have options about the dataset, but as you work with the TUH dataset, I think you would like to know about it. If you have time to review, that would be nice, but okay otherwise.

@javadbayazi

Copy link
Copy Markdown
Contributor Author

Hey @apmellot, @jpaillard and @dengemann!

FYI.

We are adding this potential brain-age dataset, smaller than the Temple University Dataset, as the community uses the dataset, cc. @MohammadJavadD.

You may have options about the dataset, but as you work with the TUH dataset, I think you would like to know about it. If you have time to review, that would be nice, but okay otherwise.

We tried sex prediction on this dataset here, I expect it should work for age as well, but I'm curious to know more about it.

@bruAristimunha

Copy link
Copy Markdown
Collaborator

Hey @MohammadJavadD,

Fetch working!

Can you upload your branch and test it to see if the dataset download is working? Too big to be tested in the CI.

@bruAristimunha

Copy link
Copy Markdown
Collaborator

I also think that one of your tests is not working.

@bruAristimunha
bruAristimunha self-requested a review April 3, 2024 22:04
@bruAristimunha
bruAristimunha merged commit a8801c9 into braindecode:master Apr 3, 2024
@bruAristimunha

Copy link
Copy Markdown
Collaborator

Thank you so much @MohammadJavadD!

I am super happy with your contribution, and let's continue on the transfer learning tutorial ;)

@bruAristimunha

Copy link
Copy Markdown
Collaborator

and btw, thank you @neerajwagh for your google drive!

@javadbayazi

Copy link
Copy Markdown
Contributor Author

Thank you @bruAristimunha for all your help.

@neerajwagh

Copy link
Copy Markdown

and btw, thank you @neerajwagh for your google drive!

No problem! It's worth checking with the NMT authors to see why the link was taken down or not maintained. They could be working on a newer version and can notify you of updates.

@bruAristimunha

bruAristimunha commented Apr 4, 2024 via email

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants