Adding Sleep Models #341

Div12345 · 2021-10-19T11:48:29Z

Closes #334

Added the AttnSleep Model and have the example tried once here. AttnSleep requires Single-Channel EEG Data. So I used a new windows_dataset with a picked channel. I wasn't sure whether to update the existing example because then it probably won't look good with 2 different datasets creations?

Div12345 · 2021-10-19T11:57:20Z

Also primarily the code of the model is from this repo. I've put the paper reference, should I add the reference to this too somewhere?

codecov · 2021-10-19T12:14:45Z

Codecov Report

Merging #341 (fc919a3) into master (4dd51b3) will increase coverage by 0.83%.
The diff coverage is 98.47%.

@@            Coverage Diff             @@
##           master     #341      +/-   ##
==========================================
+ Coverage   81.90%   82.74%   +0.83%     
==========================================
  Files          51       53       +2     
  Lines        3515     3714     +199     
==========================================
+ Hits         2879     3073     +194     
- Misses        636      641       +5

agramfort

@Div12345 you'll need to add test and update the documentation

ask if you need help

braindecode/models/attn_sleep.py

robintibor · 2021-10-19T17:01:29Z

Yeah you could just write a simple test checking that the model forward works and has expected shapes, see https://github.com/braindecode/braindecode/blob/7f524e5f81f4caff42a1e205ae8d92281d30eeb8/test/unit_tests/models/test_models.py for inspiration, you could add a test there
Also yes, I would add the link to that repo in the docstring

Div12345 · 2021-10-21T08:14:03Z

Regarding the docs, what all should be updated? From the USleep PR I see api.rst and whats_new.rst updated. Should these just be updated manually?

agramfort

search for SleepStagerChambon2018 in the code base to see where the new models should be mentionned. Just add the new model names below

agramfort · 2021-10-21T08:57:32Z

braindecode/models/attn_sleep.py

+
+
+class AttnSleep(nn.Module):
+    """Sleep Staging Architecture from Eldele et al 2021.


you add 2 publics models here no?
to be consistent this could be:

SleepStagerEldele2021

?

Is the same convention to be followed for all the sleep staging models (USleep also)? Or only because here I'm adding these together?

you add apparently 2 new public models so to me this deserves for each a doc entry, tests etc.

feel free to start with one if you prefer / find it easier.

Div12345 · 2021-10-25T06:51:57Z

I just added the tests. There is one thing with the Eldele2021, it is written very specifically for 100Hz and 30s (the convolutional layers and fully connected layers after are restricting that). There can either be a resampling at the beginning to make the overall number of samples to 3000 or I can try replacing the pooling layers with AdaptiveMax/AvgPool which provides a constant output size.

sliwy · 2021-10-25T09:09:22Z

Maybe we can use similar solution to the one used in SleepStagerChambon, so the conv/pool sizes are adjusted according to the sfreq?

braindecode/braindecode/models/sleep_stager_chambon_2018.py

Lines 60 to 63 in 3102ad5

    
           time_conv_size = np.ceil(time_conv_size_s * sfreq).astype(int) 
        
           max_pool_size = np.ceil(max_pool_size_s * sfreq).astype(int) 
        
           input_size = np.ceil(input_size_s * sfreq).astype(int) 
        
           pad_size = np.ceil(pad_size_s * sfreq).astype(int)

agramfort

do you have some learning curves eg on physionet to see how well it learns?
you can start from the example of Chambon's model.

agramfort · 2021-10-25T08:25:53Z

braindecode/models/sleep_stager_blanco_2020.py

+# Authors: Divyesh Narayanan <divyesh.narayanan@gmail.com>
+# #
+# # License: BSD (3-clause)


Suggested change

# Authors: Divyesh Narayanan <divyesh.narayanan@gmail.com>

# #

# # License: BSD (3-clause)

# Authors: Divyesh Narayanan <divyesh.narayanan@gmail.com>

#

# License: BSD (3-clause)

agramfort · 2021-10-25T08:44:50Z

braindecode/models/sleep_stager_blanco_2020.py

+        """
+        Forward pass.
+        Parameters


Suggested change

"""

Forward pass.

Parameters

"""Forward pass.

Parameters

agramfort · 2021-10-25T08:51:28Z

braindecode/models/sleep_stager_eldele_2021.py

+import torch
+from torch import nn
+import torch.nn.functional as F
+import numpy as np
+import math
+import copy
+from copy import deepcopy


Suggested change

import torch

from torch import nn

import torch.nn.functional as F

import numpy as np

import math

import copy

from copy import deepcopy

import math

import copy

from copy import deepcopy

import numpy as np

import torch

from torch import nn

import torch.nn.functional as F

first standard lib, then order by core component in the ecosystem (from global to local)

agramfort · 2021-10-25T09:17:13Z

docs/whats_new.rst

@@ -73,6 +73,7 @@ Enhancements
 - Only load data if needed during preprocessing (e.g., allow timecrop without loading) (:gh:`164` by `Robin Tibor Schirrmeister`_)
 - Adding option to sort filtered channels by frequency band for the filterbank in :func:`braindecode.datautil.filterbank` (:gh:`185` by `Lukas Gemein`_)
 - Adding the USleep model :class:`braindecode.models.USleep` (:gh:`282` by `Theo Gnassounou`_ and `Omar Chehab`_)
+- Adding SleepStagerEldele2021 and SleepStagerBlanco2020 models :class:`braindecode.models.SleepStagerEldele2021` and :class:`braindecode.models.SleepStagerBlanco2020` (:gh:`341` by `Divyesh Narayanan`_)


Suggested change

- Adding SleepStagerEldele2021 and SleepStagerBlanco2020 models :class:`braindecode.models.SleepStagerEldele2021` and :class:`braindecode.models.SleepStagerBlanco2020` (:gh:`341` by `Divyesh Narayanan`_)

- Adding :class:`braindecode.models.SleepStagerEldele2021` and :class:`braindecode.models.SleepStagerBlanco2020` models for sleep staging (:gh:`341` by `Divyesh Narayanan`_)

Div12345 · 2021-10-25T09:31:40Z

@agramfort The learning curve is available here
Eldele2021 -

Blanco2020 -

The Eldele2021 learns fairly well whereas the Blanco2020 is pretty poor for this case.

@sliwy I'll check that out.

agramfort · 2021-10-25T10:06:58Z

If there is no bug in Blanco2020 then I would not add a "so so" model to braindecode and it's not free in terms of maintenance and testing time

…

Div12345 · 2021-10-25T11:57:50Z

In my use case here it shows decent learning. But I understand, let me know if you want me to make any further specific tests to see if it's worth it or otherwise I can remove it.

agramfort · 2021-10-25T12:43:54Z

what is changing compared to the graph you showed above? less epochs? different data?

…

On Mon, Oct 25, 2021 at 1:58 PM Divyesh Narayanan ***@***.***> wrote: In my use case here <https://github.com/Div12345/SleepStaging-TransferLearning/blob/main/Blanco2020_Model_Sleep_TL.ipynb> it shows decent learning. But I understand, let me know if you want me to make any further specific tests to see if it's worth it or otherwise I can remove it. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#341 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABHKHBN7PW2G5IQVRA6YWDUIVA4TANCNFSM5GI6AZRQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Div12345 · 2021-10-25T17:21:27Z

Same Physionet, 6 class (without reduction to AASM guidelines), more data, SGD optim (vs Adam in example), data sampler to give balanced epochs.

agramfort · 2021-10-26T08:23:13Z

Same Physionet, 6 class (without reduction to AASM guidelines), more data, SGD optim (vs Adam in example), data sampler to give balanced epochs.

that makes many changes each of them could also benefit to the already available models. ideally we should have a benchmark file/notebook with soemthign like: for model in benchmarked_models: # train, and plot perf so we can compare the different models in a fair manner

Div12345 · 2021-10-26T08:48:01Z

I agree. I've been working on exactly that for MOABB NeuroTechX/moabb#190, there's also the corresponding issue in Braindecode #46, but I think the datasets are bigger here? So to download, save the datasets and run the DL runs, I personally don't have the resources required to do that.

agramfort · 2021-10-26T09:12:49Z

@Div12345 you mean the time or the computational ressources?

Div12345 · 2021-10-26T09:16:21Z

I mean computational resources.

agramfort · 2021-10-26T09:32:53Z

this I can help at Inria. If you share a script we can run it and share the results with you

…

Div12345 · 2021-10-26T09:38:49Z

Okay, we can continue this discussion on the Braindecode benchmarking issue, I guess then? In terms of discussing what all are to be tested out and on what datasets?

agramfort · 2021-10-26T09:53:35Z

works for me

…

Div12345 · 2021-10-29T07:50:36Z

@sliwy I looked into the method used in SleepStagerChambon, so the conv/pool sizes are adjusted according to the sfreq. As I understand it, though this specifically helps in the "resolution" sort of, of the CNN there, it doesn't necessarily make the final output of the same length (for example when the length of the input signal changes with sfreq same)? I think that is taken care of by the allocation of the size of the last layer using len_last_layer? Here, there are sort of 2 main submodules and the size is varying at the end of the first submodule itself.
So I tried adding a function that finds the length of the output at the end of this first submodule and using that to inform the second submodule at initialization. But this creates a problem due to a requirement where the number of attention heads in the model has to be a factor of this length. I'm not sure if making the number of attention heads in some way a dependent variable is such a good idea.

I don't know the particular usefulness of the sfreq dependent CNN sizes, but if you say it'll be useful, I can add it anyways.

agramfort · 2021-10-29T14:32:59Z

I would suggest you resample the data to the sfreq you want for the architecture you have. It's much easier than updating the architecture

…

agramfort · 2021-11-08T13:51:01Z

braindecode/models/sleep_stager_eldele_2021.py

+
+        # exception case for shhs to make modification in mrcnn as per implementation in the paper
+        shhs = False
+        if input_size_s == 30 and sfreq == 125 and d_model == 100:


all the sleep scoring models I know work with 30s windows. I think could even enforce this. Then it's just a matter of sfreq. If sfreq==100 do this and if 125 do that. If different crash. No need to mention shhs in the code. Just put the doctsring that if sfreq==125Hz then we will use the reference architecture from XXX et al. which was validated on SHHS dataset. This is just doc, no variable name etc.

makes sense?

but from what I understood,f or the "shhs case" we still have to perform the adaptation necessary for that case so that the model doesn't just crash (change kernel length). so we have to somehow do that right?

agramfort · 2021-11-08T15:55:52Z

what do you mean do the "adaptation"? it's more than the "if else" I suggested?

…

robintibor · 2021-11-08T16:00:31Z

I mean https://github.com/braindecode/braindecode/pull/341/files#diff-4e03e96560b897b4bf4f6353fff19ca630f66347eafd96f741a62d3da843537fR197-R232

I am still not 100% sure I understood correct, but i guess we have to supply _MRCNN with the information that kernel_length=6 for that case? could also supply kernel_length=6 directly? or how do you mean? Because I understood you don't like the shhs flag passed to _MRCNN

agramfort · 2021-11-08T16:32:25Z

I mean https://github.com/braindecode/braindecode/pull/341/files#diff-4e03e96560b897b4bf4f6353fff19ca630f66347eafd96f741a62d3da843537fR197-R232

yes this copy paste should be fixed

I am still not 100% sure I understood correct, but i guess we have to supply _MRCNN with the information that kernel_length=6 for that case? could also supply kernel_length=6 directly? or how do you mean?

yes for example.

robintibor · 2021-11-19T14:46:28Z

So @Div12345 is a bit busy atm and would take a bit time before he can continue. Should we merge this in its current form or make a braindecode release without this? Otherwise from my side release could be done. What do you think @agramfort @gemeinl @hubertjb ?

… more standardized for sleep examples

examples/plot_sleep_staging_eldele2021.py

agramfort · 2021-11-19T20:27:41Z

@Div12345 you need fix conflicts here

Div12345 · 2021-11-19T21:48:42Z

@agramfort I changed the name of the plot_sleep_staging example to plot_sleep_staging_chambon2018 to be more standardized like the other sleep staging examples. I think that is what is causing the conflict?

agramfort · 2021-11-20T20:33:34Z

you need to merge the main branch into your branch. There has been changes in this example in parallel and this creates a conflict.

…

agramfort · 2021-11-20T21:57:34Z

@Div12345 you need to git merge main into this PR to fix the conflict.

Div12345 · 2021-11-20T22:00:19Z

Gives Already up to date

agramfort · 2021-11-20T22:03:49Z

I pushed a fix please share the urls of the rendered examples in the doc build so we can see how it looks

Div12345 · 2021-11-20T22:55:25Z

The Eldele example - Link - Just have to change the title and references a bit. The renamed Chambon - Link

robintibor · 2021-11-21T17:45:15Z

I would strongly be in favour of drastically reducing nr of epochs and/or possibly dataset size to speed up example on CI. now we know it would have good performance with reasonable epochs, so we can just cut to just 1 epoch from my point of view.. right now it has 11 min running time on CI just for this example... As before, let's just be clear to the reader that this is a modified example for faster runtime and be superclear how to run it for reasonable performance...

Div12345 · 2021-11-23T16:25:29Z

@robintibor done.

robintibor · 2021-11-29T09:11:56Z

so let's merge this and have a release! how do you think @agramfort ?

Co-authored-by: Alexandre Gramfort <alexandre.gramfort@m4x.org>

robintibor · 2021-12-01T13:47:38Z

Since there were no further comments, have commited the last suggestions inside and now it's merged! Thanks so much @Div12345 !! :)

agramfort · 2021-12-01T14:19:49Z

thanks @Div12345

…

Div12345 added 3 commits October 19, 2021 15:25

Adding AttnSleep Model

638db5b

Merge branch 'braindecode:master' into Sleep-Models

5773d7c

rollback attempt to extend to multiple channels

edb836a

agramfort reviewed Oct 19, 2021

View reviewed changes

braindecode/models/attn_sleep.py Outdated Show resolved Hide resolved

braindecode/models/attn_sleep.py Outdated Show resolved Hide resolved

braindecode/models/attn_sleep.py Outdated Show resolved Hide resolved

Div12345 added 2 commits October 20, 2021 11:02

Update attn_sleep.py

5037b66

Adding Blanco2020 and minor edits

3272ee1

agramfort reviewed Oct 21, 2021

View reviewed changes

Div12345 added 2 commits October 22, 2021 19:21

Model Renaming, flake8, doc

aa2335e

adding tests

7b4ec96

agramfort reviewed Oct 25, 2021

View reviewed changes

agramfort reviewed Nov 8, 2021

View reviewed changes

Div12345 added 3 commits November 19, 2021 22:03

remove redundancy, remove reference to shhs in code, add in docstring

cfee004

flake8 correct

9dd9ff9

example for eldele2021, tweaking name of chambon example to make name…

51a378e

… more standardized for sleep examples

agramfort reviewed Nov 19, 2021

View reviewed changes

examples/plot_sleep_staging_eldele2021.py Outdated Show resolved Hide resolved

Update plot_sleep_staging_chambon2018.py

b883fdc

Div12345 added 3 commits November 21, 2021 03:22

Delete plot_sleep_staging_chambon2018.py

8c71c41

Create plot_sleep_staging.py

0567d50

Rename plot_sleep_staging.py to plot_sleep_staging_chambon2018.py

88a05c9

Merge branch 'master' into Sleep-Models

2b8373b

edits in references, epochs to 1, note to increase it

1ee46e9

import from higher level; fix typos

fc919a3

Co-authored-by: Alexandre Gramfort <alexandre.gramfort@m4x.org>

robintibor merged commit b376d6a into braindecode:master Dec 1, 2021

robintibor mentioned this pull request Dec 1, 2021

[WIP] version 0.6 release #336

Merged



		class AttnSleep(nn.Module):
		"""Sleep Staging Architecture from Eldele et al 2021.

	- Adding SleepStagerEldele2021 and SleepStagerBlanco2020 models :class:`braindecode.models.SleepStagerEldele2021` and :class:`braindecode.models.SleepStagerBlanco2020` (:gh:`341` by `Divyesh Narayanan`_)
	- Adding :class:`braindecode.models.SleepStagerEldele2021` and :class:`braindecode.models.SleepStagerBlanco2020` models for sleep staging (:gh:`341` by `Divyesh Narayanan`_)

Adding Sleep Models #341

Adding Sleep Models #341

Conversation

Div12345 commented Oct 19, 2021

Div12345 commented Oct 19, 2021

codecov bot commented Oct 19, 2021 • edited Loading

Codecov Report

agramfort left a comment

Choose a reason for hiding this comment

robintibor commented Oct 19, 2021

Div12345 commented Oct 21, 2021

agramfort left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Div12345 commented Oct 25, 2021

sliwy commented Oct 25, 2021

agramfort left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Div12345 commented Oct 25, 2021

agramfort commented Oct 25, 2021 via email

Div12345 commented Oct 25, 2021

agramfort commented Oct 25, 2021 via email

Div12345 commented Oct 25, 2021

agramfort commented Oct 26, 2021 via email

Div12345 commented Oct 26, 2021

agramfort commented Oct 26, 2021

Div12345 commented Oct 26, 2021

agramfort commented Oct 26, 2021 via email

Div12345 commented Oct 26, 2021

agramfort commented Oct 26, 2021 via email

Div12345 commented Oct 29, 2021 • edited Loading

agramfort commented Oct 29, 2021 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agramfort commented Nov 8, 2021 via email

robintibor commented Nov 8, 2021 • edited Loading

agramfort commented Nov 8, 2021 via email

robintibor commented Nov 19, 2021

agramfort commented Nov 19, 2021

Div12345 commented Nov 19, 2021

agramfort commented Nov 20, 2021 via email

agramfort commented Nov 20, 2021

Div12345 commented Nov 20, 2021

agramfort commented Nov 20, 2021 via email

Div12345 commented Nov 20, 2021

robintibor commented Nov 21, 2021

Div12345 commented Nov 23, 2021

robintibor commented Nov 29, 2021

robintibor commented Dec 1, 2021

agramfort commented Dec 1, 2021 via email

codecov bot commented Oct 19, 2021 •

edited

Loading

Div12345 commented Oct 29, 2021 •

edited

Loading

robintibor commented Nov 8, 2021 •

edited

Loading