-
Notifications
You must be signed in to change notification settings - Fork 65
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Sleep] Generalize Sleep
module to facilitate sleep scoring of non-human EEG/LFP data
#35
Comments
Sleep
module to facilitate sleep scoring of non-human EEG/LFP data Sleep
module to facilitate sleep scoring of non-human EEG/LFP data
Hello @paulbrodersen , First of all, thanks for compliments. Indeed, the entire idea of Visbrain is to offer modularity. About the doc and the description of the installation, feel free to fix/enhance the current description. For the generalization to non-humans, I've nothing against but may be @raphaelvallat have something to add. But it will requires some minor and major adjustments. May be the best would be to have a json configuration file. Something like this : {
"REM": {
"color": "green",
"shortcut": "r",
"value": 0,
},
"N1": {
"color": "red",
"shortcut": "1"
"value": 1
},
"N2": {
"color": "blue",
"shortcut": "2"
"value": 2
}
} An then the |
That is exactly what I had in mind. The human config file would obviously be the default (thus keeping backwards compatibility) but then we could provide other config files and maybe a recipe in the docs on how to make your own. Thanks for the quick reply! |
+1, I would also love to see this project add support scoring for non-human EEG, LFP, and EMG. My experience very much echos Paul's. I work in a lab that uses SleepSign, because it is the industry standard software for scoring animal sleep. I would love to see an open source alternative. As Paul mentioned, most people in the field score in 4-second epochs. Perhaps the largest impediment to using Sleep for manual scoring of animal sleep is that there does not currently appear to be a way to separate the epoch length from the window length. Unlike with human sleep, it is essential to be able to see epochs immediately before and after the 4-second epoch being scored. SleepSign's UI for this is excellent: example. Flexible vigilance state definitions are also essential. As Paul mentioned, rodent vigilance states differ from those of humans, and also differ from lab to lab. Some labs, for instance, allow for an "intermediate stage" during transitions to REM, and often distinguish between "quiet wake" and "active wake." I know that my lab often needs to score non-sleep states (e.g. drug induced EEG states) as well. I think it is worth pointing out that this feature would likely also be useful for researchers doing human studies as well. For example, my lab uses custom software that include "seizure" and "post-ictal" vigilance states, and uses this to score sleep in epileptic patients. If these two things were changed, I think that you would likely see other researchers transition to Sleep. |
@raphaelvallat what do you think about it? |
Hi all, @EtienneCmb it seems to me that the hardest part is to separate the epoch length from the window length, isn't it? That said, I agree that this could indeed represent a big step forward for sleep. @paulbrodersen quick question, is there any chance that you could send us a SleepSign hypnogram file? I think we could definitely add an option to load such file formats if you think that this might be useful too. Have a great day, |
@raphaelvallat I can send you an example file, sure. I don't have a copy of sleepsign myself but I am meeting my collaborator early next week, and then I can get one for you. However, a problem with the sleepsign hypnogram format is that it is not a format. It is my understanding that pretty much everything in that file can change depending on your export settings, including number of header rows, number of columns in the body, etc. You can probably mitigate most of that with some regex but it might still end up a nightmare to maintain. Personally, I would stick to the sane formats that you are already supporting. Anybody I know who uses sleepsign then analyzes their data in python or matlab. So I would maybe add a recipe in the docs on how to convert sleepsign hypnograms to your format (and then let the user do the work) but I wouldn't necessarily make it part of the pipeline. All of that being said, I can ask my collaborator to export the hypnogram of one example data set with a bunch of different export settings, and then either you or I can have a go at trying to figure out if we can reliably recover the states and time intervals from those regardless of the export settings. It might end up not being all that bad -- I just haven't tried yet. |
@raphaelvallat actually I don't think the hardest part would be to separate epoch length from the window length. To me the hardest part is to make stages and properties modular (i.e colors, shortcuts...). But this is totally doable, just a question of time. In the first place, I can to try to make our current version modular using a json file without hard coded stages and properties. The split of epoch length and window length could comes in a second place.
|
@raphaelvallat The SleepSign "Results Analysis File" format is proprietary, but you can export the hynogram info as a text file by going to "Trend -> Text output." As Paul mentioned, there are various options, but here is an minimal example of what is produced by default: SleepSign_TrendExample_TSV.txt. I agree with Paul, that providing a recipe for people to convert these files to a format you already support makes sense. @EtienneCmb When I first installed Sleep, it took me a moment to realize that it is a module of the visbrain package, and not available as a standalone package. It only took me about 30 seconds to figure this out, but I think that adding a note in the "Import and Use Sleep" section explicitly saying that you need to install visbrain first might be useful. |
@EtienneCmb Regarding the installation instructions, I would additionally note that if you want to use any of the Regarding the automatic scoring, I am happy to write some bindings to the stuff that I have been developing. The advantages of my algorithm are that
I will have a look at human data next week. If you are interested, I will keep you updated. @raphaelvallat For comparison with @grahamfindlay's sleepsign hypnogram, here are the first 100 lines from one of my files. @grahamfindlay Thanks for posting the default file format for me, now I don't have to chase my collaborator. |
Thanks @paulbrodersen for your feedback about the installation. I don't think you need to install every optional dependencies of both visbrain and MNE in order to only load files. But I can list the needed packages for Sleep only so that users install only what's needed. Also, I can specify that Sleep is a module of visbrain in the doc (thanks @grahamfindlay ). |
I don't think you really need the MNE environment unless you plan on using |
My lab is exploring software for mouse EEG analysis and @grahamfindlay just pointed me to this thread when I asked him by email for software suggestions. I just wanted to chime in and say how great it'd be to see this feature get implemented. |
I'll take a look at it next week. I'll post updates on this thread as soon as possible. Thanks for your comments ! |
Hi @EtienneCmb and @raphaelvallat, my work schedule is a bit less crazy the next couple of weeks so I am planning to fork out a day a week to work on this. I think it would be good if we could get together (virtually) and make a list of architecture changes that is as concrete as possible (i.e. function |
Hi all, To frame the discussion a bit, here is a first batch of ideas about what would/could be needed, and proposals on what it could look like, hopefully without messing to much with the current architecture.
This is the most urgent aspect as far as we are concerned (since we can hack around each of the following points) so I'd work on this first. There's probably a whole lot of subtleties here that I'm not envisioning (yet?) so feedback is most welcome :)
I'd be very happy to discuss and get feedback on this before I get my hands too dirty! @EtienneCmb , I am currently in Paris, maybe we could have a chat on the phone and why not possibly meet at some point ? |
@TomBugnon thanks for your feedback, clearly explains ! This represents quite a lot of work, and to be honest with you, I'll have no time to contribute to the development of animal support (also, I've never worked with animal data !). I think the easiest thing for now would be that you fork visbrain, make your changes and once it's working ping me and @raphaelvallat so that we can see your modifications. About file support, we tried to keep the number of dependencies as low as possible, also because visbrain contains other modules than Sleep. Cheers from marseille, |
Hi @EtienneCmb @raphaelvallat @grahamfindlay and all, I implemented the decoupling between scoring window and display window, and the "window hypnogram" that I outlined above in this fork: https://github.com/TomBugnon/visbrain/commits/scoring_window On startup, the GUI is in "Locked" mode, meaning the scoring window and the slider stepsize are equal to, and follow the display window, so user experience remains the same. When changing the "scoring window" size, we "unlock" the scoring window from the display window. The indicator bars for the scoring window are now visible, and the slider stepsize now follows the scoring window. The user can now score in shorter slices while seeing the peripheral data, and also see the stages of the window data by displaying the "window hypnogram" Re-locking the scoring window onto the display window hides the "window hypnogram" and the scoring window indicator bars, and sets the scoring window and stepsize back to the display window value as in the original state. Note that the staging of the very start and end of the recording is not possible (or a bit funky) but I suppose that's not too big a deal for now. Let me know if you have specific suggestions about the coding or interface and I can incorporate them, otherwise I'll send a pull request? |
Wow, impressive! This looks great @TomBugnon! I'll have to try it with some of my own data, but it seems like a great addition to Visbrain. Have you updated the documentation and the tests as well? Please feel free to submit a PR so that @EtienneCmb and I can have a deeper look at the changes. |
Hi @raphaelvallat and @EtienneCmb, I updated the documentation and tests and submitted the pull request. It would be great if you could have a look at it, we are starting to actually use this updated version to sleep score real data in our lab and it would make things easier if these changes were merged. I'll be happy to make some changes to the PR as you see fit. |
Scoring window is done 🍾 |
First of all: fantastic project. It installs fine (although I would change the installation instructions a bit), mostly everything just works, the docs are great, and the code is very readable (at least in the parts that I have looked at).
Secondly, I am raising this issue because I hope this will ultimately end in a PR (from me) but I wanted to get the conversation started before I start coding, so that you can stop me dead in the track before I veer off into areas that you (or any of the other maintainers) don't want to go.
Goal
Generalize the
Sleep
submodule to allow / facilitate sleep scoring of non-human EEG/EMG data, specifically from mouse.Motivation
I have implemented an automated sleep scoring algorithm that achieves a very high accuracy / concordance with experienced experimenters (98.5% on a held out test set). Nevertheless, most experimenters would still like to be able to amend the annotation, e.g. to annotate artifact states.
However, the sleep scoring software (sleepsign) that people in my lab and other labs in the institution use is a) proprietary (including the file format) and b) doesn't allow the import of a hypnogram from a different source (it has a lot of other issues, too, but those are the most pressing for me at the moment). Trying to integrate my pipeline with their existing sleep scoring pipeline is hence a bit of a non-starter. The integration with visbrain, in contrast, seems pretty straightforward (I love the fact that you provide templates to replace existing functionality).
Proposed changes
This list will undoubtedly grow as I and others in my lab actively use visbrain / Sleep.
So far I have got:
Allow definition of arbitrary vigilance states, e.g. in a config file, instead of hard coding N1, N2, N3, REM, wake, and artifact. "Mouse people" don't distinguish between different non-REM states; however, they do denote an additional "sleep movement" state, which occurs when the animal moves (and is potentially briefly awake) during sleep phases.
Associated with arbitrary vigilance states, allow arbitrary(-ish) shortcut keys, which could again be implemented by separating out some keybindings to a config file. My intended target audience has scored dozens and dozens of complete days of data at 4 second intervals, and I don't want to mess with their muscle memory at this stage.
I would love to hear your thoughts and feedback on this proposal.
I would completely understand if you would rather keep everything human-centric, as the proposed generalisation will increase the complexity and scope of the project. That being said, the code base seems pretty modular so I don't think this would entail massive changes all in all (but you would have a better idea of this). Also, such a move could drastically increase your user base (which would be much deserved!), and bring in some fresh ideas.
Looking forward to your reply,
Paul
The text was updated successfully, but these errors were encountered: