Extracting hypno as csv?

[PhD-GOAT]

Sleep stages (aka hypnogram, .csv format, 3KB), in 30-seconds epoch. The sleep stages are encoded as follow: 0 = Wake, 1 = N1, 2 = N2, 3 = N3, and 4 = REM.

In your quick start, hypno is a one-dimensional arrangement. But normally, clinical data doesn't seem to be one-dimensional, so how do you convert str arrays?

[raphaelvallat]

Hi @PhD-GOAT I don't understand what you mean by "one-dimensional", please clarify

[remrama]

Your original post includes a link to a csv file, so I was under the impression that's what you were trying to load in. You don't need to convert the edf to csv. Forget about the edf and download the csv that you linked to in your original post. Name that accordingly and load it in.

[PhD-GOAT]

https://physionet.org/content/sleep-edfx/1.0.0/

I referred to this site, but I don't think there is a separate csv file. First of all, thank you.

[remrama]

Oh okay. I think I'm having a moment of clarity.

The hypnogram from the physionet dataset comes in edf format. The hypnogram from yasa comes in csv format. So you are able to load in the yasa hypnogram using pandas, but you're struggling with the physionet hypnogram. Well, they're different filetypes so you can't treat them interchangeably. You will have to load the physionet hypnogram using MNE and explore that file to extract the hypnograms. Notably, I don't think this is the place to discuss that further.

Based on how this and many of your other threads are ending, I am now realizing that your struggles are with the physionet dataset specifically and probably don't relate to yasa in any specific way. I mean, the issues are not about yasa's ability to handle the data, or how yasa works, it's just about the structure/format of the physionet dataset and how to read it. It's not as simple as running just any edf file through the tutorial scripts and expecting things to run smoothly.

In my opinion the yasa discussion threads are probably not the best place to discuss the details of the physionet dataset (again, unless there are some yasa-specific issues or something, but I don't think that is or would be the case). I understand why it seems relevant and how the questions are arising, but ultimately the only connection is that it's sleep data. @PhD-GOAT my suggestion is to dedicate some time to understanding the ins-and-outs of the physionet dataset before trying to analyze it in detail. That may or may not involve finding another resource for asking questions about how to read it into python. I hope that helps.

[PhD-GOAT]

I agree. So I wonder if I should change the API of Yasa by myself or modify the data if it is a physical data problem. If it is the former, I think there will be a functional problem because I changed it. Don't you think so?

[remrama]

Great! Now we're talking!

You do not need to mess with yasa in any way. It's not going to solve any of your issues. Again, this isn't related to how yasa handles data. Instead, you need to work on extracting the data you want from physionet before even considering how to process it with yasa. Don't even load in yasa until you have the data you want from physionet. For example, you're final script might involve a top section that is purely dedicated to extracting the relevant physionet data and getting it to look the way yasa wants it, only to then process it below with yasa.

So you need to play with physionet in python before loading yasa. I would suggest looking at this page in detail. That tutorial classifies sleep stages using just MNE and sklearn, and it uses the physionet dataset! So you should see how they extract hypnograms in that script.

Now yasa will likely outperform that tutorial, so you just want to use that to understand physionet data in python and then take what you want to run your own yasa script. I'd still suggest working all the way through that one before yanking from it.