Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inaccurate light/deep sleep detection #250

Closed
bigretromike opened this issue Mar 11, 2016 · 28 comments

Comments

Projects
None yet
6 participants
@bigretromike
Copy link

commented Mar 11, 2016

Im not sure if this is mi band 1s related or not but...
Im using the GB for 2 days now - with few yey and nay...
But in the sleep data I have mine sleep chart reverse with compare to mi fit that I used before flashing phone.
currently i Have 7:26 h sleep, with 6:25 deep sleep and 1:01 light sleep. This been allways the oposite in mi fit - so im not sure if this is reversed or the data from mi band have reverse standard... I doube that I had good night ;-)

@danielegobbetti

This comment has been minimized.

Copy link
Contributor

commented Mar 16, 2016

see the comment I left in #257

if you have resources related to actimetry algorithms to recognise deep sleep please post them to #232 or our wiki!

@bigretromike

This comment has been minimized.

Copy link
Author

commented Mar 17, 2016

Like you post, if its inverted it would look ok.
For me its ONLY inverted: 6:39 = 0:54 Light + 5:45 Deep (definityly MY TYPE of Data but inverted) Its looks almost legit as mi-fit app would throw at me, but inverted.

I think you nailed it but need to just invert for mi band 1s !
Im 100% sure of those numbers.

@bigretromike

This comment has been minimized.

Copy link
Author

commented Mar 23, 2016

0.9.0 still have those value inverted

@danielegobbetti

This comment has been minimized.

Copy link
Contributor

commented Mar 24, 2016

-- deleted a few comments --

@computerlyrik

This comment has been minimized.

Copy link
Contributor

commented Apr 2, 2016

Just updated to FW 4.15.12.10 and GB 0.9.3
Will report tomorrow about sleep data.
With GB 0.9.1 and FW 4.11 sleep data was indeed inverted.

@bigretromike

This comment has been minimized.

Copy link
Author

commented Apr 2, 2016

sadly FW update to 04.15.12.10 wont flip those, also authors didnt confirm fliping them on 0.9.3 :-) so we will have to wait till they make enought research to flip those values :/ until then i will lie to myself that I have sooo much deep sleep ;-)

@bigretromike

This comment has been minimized.

Copy link
Author

commented Apr 29, 2016

In "Mi Band Settings" I enabled "Use Heartrate Sensor to improve sleep ..." and I see hr on sleep graph for few days, but that didn't helped with inverted values.

@bigretromike

This comment has been minimized.

Copy link
Author

commented Jul 7, 2016

0.11.1 - the data is still inverted, "Use Heartrate Sensor to improve sleep ..." is enabled all the time. It looks like M1S flip those because I dont see anyone that report those "flips" and probably M1S is less popular her anyway.
Its been some time now... Will this be fixed or should m1s owners move out ?

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Jul 15, 2016

It will get fixed eventually. If there are other people that can confirm that Gadgetbridge displays the opposite of Mi Fit, then we can indeed simply flip those values.

@rplevka

This comment has been minimized.

Copy link
Contributor

commented Jul 16, 2016

@cpfeiffer I think I can confirm what @bigretromike says. I've been using GB with Mi Band 1S for 3 days now and here's what it says about my last two nights:

Light Deep Total
42min 4h 49min 5h 31min
44min 4h 12min 4h 56min
@rplevka

This comment has been minimized.

Copy link
Contributor

commented Jul 17, 2016

There's definitely got to be some post processing in place on the APP side of things. The device seems to send you data for individual 'epochs' (every minute) with labeling them with activity type. However, you can't get the real deep sleep duration just by counting the number of minutes marked as 'deep sleep'. There should be some algorithm that would look for some patterns (e.g. If there were any light sleep epochs in some interval, mark whole interval as Light Sleep).

@bigretromike

This comment has been minimized.

Copy link
Author

commented Jul 17, 2016

To be honest those inverted values are almost tge same as official mi band application, but like i wrote those values are inverted, if we invert those it will look almost as the data from mi fit, so I wouldnt worry much about post processing

On July 17, 2016 11:31:17 AM GMT+02:00, Roman Plevka notifications@github.com wrote:

There's definitely got to be some post processing in place on the APP
side of things. The device seems to send you data for individual
'epochs' (every minute) with labeling it with activity type. However,
you can't get the real deep sleep just by counting the number of
minutes marked as 'deep sleep'. There should be some algorithm that
would look for some patterns (e.g. If there were any light sleep epochs
in some interval, mark whole interval as Light Sleep).


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#250 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

@rplevka

This comment has been minimized.

Copy link
Contributor

commented Jul 17, 2016

@bigretromike I checked on the values, but aside the activity type there's also an intensity value (intensity of movement) - for those marked as 'deep sleep' the intensity is very low and for those marked as 'light sleep', the intensity is quite high. That's why I'm not sure whether they're just inverted.

@bigretromike

This comment has been minimized.

Copy link
Author

commented Jul 17, 2016

Im unsure because I dont debug is as hard but for me its clearly that deep sleep is inverted with light sleep, maybe those values are in % where 100% is deep sleep so the deep sleep hours have bigger counter, that im unsure but im sure that mi fit showed me from day one that my deep sleep is usuly less than hour and gb tell me that my light sleep is only less than 1h but me deep sleep is 5-7 h, which is inaccure as hell.

On July 17, 2016 12:04:40 PM GMT+02:00, Roman Plevka notifications@github.com wrote:

@bigretromike I checked on the values, but aside the activity type
there's also an intensity value (intensity of movement) - for those
marked as 'deep sleep' the intensity is very low and for those marked
as 'light sleep', the intensity is quite high. That's why I'm not sure
whether they're just inverted.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#250 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

@rplevka

This comment has been minimized.

Copy link
Contributor

commented Jul 17, 2016

@bigretromike totally, same here. My point was that simply inverting these values won't be right either. It's just a funny correlation that inverted values are closer to the reality. I believe the most of the 'deep sleep' epochs would be simply converted to 'light sleep' if a correct sleep-cycle detection algorithm would be in place. That'd almost look like an inverted DS/LS but still would be different.
Even if I invert the values now, I still see 1+ hour error comparing to my mi fit measurements.

@bigretromike

This comment has been minimized.

Copy link
Author

commented Jul 17, 2016

So i understand you use both app right? Its problem for me because i totaly droped all app and stayed with gb :-) thats why it iritates me that much. But showing friend gb and sayig 'its best fit app and its open source but it invert my data' aint that good for me ;)

On July 17, 2016 12:24:17 PM GMT+02:00, Roman Plevka notifications@github.com wrote:

@bigretromike totally, same here. My point was that simply inverting
these values won't be right either. It's just a funny correlation that
inverted values are closer to the reality. I believe the most of the
'deep sleep' epochs would be simply converted to 'light sleep' if a
correct sleep-cycle detection algorithm would be in place. That'd
almost look like an inverted DS/LS but still would be different.
Even if I invert the values now, I still see 1+ hour error comparing to
my mi fit measurements.


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#250 (comment)

Sent from my Android device with K-9 Mail. Please excuse my brevity.

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2016

Hehe. I totally agree that we need some improvements here.

As a quick fix, I can indeed invert those values, so that the values are at least a bit more realistic. Are you willing to work with me on getting a better algorithm? You don't even need to hack Gadgetbridge, know Android or Java. A description of an algorithm how to calculate the amounts from the samples you see would already help.

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2016

Oh, and comparisons with other apps, including Mi Fit help as well.

@bigretromike

This comment has been minimized.

Copy link
Author

commented Jul 18, 2016

I would have to install mi fit then, but Im fine with helping you. Just say what I need to install, prepare so I will collect data you need. Currently im using GB only, but if I need mi fit or other app just say.

@bigretromike

This comment has been minimized.

Copy link
Author

commented Jul 18, 2016

Just invert those values 👍

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Jul 18, 2016

If you don't have Mi Fit, that's fine. Maybe there are others who have it installed and can share comparisons, e.g. screenshots of their data.

@rplevka

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2016

@cpfeiffer for a 3rd night in a row I've been trying to download my sleep data to GB with "keep data on a band" option checked, but when i fire up Mi Fit to sync the data, it says there's none anymore. Is that a bug in GB or am I doing it wrong?

cpfeiffer added a commit that referenced this issue Jul 19, 2016

Switch light sleep and deep sleep #250
Apparently REM is considered deep sleep and NREM is considered light
sleep even though NREM (non-rapid-eye-movement) phase 3 is actually
defined as deep sleep.
@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Jul 19, 2016

@rplevka For me it does work. Didn't test with Mi Fit, but if I have this option enabled, I can sync twice and get the same samples every time.

Maybe Mi Fit already fetched the data? Or is it simply not available in Mi Fit?

@rplevka

This comment has been minimized.

Copy link
Contributor

commented Oct 3, 2016

I can confirm, the data stay on the band, however mifit won't pick them up for some reason

cpfeiffer added a commit that referenced this issue Oct 5, 2016

@gadgetguy08

This comment has been minimized.

Copy link

commented Nov 7, 2016

Just got a Mi Band 1S.
FW 4.6.4.22
HR 1.6.1.22
GB 0.13.7

Arm right

Sleep data shows Deep sleep is lighter color and Light sleep darker. But the "activity" bars are taller for the light bars. Also there were lighter bars at one point when I know I was awake, making me wonder if they are reversed for me.

Attemped to post a screen shot with the Android app..

@rplevka

This comment has been minimized.

Copy link
Contributor

commented Nov 7, 2016

@gadgetguy08 yes. This is simply caused by the fact that guys felt like switching light and deep sleep types gets more appropriate results when it comes to total light and total deep sleep.
(just read the thread above).
I myself beleive, the values were right, but there was some additional postprocessing which turned some sequences of deep sleep to the light sleep etc.

so I also believe, we should not really invert the values

@bigretromike

This comment has been minimized.

Copy link
Author

commented Nov 7, 2016

for me current values look ok;
sadly its hard for me to check each version as always I end up wiping those data out of both db and band :/

@cpfeiffer

This comment has been minimized.

Copy link
Contributor

commented Nov 7, 2016

We're aware that the mobile is not the ideal place to store the data forever. An external storage service is planned and there is also some experimentation code already.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.