Modeling does not produce reasonable values

[trixing]

Hi,

I wrote a quick nightscout-to-tune input converter (https://github.com/trixing/tune). It produces json files like attached, which seems to conform to the specified format (for 1, 3, 7 days respectively in this case).

Unfortunately the produced basal schedule is rather meaningless: a factor of ~10x too high, and also the same result if I run on 3 or 7 days.

3-day data
{"version": 1, "timezone": "Europe/Berlin", "insulin_sensitivity_schedule": {"index": [0, 390], "values": [170.0000000000016, 169.99999999999957]}, "carb_ratio_schedule": {"index": [0, 360, 660, 1050], "values": [9.000000000000052, 9.000000000000027, 9.000000000000002, 9.000000000000005]}, "basal_rate_schedule": {"index": [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380], "values": [6.060164554997598, 5.789143251467735, 6.669406893763722, 7.100211844444492, 7.206123554462863, 7.198766166916485, 7.2001180743763005, 7.200090756536767, 7.1998197971922835, 7.200449445844885, 7.198854832952293, 7.202448204774257, 7.1960631269896504, 7.202852101333228, 7.2101639269212, 7.1397788680270615, 7.410869150932365, 6.530692446003393, 6.099377983276996, 5.995011592147588, 5.998777410458621, 6.0038361020430315, 5.997054760167179, 5.98992515397004]}, "training_loss": NaN}

7-day-data
{"version": 1, "timezone": "Europe/Berlin", "insulin_sensitivity_schedule": {"index": [0, 390], "values": [170.0000000000016, 169.99999999999957]}, "carb_ratio_schedule": {"index": [0, 360, 660, 1050], "values": [9.000000000000052, 9.000000000000027, 9.000000000000002, 9.000000000000005]}, "basal_rate_schedule": {"index": [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380], "values": [6.060164554997598, 5.789143251467735, 6.669406893763722, 7.100211844444492, 7.206123554462863, 7.198766166916485, 7.2001180743763005, 7.200090756536767, 7.1998197971922835, 7.200449445844885, 7.198854832952293, 7.202448204774257, 7.1960631269896504, 7.202852101333228, 7.2101639269212, 7.1397788680270615, 7.410869150932365, 6.530692446003393, 6.099377983276996, 5.995011592147588, 5.998777410458621, 6.0038361020430315, 5.997054760167179, 5.98992515397004]}, "training_loss": NaN}

I'm uncertain though if I'm inputing everything correctly. My understanding is that "tune" expects a delta amount from the scheduled basal rate for example? The delta modeling is also easy to get wrong, but it looks correct.

I'd be interested to get the model to work, just as a reference point to autotune and manual tuning.

There is also code in model.py which does a cumsum ("undelta") on the basal durations. I'm not sure how that is useful but uncommenting the line doesn't change the result, which is a bit surprising. In general the algorithm seems a bit too stable?

Archive.zip

[trixing]

Never mind, I didn't get the delta encoding right.

[trixing]

Alright, 2 hours later, I believe I got the encoding right, and the time series frames in model.py have reasonable values (see updated input).

But, now in make_frame the filter functions do filter away pretty much all the rows, as the insulin_quantile of 0.9 is close to zero (after all most 5 minute buckets don't have any insulin deviations).

The results now at least looks very different based on the run, e.g. for 7 days
{"version": 1, "timezone": "Europe/Berlin", "insulin_sensitivity_schedule": {"index": [0, 390], "values": [168.7648071775515, 166.580849031434]}, "carb_ratio_schedule": {"index": [0, 360, 660, 1050], "values": [29.543442916910497, 60.89577424725448, 42.71775564966234, 86.57834389137774]}, "basal_rate_schedule": {"index": [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380], "values": [0.0, 0.01683845443292576, 0.009304439179148784, 0.0, 0.015550291140290294, 0.0, 0.021544467924777225, 0.030454619090469835, 0.0, 0.004094063022135545, 0.0010431431256149853, 0.0, 0.0, 0.0, 0.0, 9.337685136187613, 0.0, 0.0, 0.0, 6.359425603971928, 0.0, 0.0, 0.0, 0.0]}, "training_loss": 1.6218057934112462}

For 1 day
{"version": 1, "timezone": "Europe/Berlin", "insulin_sensitivity_schedule": {"index": [0, 390], "values": [169.9861522447872, 169.95984282907307]}, "carb_ratio_schedule": {"index": [0, 360, 660, 1050], "values": [8.960328718953333, 9.010907141272734, 8.91831603560555, 10.219587045976263]}, "basal_rate_schedule": {"index": [0, 60, 120, 180, 240, 300, 360, 420, 480, 540, 600, 660, 720, 780, 840, 900, 960, 1020, 1080, 1140, 1200, 1260, 1320, 1380], "values": [0.0, 4.213294125952607, 7.730384211426936, 6.643074938538296, 7.710495509939287, 5.62405563839342, 11.41221821793468, 0.0, 0.0, 4.455661447632698, 11.36106643780809, 0.0, 4.20350736941698, 9.413230328422756, 6.071096940924363, 7.8219406331574, 6.8080213748445555, 6.8108565053164325, 7.630663852887077, 0.0, 7.489041970183893, 0.0, 0.0, 6.933545395325286]}, "training_loss": 3.114477865317038}

Some factor is probably still quite a bit off. Can you spot it?

Archive.zip

[PieterGit]

Hi @trixing . Did you do some more work on the @mariusae scripts and models?
I just started to review your nightscout script. I noticed you use the following basal_insulin_parameters

                # +- Fiasp, not sure about duration
                'delay': 8,
                'peak': 44,
                'duration': 200,

According to these references: https://seemycgm.com/2017/10/21/exponential-insulin-curves-fiasp/ and https://github.com/LoopKit/Loop/blob/26b11d72399b99677423b176d5cf1826dedb8def/LoopCore/Insulin/ExponentialInsulinModelPreset.swift#L18-L51 a better version for Fiasp would be

                'delay': 10,
                'peak': 55,
                'duration': 360,

Updates:

I can also confirm I see a basal schedule a factor of ~10 too much. Will try to do some more investigation later.

@trixing: I still didn't find the root cause why the basal schedule seems off. I also see almost no correction to ISF or CR.
I noticed some things with basals:

• I see negative durations for basals. I would not expect that
• According to docs "Basal insulin values are given in milliunits per hour (mU/h).". You seem to use U/h and subtract the current basal profile. Shouldn't the amount of basal insuline t['rate'] be used (multiplied by 10^3)?

I will investigate further

[PieterGit]

@trixing I reimplemented the basals export and did some additions to your script (startdate/enddate, fix max instead of min).
See https://github.com/PieterGit/tune or trixing/tune@master...PieterGit:master for differences compared to your branch. Unfortuately I have no luck in getting usefull model/train output yet. There might be something wrong with my basals for multiple days.

[trixing]

I ended up giving up on trying to fix it back then and implemented the gist of it as a colab model using scipy:

https://colab.research.google.com/drive/1bbjvabg9_y98ULzO1ZVFFy2pc4gkLgJO?usp=sharing

Feel free to play around with it. It's unfortunately pretty undocumented, but I'm sure you can follow along.

To convert Nightscout data into the format "tune" (and the colab model) expects, I wrote an elaborate script back then, not sure if it still works, but feel free to give it a try. I pushed it here: https://github.com/trixing/tune/blob/master/nightscout_to_json.py

[trixing]

I also pushed the converter and colab to my fork of tune, might be easier for collaboration than having the converter in another repository.

https://github.com/trixing/tune