💪 handle nans

[jvdd]

Assuming that there are no NaNs in the data is the largest assumption that this library makes.

Why is this a large assumption

Plotting libraries use NaN values to represent gaps (i.e., disconnected parts of the time series).

How should we tackle / handle NaNs

Ideally we would have a NaN-policy. Currently I see only 2 options that this policy should support:

• omit NaNs (skip / ignore them when performing the downsampling)
• return NaN once there is one present in the considered data (e.g., in the bin)

For the MinMaxDownsampler, M4Downsampler, and (largely) the MinMaxLTTBDownsampler, this would boil down to adding NaN support (with some sort policy as described above) to the argminmax crate.

How we would support NaNs in the LTTBDownsampler is not yet 100% clear to me.

[gfmartins]

Don't know if this helps, but timescaledb implements an gap preserving function for the LTTB algorithm
https://docs.timescale.com/api/latest/hyperfunctions/downsample/gp_lttb/

source: https://github.com/timescale/timescaledb-toolkit/blob/7fc374999160c42b55c8be3c3723314ec129a354/extension/src/lttb.rs

[jvdd]

Looks quite interesting - will look into it! Perhaps we can integrate / re-use functionality :)

Thanks for sharing this @gfmartins