Skip to content
A time series + geographical coordinates compression
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Build Status


TSD is a time series storage specialized in compressing increasing timestamp and geo spacial data (lat, lng).

It performs better than Snappy & LZ4, on this specific usage without any advanced optimizations but storing deltas and storage size reduction.

Size: 64776 Snappy 61007    LZ4 57225       TSD 23948
Size: 18864 Snappy 18870    LZ4 18267       TSD 9883
Size: 15768 Snappy 12721    LZ4 11860       TSD 4775
Size: 7764  Snappy 5440     LZ4 5148        TSD 2122
Size: 9168  Snappy 7251     LZ4 6933        TSD 3018

Here is a blog post I wrote to describe this project


It implements BinaryMarshaler and BinaryUnmarshaler

ts := New()
ts.Push(1201984833, 39.91071, 116.50962)
ts.Push(1201985433, 39.91588, 116.52231)
ts.Push(1201986033, 39.91445, 116.56444)
b, _ := ts.MarshalBinary()

Format is as follow

header ts uint32,   lat float32, lng float32
      32                  32           32

info,  time delta dyn,   lat delta dyn, lng delta dyn
 8            n,               n,             n

n can be 0, an int8, int16 or int32.


The Facebook's Gorilla paper would perform better but my needs were a bit different.

I needed a time window larger than the limit imposed by the paper (14 bits ~ 4 hours), and a way to store 2 values lat, lng that are closed to the previous one (moving devices).
The timestamp storage is almost the same as the paper but for the values a delta of delta and appropriate storage per entry are good enough and perform better than dict compression.

You can’t perform that action at this time.