A Time Series data model API for Go and MongoDB
Mango TS is a Google Go package built on the mgo package which implements an optimized model for periodic and nonperiodic Time Series data stored in MongoDB.
From Wikipedia:
A time series is a sequence of data points, typically consisting of successive measurements made over a time interval.
Period data appears in cronological order, at regular, predefined intervals.
Examples:
- Performance monitoring metrics every minute
- Scheduled tasks every hour
Periodic data is arrange into pages, each representing a linear segment of time, with a slot preallocated for each expected data point. For example, if the data interval is one minute and the page size is one hour, each page will contain 60 preallocated slots; one for each minute of the hour.
The model implemented in mgots is loosely based on Sandeep Parikh's time series schema design.
Nonperiodic data appears in cronological order, but at irregular intervals. The data model used for periodic data is not appropriate for datasets of this kind as a significant amount of storage is wasted on preallocated intervals for which no data is entered.
Examples:
- Monitoring events at random
- User transactions at random
Nonperiodic data is arranged into a page until a configurable page size is exhausted (E.g. 4096 bytes) when a new page is allocated. Each page represents a linear segment of time, starting from the last entry point in the previous page, until the first entry point of the next page.
The model implemented in mgots is loosely based on Sylvain Wallez's model used in Actoboard.
A number of facters contribute to optimizing page sizes. These include but are no limited to:
- Disk alignment
- Alignment in memory of memory-mapped storage
- MongoDB's Power of 2 allocation strategy
- Memory consumption and network utilization when querying pages and returning data