Menoetius is a timeseries storage system designed to store prometheus-compatible data.

Build requirements:
 - make
 - gcc
 - libmicrohttpd
 - libcrypto
 - libcunit (for testing only)

Building:

    make

Running tests:

    # unit tests
    ./test

    # we have some heavier-tests that ensure the server can handle reloading data after crashes
    # this test can take 10 minutes to run. If it fails, check the server.log file for additional errors
    ./smoke test1


Running the server:

    MENOETIUS_STORAGE_PATH=/tmp/storage_path MENOETIUS_INDICES=instrument,user_id ./menoetius-server
    
Sending data to the server via the cli:

    ./menoetius-cli put 'num_instruments{instrument="flute"}=3'

    # or send historical data
    ./menoetius-cli put 'num_instruments{instrument="flute"}=2@2019-10-23T17:10:02'

Fetching data to from server via the cli:

    ./menoetius-cli get 'num_instruments{instrument="flute"}'

    # this will output something similar to:
    1572537106 3.149999
    1572548947 3.149999
    1572548974 3.149999
    1572549018 6.000000

    # use the -t flag to format the time
    ./menoetius-cli get -t 'num_instruments{instrument="flute"}'

    # which will output
    2019-10-31T15:51:46 3.149999
    2019-10-31T19:09:07 3.149999
    2019-10-31T19:09:34 3.149999
    2019-10-31T19:10:18 6.000000

Querying metrics:

    # send metrics for 3 different instruments:
    ./menoetius-cli put 'num_instruments{instrument="triangle"}=0'
    ./menoetius-cli put 'num_instruments{instrument="guitar"}=2'
    ./menoetius-cli put 'num_instruments{instrument="flute"}=1'

    # then query all available LFMs:
    ./menoetius-cli query 'num_instruments'

    # which outputs
    num_instruments{instrument="flute"}
    num_instruments{instrument="triangle"}
    num_instruments{instrument="guitar"}

    # one can also query by labels:
    ./menoetius-cli query '{instrument="flute"}'

    # which outputs
    num_instruments{instrument="flute"}

Benchmarking:

    # build a release version
    make clean && make release

    # clear old storage (if applicable)
    rm /tmp/storage_path/*

    # start the server
    LOG_LEVEL=ERROR MENOETIUS_STORAGE_PATH=/tmp/storage_path MENOETIUS_INDICES=instrument,user_id ./menoetius-server

    # run a benchmark
    LOG_LEVEL=ERROR ./menoetius-cli benchmark

    # example output:
    benchmark progress: 100%
    sent 14400000 total samples in 61.434876 seconds
    number of metric names = 10000
    number of points per metric = 1440
    ingestion rate = 234394.55 metrics/second

Sample Benchmark:

    Running on a i7-8650U CPU @ 1.90GHz
    
    test 1: 10,000 unique metrics named "benchmark{i=0..9999}"; each metric storing 3.14 as the value repeated 1,440 times.
    ingestion rate of 234394.55 metrics/second. Disk usage of 1.52 bytes per data sample (21MB*1024*1024 / 14,400,000 data points).
    
    test 2: 1,000 unique metrics named "benchmark{i=0..999}"; each metric storing 3.14 as the value repeated 14,400 times. (10x more data points, 1/10th metric names)
    ingestion rate of 254599.16 metrics/second. Disk usage of 0.37 bytes per data sample (5.1MB*1024*1024 / 14,400,000 data points).
    
    test 3: 100 unique metrics named "benchmark{i=0..99}"; each metric storing 3.14 as the value repeated 144,000 times. (10x more data points, 1/10th metric names)
    ingestion rate of 258741.93 metrics/second. Disk usage of 0.28 bytes per data sample (3.8MB*1024*1024 / 14,400,000 data points).
    
    TODO: need to get actual server metric timeseries to use for this benchmark, then add some graphs here.

    


Building a deb package: a dockerfile is provided for building a debian package, first
install docker, then run:

    ./build-deb.sh

Writing data to the server: currently we only have a client library available
(see src/client/client.h), additionally the src/smoketest directory contains
examples of using the client.

Bug reports and pull requests can be submitted via the github project
located at https://github.com/digitalocean/menoetius

This program is distributed under the GPLv3.  Please see the file LICENSE
for full license information.