User Guide

Table of Contents

1 Installation

1.1 Runtime Requirements

1.2 Running TickTock inside docker container

1.3 Install TickTock from a binary package

1.4 Compile TickTock from source

1.5 Verify TickTock running successfully

2 Configuration

2.1 Format of the configuration file

2.2 List of supported configurations

2.3 Set configuration options in TickTock docker

2.4 Performance tuning

3 Writing data to TickTock

3.1 Using TCollector

3.2 Using Telegraf

3.2.1 Using Opentsdb protocol for write

3.2.1.1 HTTP protocol

3.2.1.2 TCP protocol (suggested)

3.2.1.3 Restart Telegraf

3.2.2 Using Influxdb line protocol for write (suggested)

3.2.2.1 HTTP protocol

3.2.2.2 TCP protocol

3.2.2.3 Restart Telegraf

3.2.2.4 Query examples

3.3 Doing it yourself

4 Querying from TickTock

4.1 Querying TickTock on command line

5 Visualizing data using Grafana

1. Installation

The quickest way to get TickTock up and running is to run TickTock inside a Docker container. You can also install TickTock from a binary package (coming soon). Compiling from the source is another option.

1.1 Runtime Requirements

To run TickTock, you will need,

• A Linux system (e.g. Ubuntu, CentOS, RaspberryPI OS Buster & Bullseye, etc). We tested both Ubuntu, CentOS, RPI. Other Linux systems are not tested yet.
• glibc 2.28 or up (Buster/Bullseye with 2.28 or higher are tested)
• zlib 1.2.11 or up

Here's the list of shared object dependencies, from ldd.

$ ldd ticktock
    linux-vdso.so.1 (0x00007ffe701a4000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6ae9b28000)
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6ae9946000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6ae97f7000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6ae97dc000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6ae97b9000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6ae95c7000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f6ae9bc9000)

We strongly suggest you to update the open files setting in your OS from 1024(default) to a large number, e.g., 1048576. In ubuntu, do this:

ylin30@raspberrypi:~/ticktock $ ulimit -a
real-time non-blocking time  (microseconds, -R) unlimited
core file size              (blocks, -c) 0
data seg size               (kbytes, -d) unlimited
scheduling priority                 (-e) 0
file size                   (blocks, -f) unlimited
pending signals                     (-i) 1384
max locked memory           (kbytes, -l) 65536
max memory size             (kbytes, -m) unlimited
open files                          (-n) 1024
pipe size                (512 bytes, -p) 8
POSIX message queues         (bytes, -q) 819200
real-time priority                  (-r) 0
stack size                  (kbytes, -s) 8192
cpu time                   (seconds, -t) unlimited
max user processes                  (-u) 1384
virtual memory              (kbytes, -v) unlimited
file locks                          (-x) unlimited
ylin30@raspberrypi:~/ticktock $ sudo vim /etc/security/limits.conf
# Add these sample lines to the file. The followings are mine. Replace ylin30 with your account name.
*                soft    nofile          1048576
*                hard    nofile          1048576
ylin30           soft    nofile          1048576
ylin30           hard    nofile          1048576
# End of file
ylin30@raspberrypi:~/ticktock $ sudo vim /etc/sysctl.conf
# Add this line
fs.file-max=1048576
fs.nr_open = 1048576
ylin30@raspberrypi:~/ticktock $ sudo shutdown -r now

Note that you need to restart after modifying /etc/security/limits.conf in order for the new setting to take effect.

1.2 Running TickTock inside Docker Container

You need to install Docker Engine first (see instructions). You can run Docker Engine on any platform that supports it. Once the Docker Engine is installed, simply run

docker run -d --name ticktock -p 6181-6182:6181-6182 -p 6181:6181/udp ytyou/ticktock:latest

The first time you run this command, it will pull down the official TickTock image from Docker Hub before running it.

1.3 Install TickTock from a Binary Package

1. Download the latest (v0.20.0 on 4/21/2024) package here

2. Extract the file, e.g, ticktock-rpi-32bit.tar.gz, into a folder, ticktock/

   pi@raspberrypi:~/tmp $ gunzip ticktock-rpi-32bit.tar.gz
   pi@raspberrypi:~/tmp $ tar xvf ticktock-rpi-32bit.tar.gz
   

3. The structure of the ticktock is as:

   pi@raspberrypi:~/tmp/ticktock $ ls -l
   total 296
   drwxr-xr-x 2 pi pi   4096 Feb 17 20:15 admin         // A list of scripts for admin
   drwxr-xr-x 2 pi pi   4096 Feb 17 10:33 bin           // binaries
   drwxr-xr-x 2 pi pi   4096 Feb 17 20:26 conf          // a default config, conf/tt.conf
   drwxr-xr-x 2 pi pi  36864 Feb 17 20:28 data          // data dir in default config, conf/tt.conf
   -rw-r--r-- 1 pi pi  35149 Feb 17 20:16 LICENSE
   drwxr-xr-x 2 pi pi   4096 Feb 17 20:28 log           // log dir in default config, conf/tt.conf
   drwxr-xr-x 2 pi pi   4096 Feb 17 20:15 scripts       // A list of helpful scripts 
   drwxr-xr-x 2 pi pi   4096 Feb 17 20:17 tools         // A python file to collect metrics sent to Prometheus
   pi@raspberrypi:~/tmp/ticktock $
   

4. Run ticktock:

   pi@raspberrypi:~/tmp/ticktock $ bin/tt -c conf/tt.conf
   

Note that the default data dir is ./data and log dir is ./log. You may also override the configs in conf/tt.conf by specifying configs in the command line. E.g.,

   pi@raspberrypi:~/tmp/ticktock $ bin/tt -c conf/tt.conf --tsdb.timestamp.resolution millisecond --tsdb.data.dir /tmp

5. Gracefully stop ticktock:

We STRONGLY suggest to gracefully shutdown ticktock using <ticktock>/admin/stop.sh, although TT can still survives with kill -9 or host losing power suddenly. Without graceful shutdown, TT may lose data up to 5 minutes (default) in caches. Besides, there might be some loss of in-memory settings critical to TT.

ylin30@pi5:~/ticktock $ ./admin/stop.sh
Exiting now.
ylin30@pi5:~/ticktock $ Start shutdown process...
Shutdown process complete

[1]+  Done                    ./bin/tt -c conf/tt.conf --tsdb.timestamp.resolution millisecond --http.server.port 6182,6183

1.4 Compile TickTock from Source

You will need

• A Linux system (Ubuntu or CentOS)
• Git
• Make
• GCC
• Zlib 1.2.11 or later

On Ubuntu (latest version tested: Ubuntu 20.04.3 LTS), install these tools with

apt update
apt install -y git zlib1g-dev
apt install -y build-essential

On CentOS (latest version tested: 8), do this

yum install -y git zlib-devel
yum groupinstall -y "Development Tools"

Now clone the source code from GitHub, and build it using either Makefile.ubuntu or Makefile.centos, depending on your platform,

git clone https://github.com/ytyou/ticktock.git
cd ticktock
make -f Makefile.<os> all

If all goes well, you can now run TickTock,

bin/tt -c conf/tt.conf [-d]

Use the option -d to run TickTock as daemon.

1.5 Verify TickTock Running Successfully

1.5.1 You may need to install 'curl' if you haven't.

For ubuntu:

apt install -y curl

For centos:

yum install -y curl

1.5.2 Make sure TickTock process is running.

curl -XPOST 'http://localhost:6182/api/admin?cmd=ping'

If TickTock is healthy, it should response with pong.

1.5.3 Write a data point to ticktock.

curl -v -XPOST 'http://localhost:6182/api/put' -d 'put testM1 1633412175 123 host=foo'

1.5.4 Read the data point from ticktock

curl -v 'http://localhost:6182/api/query?start=1600000000&m=avg:testM1'

or:

curl -v -g 'http://localhost:6182/api/query?start=1600000000&m=avg:testM1{host=foo}'

Note the additional option "-g" is to encode '{' and '}' in curl.

1.5.5 Log locations and online setting

If you are interested in TickTock's logs, you can go to Deleveoper Guides. The default log location for ytyou/ticktock is /var/lib/ticktock/log/ticktock.log.

2. Configuration

Use the -c command line option to tell TickTock the location of your configuration file. The configuration file conf/tt.conf contains all the supported configurations, together with their default values. If you need to change a configuration to something other than its default, simply uncomment it (remove the semicolon at the beginning of the line), and update the value.

2.1 Format of the Configuration File

Each line in the configuration file specifies one configuration, with the format

<configuration.name> = <value>

Value can be numbers, boolean (true or false), or string. Please do NOT quote strings, or any other type of values, even when they contain spaces. Sometimes numbers are appended with a unit, e.g. 2s, which means 2 seconds.

2.2 List of Supported Configurations

Here's a complete list of supported configurations, with descriptions.

Configuration Name	Type	Required	Description	Default
append.log.dir	String	Optional	Directory under which append logs will reside	. (CWD)
append.log.enabled	Boolean	Optional	Enable append log	false
append.log.flush.frequency	Time	Optional	How often to flush append logs to disk	2s (2 sec)
append.log.rotation.frequency	Time	Optional	How often to rotate append logs (create new)	1h (2 hour)
append.log.retention.count	Integer	Optional	How many append logs to keep	2
http.listener.count	Integer	Optional	#threads to monitor incoming HTTP requests (connection & requests)	2
http.request.format	String	Optional	Either plain or json	plain
http.responders.per.listener	Integer	Optional	#threads to process requests, per listener	2
http.server.port	Integer	Optional	HTTP Server port number	6182
log.file	File Path	Optional	Where to write logs to	/var/log/ticktock.log
log.level	Enum	Optional	TRACE|DEBUG|INFO|WARN|ERROR|FATAL	INFO
log.retention.count	Integer	Optional	How many old log files to keep	10
log.rotation.size	Integer	Optional	When log file reached this size, create new	10mb
query.executor.queue.size	Integer	Optional	Size of queue to store pending query requests	1024
query.executor.thread.count	Integer	Optional	#threads to process query requests	8
tcp.connection.idle.timeout	Time	Optional	Duration of inactivity before disconnect	10min
tcp.listener.count	Integer	Optional	#threads to monitor incoming TCP requests (connection & requests)	2
tcp.max.epoll.events	Integer	Optional	Max number of pending TCP requests to keep before dropping	128
tcp.buffer.size	Bytes	Optional	Needs to be big enough to hold biggest request	1mb
tcp.responders.per.listener	Integer	Optional	#threads to process TCP requests (write requests)	2
tcp.server.port	Integer	Optional	TCP Server port number	6181
tcp.socket.rcvbuf.size	Integer	Optional	Used to set socket SO_RCVBUF option	none
tcp.socket.sndbuf.size	Integer	Optional	Used to set socket SO_SNDBUF option	none
tsdb.archive.threshold	Time	Optional	Data and meta-data older than this will be purged out of memory	1w (1 week)
tsdb.compact.frequency	Time	Optional	How often to check for things to compact?	2h
tsdb.data.dir	Dir Path	Optional	Where to store TSDB data?	. (CWD)
tsdb.page.count	Integer	Optional	Number of 4KB pages in one data file. This decides size of each data file.	65536
tsdb.flush.frequency	Time	Optional	How often to flush data to disk? This determines how much data will be lost in the event of abnormal termination.	5min
tsdb.read_only.threshold	Time	Optional	Data (not meta-data) older than this will be purged out of memory	1h (1 hour)
tsdb.retention.threshold	Time	Optional	Data older than this will be permanently deleted.	none (keep forever)
tsdb.rotation.frequency	Time	Optional	How often to create a new data file?	1d (1 day)
tsdb.self_meter.enabled	Boolean	Optional	Shall we collect TickTock's own metrics?	false
tsdb.timestamp.resolution	Enum	Optional	second|millisecond	second
udp.listener.count	Integer	Optional	#threads to monitor UDP requests	2
udp.batch.size	Integer	Optional	Max #msgs in one UDP batch request	256
udp.server.enabled	Boolean	Optional	Shall we start a UDP server?	false
udp.server.port	Integer	Optional	UDP Server port number	6181

2.3 Set configuration options in TickTock docker

You can set configuration options when running a TickTock docker, e.g.,

docker run -d --name ticktock -h ticktock -p 6182:6182 -p 6181:6181 --cpus 1 -m 8GB --memory-reservation 6GB ytyou/ticktock:latest --tsdb.timestamp.resolution millisecond --http.request.format json

The command above has two additional configuration options setup. Note that they must be after "ytyou/ticktock:latest".

• --tsdb.timestamp.resolution millisecond

• Timestamp of data points in PUT and GET are in the millisecond format.

• --http.request.format json

• PUT requests must be in json format:

Now you can write a data point like this:

curl -v -XPOST 'http://localhost:6182/api/put' -d '[{"metric":"testM1","value":2.33266,"timestamp":1514779734000,"tags":{"host":"foo"}}]'

instead of plain format:

curl -v -XPOST 'http://localhost:6182/api/put' -d 'put testM1 1633412175 123 host=foo'

2.4 Performance Tuning

For maximum performance, the total number of requests processing threads should be the same as, or slightly above, the number of cores of the host. For example, if the host has 8 cores, then tcp.listener.count = 2 and tcp.responders.per.listener = 4.

If, for some reason, TickTock was terminated abnormally, you will lose the last few minutes of data that can only be restored if you have append log enabled. Exactly how much data will be lost depends on the tsdb.flush.frequency setting, which has a default value of 5 minutes. A smaller value here will mean less data will be lost in the event of abnormal termination, but at a cost. Our experiment shows that anything less than 1 minute will have a significant impact on performance.

3. Writing Data to TickTock

Use OpenTSDB's official TCollector, or collectd's write_tsdb plugin, or InfluxDB's Telegraf, or any other agent that is compatible with OpenTSDB. Remember to change the port number from 4242 to 6181 (or whatever port number you are using).

! WARNING: (At least until v0.12.1-beta)

! Please make sure your write requests are in correct formats of Opentsdb put or Influx line protocol.
! We trust clients to send requests in correct formats. 
! If not, clients should have found the problems during testing. 
! The consequence of write requests in weird formats might result TickTockDB's meta data 
! in bad states and crashing TickTockDB.

3.1 Using TCollector

TCollector is the default collector framework for OpenTsdb. To use TCollector to collect metrics, simply download it and run,

git clone https://github.com/ylin30/tcollector.git
cd tcollector
./tcollector start --host <ticktock-host> --port <ticktock-port> (default: 6181)

We suggest our forked version in https://github.com/ylin30/tcollector since the office tcollector (https://github.com/opentsdb/tcollector) has bugs, especially not supporting python3. We also add detailed usage guides in our tcollector repo.

3.2 Using Telegraf

Telegraf is a collector framework developed by Influxdb. It collects metrics in different data format (e.g., Influxdb input line protocol, CSV, Json, collectd, prometheus remote write, Grok, MQTT etc) through hundreds of input plugins and third party APIs, and sends metrics to different databases (e.g., Influxdb, Opentsdb, Graphite, Datadog, Kafka etc).

To install and start Telegraf, please follow the official instructions here.

3.2.1 using Opentsdb protocol for writes

TickTock can work with Telegraf as an output database as Opentsdb. You just need to add a plugin section outputs.opentsdb in telegraf.conf. (You'd better just copy the sample config provided in installation).

ylin30@yi-IdeaPad:~$ sudo cp /etc/telegraf/telegraf.conf.sample /etc/telegraf/telegraf.conf

3.2.1.1 HTTP protocol

Sample config is shown below. You can use most of the default settings in the sample telegraf.conf, except host and port.

ylin30@yi-IdeaPad:~$ sudo vim /etc/telegraf/telegraf.conf 
...
# Configuration for OpenTSDB server to send metrics to
[[outputs.opentsdb]]
    ## prefix for metrics keys
    prefix = "telegraf."

    ## DNS name of the OpenTSDB server
    ## Using "opentsdb.example.com" or "tcp://opentsdb.example.com" will use the
    ## telnet API. "http://opentsdb.example.com" will use the Http API.
    host = "http://127.0.0.1"

    ## Port of the OpenTSDB server
    port = 6182

**Note that all writes are in Json format. E.g., ***

{"metric":"telegraf.diskio_read_time","timestamp":1678407970,"value":4780,"tags":{"host":"yi-IdeaPad","name":"loop11"}}
,{"metric":"telegraf.diskio_reads","timestamp":1678407970,"value":43,"tags":{"host":"yi-IdeaPad","name":"loop11"}}
,{"metric":"telegraf.diskio_merged_writes","timestamp":1678407970,"value":0,"tags":{"host":"yi-IdeaPad","name":"loop11"}}
,{"metric":"telegraf.diskio_io_time","timestamp":1678407970,"value":4592,"tags":{"host":"yi-IdeaPad","name":"loop11"}}
,{"metric":"telegraf.diskio_writes","timestamp":1678407970,"value":0,"tags":{"host":"yi-IdeaPad","name":"loop11"}}

Json format is much heavier than the plain put format which is supported in Telegraf with TCP. See the section below.

3.2.1.2 TCP protocol (suggested)

Sample config is shown below. You can use most of the default settings in the sample telegraf.conf, except host and port.

ylin30@yi-IdeaPad:~$ sudo vim /etc/telegraf/telegraf.conf 
...
# Configuration for OpenTSDB server to send metrics to
[[outputs.opentsdb]]
    ## prefix for metrics keys
    prefix = "telegraf."

    ## DNS name of the OpenTSDB server
    ## Using "opentsdb.example.com" or "tcp://opentsdb.example.com" will use the
    ## telnet API. "http://opentsdb.example.com" will use the Http API.
    host = "tcp://127.0.0.1"

    ## Port of the OpenTSDB server
    port = 6181

**Note that all writes are in plain put format. E.g., ***

put telegraf.cpu_usage_system 1678408370 0.000000 cpu=cpu10 host=yi-IdeaPad
put telegraf.cpu_usage_softirq 1678408370 0.000000 cpu=cpu10 host=yi-IdeaPad
put telegraf.cpu_usage_irq 1678408370 0.000000 cpu=cpu10 host=yi-IdeaPad
put telegraf.cpu_usage_steal 1678408370 0.000000 cpu=cpu10 host=yi-IdeaPad
put telegraf.cpu_usage_guest 1678408370 0.000000 cpu=cpu10 host=yi-IdeaPad

3.2.1.3 Restart Telegraf

Don't forget to restart your telegraf service after modifying telegraf.conf.

sudo systemctl restart telegraf

You can see metrics sent by Telegraf with the prefix "telegraf." in the example config above.

[Yi-MBP ticktock (dev2)]$ curl -s 'http://127.0.0.1:6182/api/suggest?type=metrics&q=telegraf' 

["telegraf.cpu_usage_guest","telegraf.cpu_usage_guest_nice","telegraf.cpu_usage_idle",...,"telegraf.system_uptime"]

3.2.2 using InfluxDB line protocol for writes (suggested)

TickTockDB supports InfluxDB line protocol and Telegraf since v0.11.3-beta. You just need to add an output plugin in telegraf.conf. (You'd better just copy the sample config provided in installation).

ylin30@yi-IdeaPad:~$ sudo cp /etc/telegraf/telegraf.conf.sample /etc/telegraf/telegraf.conf

3.2.2.1 HTTP protocol

To let Telegraf send metrics in HTTP protocol, you need to add outputs.http plugin. You can use most of the default settings except updating the url to http://<TickTockDB>:<http port, default 6182>/api/write.

ylin30@yi-IdeaPad:~$ sudo vim /etc/telegraf/telegraf.conf 
...
# A plugin that can transmit metrics over HTTP
[[outputs.http]]
    ## URL is the address to send metrics to
    url = "http://10.0.0.2:6182/api/write"

3.2.2.2 TCP protocol

To let Telegraf send metrics in TCP protocol, you need to add outputs.socket_writer plugin. You can use most of the default settings except updating the address to tcp://<TickTockDB>:<tcp port, default 6180>.

 # # Generic socket writer capable of handling multiple socket types.
 [[outputs.socket_writer]]
 #   ## URL to connect to
     address = "tcp://10.0.0.2:6180"

3.2.2.3 Restart Telegraf

Don't forget to restart your telegraf service after modifying telegraf.conf.

sudo systemctl restart telegraf

3.2.2.4 Query examples (note the _field keyword)

You can query the metrics sent by Telegraf.

[yi-IdeaPad ~]$ curl 'http://10.0.0.2:6182/api/query?start=1m-ago&m=avg:mem\{_field=buffered\}'
[{"metric":"mem","tags":{"_field":"buffered","host":"yi-IdeaPad"},"aggregateTags":[],"dps":{"1678404420":451432448.0,"1678404430":451436544.0,"1678404440":451436544.0,"1678404450":451440640.0,"1678404460":451444736.0}}]

Please note the keyword _field which refers to a field_key in the line protocol. The syntax of the line protocol is:

<measurement>[,<tag_key>=<tag_value>[,<tag_key>=<tag_value>]] <field_key>=<field_value>[,<field_key>=<field_value>] [<timestamp>]

E.g.,

mem,host=yi-IdeaPad active=6915649536i,low_total=0i,slab=851771392i,vmalloc_chunk=0i,available=18955440128i,free=3993980928i,used=1474494464i,available_percent=91.03309480336719,cached=14902673408i,high_total=0i,huge_pages_total=0i,sreclaimable=638746624i,swap_cached=1097728i,dirty=40960i,huge_page_size=2097152i,vmalloc_used=72032256i,inactive=8697843712i,mapped=299307008i,shared=8257536i,vmalloc_total=35184372087808i,write_back=0i,committed_as=5844856832i,swap_total=2147479552i,write_back_tmp=0i,total=20822581248i,used_percent=7.081228049676236,buffered=451432448i,commit_limit=12558770176i,high_free=0i,huge_pages_free=0i,page_tables=19402752i,swap_free=2126630912i,low_free=0i,sunreclaim=213024768i 1678404420000000000

3.3 Doing it Yourself

Please refer to Usage Examples.

3.3.0 using InfluxDB line protocol for writes

TickTockDB supports InfluxDB line protocol since v0.11-beta. The syntax is:

<measurement>[,<tag_key>=<tag_value>[,<tag_key>=<tag_value>]] <field_key>=<field_value>[,<field_key>=<field_value>] [<timestamp>]

For example:

cpu,host=rpi,id=1 usr=10,sys=20,idle=70 1465839830

TickTockDB provides an endpoint at <ticktockDB>:<port>/api/write for the line protocol. For the port, you can either use 6182 for HTTP or 6180 for TCP (note that the TCP for Opentsdb plain put protocol is 6181). For example:

// Write two measurements (test.cpu and test.mem) in two lines.
ylin30@raspberrypi:~/ticktock.0.11.1/admin $ curl -XPOST 'http://localhost:6182/api/write' -d $'test.cpu,host=rpi,id=1 usr=10,sys=20,idle=70 1465839830\ntest.mem,host=rpi,id=1 available=1024,free=512,buffer=512 1465839830'

In order to query the above metrics, you can still use Opentsdb query format. However, the fields (e.g., usr, sys, idle in test.cpu) are considered a specific tag (_field) in Opentsdb query.

// Query test.cpu and _field=usr
ylin30@raspberrypi:~/ticktock.0.11.1/admin $ curl 'http://localhost:6182/api/query?start=1465839830&m=avg:test.cpu\{_field=usr\}'
[{"metric":"test.cpu","tags":{"_field":"usr","host":"rpi","id":"1"},"aggregateTags":[],"dps":{"1465839830":10.0}}]
// Query test.cpu and _field=sys
ylin30@raspberrypi:~/ticktock.0.11.1/admin $ curl 'http://localhost:6182/api/query?start=1465839830&m=avg:test.cpu\{_field=sys\}'
[{"metric":"test.cpu","tags":{"_field":"sys","host":"rpi","id":"1"},"aggregateTags":[],"dps":{"1465839830":20.0}}]

Please note the keyword {_field=usr} and {_field=sys}. It is reserved to identify different fields in the line protocol. You can try two scripts in <ticktock>/admin/ dir.

ylin30@raspberrypi:~/ticktock.0.11.1/admin $ ./write.sh
ylin30@raspberrypi:~/ticktock.0.11.1/admin $ ./query3.sh
[{"metric":"test.measurement","tags":{"_field":"field2","host":"host1","sensor":"sensor1"},"aggregateTags":[],"dps":{"1677959400":2.0}}]
[{"metric":"test.measurement2","tags":{"_field":"field4","host":"host1","sensor":"sensor2"},"aggregateTags":[],"dps":{"1677959400":4.0}}]
ylin30@raspberrypi:~/ticktock.0.11.1/admin $

3.3.1 by default, data point in Opentdb plain format

If you are interested in send data to TickTockDB yourself, the TCP Server inside TickTockDB accepts data in the following format by default (default configuration option: --http.request.format=plain):

put <metric-name> <timestamp> <value> <tag1>=<val1> <tag2>=<val2> ...

One data point per line (ended with \n), where <timestamp> is Unix timestamp (number of seconds/milliseconds since Epoch), and <value> can be either integer or floating point number. Supported floating point number formats include,

• Decimal floating point numbers, e.g. 88, +1.23, -0.45, 15e16
• Hexdecimal floating point numbers, e.g. 0x12, -0x1afp-2

Do not put quotes anywhere, even around metric names, tag names, or tag values. Metric name, tag names, and tag values all cannot contain spaces.

Metric names, tag names, and tag values all has to be ASCII encoded. TickTockDB does not support unicode, yet.

Example:

curl -v -XPOST 'http://localhost:6182/api/put' -d 'put testM1 1633412175 123 host=foo'

Note:

• The Data point will be stored in data files located in a data directory specified as tsdb.data.dir (default: /tmp) in conf/tt.conf.
• There is a data file (1633392000.1633478400.0) and a metadata file (1633392000.1633478400.meta) in /tmp for the date of the written data point at 1633412175. Each file has a prefix as ., where:

• fromSecond: the first second of a day
• toSecond: the first second of the next day

• There may be more than 1 data file for a day if there are more data points than a data file size. They will be postfixed as ".1", ".2", etc. The data file size is determined by tsdb.page.count (each page 4kB) in conf/tt.conf. There will be only meta file for a day no matter how may data files in a day.
• You can specify the granularity you want to start a new meta file as tsdb.rotation.frequency (default: 1d).

3.3.2 data point in json format

If you want to use json format in PUT requests as Opentsdb APIs, you can run TickTock with configuration --http.request.format=json. Please refer to section 2.3 above.

4. Querying Data from TickTock

We try to use the same query syntax as that of OpenTSDB as much as possible, although we do not support all the syntax that OpenTSDB support. Even so, as reference, you can check out OpenTSDB's documentation.

Currently we do not support functions and expressions in query. We hope to add those supports in the future.

4.1 Querying TickTock on Command Line

Example:

curl -v 'http://localhost:6182/api/query?start=1600000000&m=avg:testM1'

5. Visualizing Data Using Grafana

To use Grafana to consume TickTock data, treat TickTock as if it is OpenTSDB. When creating data source in Grafana, select the type OpenTSDB, and specify the URL to point to the HTTP Server port (default 6182). An example URL would be: http://localhost:6182. Note that we are currently compatible with OpenTSDB 2.2, as far as Grafana is concerned.

Once the data source is created, you can proceed to create your favorite dashboard using the data source.

If you are a Docker user, you can check out our demo container (refer to our front page for details).