Go Makefile
Switch branches/tags
Nothing to show
Clone or download
Latest commit 18ed4a1 Jul 25, 2018
Permalink
Failed to load latest commit information.
aggregator Another transitional step to being internally tag-based Jan 10, 2017
blaster Lots of performance bugs fixed, improved blaster Feb 22, 2017
cluster Fix a number of issues with graceful restart. Add a delay when proces… May 10, 2017
cmd/whisper_import Remove the window argument from FetchDataSources, it was stupid. We n… Jul 4, 2017
daemon More vet issues Jul 25, 2018
dsl Implement timeStack Jul 6, 2017
etc Make query LRU size configurable with query-cache-size Jun 30, 2017
graceful Fix a number of issues with graceful restart. Add a delay when proces… May 10, 2017
http Oh regular expressions... Jul 6, 2017
misc Add minute, minutes, week to parse duration. Allow curly braces as pa… Jul 5, 2017
receiver Fix a vet problem and a possible lock by value issue Jul 25, 2018
rrd Introduce Heartbeat 0, which (kind of) mimics Whisper-like behavior Jun 28, 2017
serde 1.10.3 insist on there being a test Jul 25, 2018
series Implement consolidateBy and summarize, fix a groupBy bug in rra_series Jul 5, 2017
statsd Another transitional step to being internally tag-based Jan 10, 2017
.gitignore Initial commit Sep 16, 2015
.travis.yml Stop testing 1.5 Jul 25, 2018
FAQ.md Introduce "watched ds" which are memory-resident (in addition to db) Jun 22, 2017
LICENSE Initial commit Sep 16, 2015
Makefile Improve version reporting and fix up the README Mar 22, 2017
README.md small typo May 1, 2018
main.go Fix a number of issues with graceful restart. Add a delay when proces… May 10, 2017
main_test.go daemon.go refactor to make it testable and a basic skeleton Init() test Nov 2, 2016
version.go Set version to 0.10.0b Mar 22, 2017

README.md

Build Status

Tgres is a program comprised of several packages which together can be used to receive, store and present time-series data using a relational database as persistent storage (currently only PostgreSQL).

You can currently use the standalone Tgres daemon as Graphite-like API and Statsd replacement all-in-one, or you can use the Tgres packages to incorporate time series collection and reporting functionality into your application.

See GoDoc for package details.

Whether you use standalone Tgres or as a package, the time series data will appear in your database in a compact and efficient format (by default as a view called tv), while at the same time simple to process using any other tool, language, or framework because it is just a table (or a view, rather). For a more detailed description of how Tgres stores data see this article

Current Status

Feb 7 2018: This project is not actively maintained. You may find quite a bit of time-series wisdom here, but there are probably still a lot of bugs.

Jul 5 2017: See this status update

Jun 15 2017: Many big changes since March, most notably data point versioning and instoduction of ds_state and rra_state tables which contain frequently changed attributes as arrays, similar to the way data points are stored eliminating the need to update ds and rra tables, these are now essentially immutable. Ability to delete series with NOTIFY to Tgres to purge it from the cache.

Mar 22 2017: Version 0.10.0b was tagged. This is our first beta (which is more stable than alpha). Please try it out, and take a minute to open an issue or even a PR if you see/fix any problems. Your feedback is most appreciated!

Feb 2017 Note: A major change in the database structure has been made, Tgres now uses the "write optimized" / "vertical" storage. This change affected most of the internal code, and as far overall status, it set us back a bit, all tests are currently broken, but on the bright side, write performance is amazing now.

Phase 1 or proof-of-concept for the project is the ability to (mostly) act as a drop-in replacement for Graphite (except for chart generation) and Statsd. Currently Tgres supports nearly all of Graphite functions.

As of Aug 2016 Tgres is feature-complete for phase 1, which means that the development will focus on tests, documentation and stability for a while.

Tgres is not ready for production use, but is definitely stable enough for tinkering for those interested.

Getting Started

You need a newer Go (1.7+) and PostgreSQL 9.5 or later. To get the daemon compiled all you need is:

$ go get github.com/tgres/tgres

Now you should have a tgres binary in $GOPATH/bin.

There is also a Makefile which lets you build Tgres with make which will use a slightly more elaborate command and the resulting tgres binary will be able to report its build time and git revision, but otherwise it's the same.

Look in $GOPATH/src/github.com/tgres/tgres/etc for a sample config file. Make a copy of this file and edit it, at the very least check the db-connect-string setting. Also check log-file directory, it must be writable.

The user of the PostgreSQL database needs CREATE TABLE permissions. On first run tgres will create three tables (ds, rra and ts) and two views (tv and tvd).

Tgres is invoked like this:

$ $GOPATH/bin/tgres -c /path/to/config

For Developers

There is nothing specific you need to know. If you'd like to submit a bug fix, or for anything else - use Github.

Migrating Graphite Data

Included in cmd/whisper_import is a program that can copy whisper data into Tgres, its command-line arguments are self-explanatory. You should be able to start sending data to Tgres and then migrate your Graphite data retroactively by running whisper_import to avoid gaps in data. It's probably a good idea to test a small subset of series first, migrations can be time consuming and resource-intensive.