To store and retrieve time series counters, average, histograms, etc. Support for Postgres, Cassandra, others.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc
src/time_series_storage
test/time_series_storage
.gitignore
LICENSE
README.md
circle.yml
project.clj

README.md

time-series-storage

A Clojure library designed to store time-series oriented metrics.

Main idea is to provide a standard API to track and query counters, histograms and other metrics, independently of the storage used.

Support for Postgres and Cassandra will be added.

Intended to be useful as a side-companion to Datomic if you need to keep track of mutable trends using the same backend.

Concepts

####Facts

Anything you need to keep track of, count or aggregate.

Can be discrete events such as conversions or visits, or events with a magnitude, such as conversion-time.

##Dimensions

Any category you want your facts to be categorized, grouped or filtered by.

Usually named values such as country, operating-system, referral, etc.

Usage

Work in progress. should be moved to the protocol api samples

(def db-spec (or (System/getenv "DATABASE_URL")
  "postgresql://postgres:postgres@localhost:5432/timeseries"))

(require '[time-series-storage.api :as api])
(require '[time-series-storage.postgres :as p])

(import '[time_series_storage.postgres Postgres])

(def service (Postgres. db-spec))

(t/define-fact service ...)
  (api/init-schema! service)

  ;;create-counter
  (api/define-fact! service :registros :counter 15 {:name "Cantidad de registros"
                                                :filler 0
                                                :units "counter"})

  ;;create-average
  (api/define-fact! service :avg_time :average 15 {:name "Tiempo promedio"
                                               :filler 0
                                               :units "seconds"})
  ;;create-histogram
  (api/define-fact! service :time-distr :histogram 15 {:name "Histograma de tiempo"
                                                   :filler 0
                                                   :units "seconds"
                                                   :start 0
                                                   :end 1000
                                                   :step 100})

  ;;
  (api/define-dimension! service :company {:group_only true :name "Compania"})
  (api/define-dimension! service :campaign {:grouped_by [[:company]] :name "Campania"})
  (api/define-dimension! service :channel {:grouped_by [[:company :campaign]] :name "Canal"})

  (api/define-dimension! service :dependency {:name "Dependencia de Correo"})
  (api/define-dimension! service :dependency_user {:grouped_by [[:dependency]] :name "Usuario"})

  (api/new-fact! service :registros 1 {:dependency "32" :dependency_user "juanele"})
  (api/new-fact! service :registros 2 {:dependency "35" :dependency_user "pepe"})

  (api/get-timeseries service
         :registros
         :dependency
         {:dependency "32"}
         #inst "2012-01-01"
         #inst "2020-01-01")
  ;; => ({:counter 1, :dependency "32", :timestamp "20141121T134500.000Z"})

  (api/get-timeseries service
           :registros
           :dependency
           {:dependency "35"}
           #inst "2012-01-01"
           #inst "2020-01-01")
  ;; => ({:counter 2, :dependency "35", :timestamp "20141121T134500.000Z"}

  (api/get-timeseries service
           :registros
           :dependency
           {}
           #inst "2012-01-01"
           #inst "2020-01-01")
  ;; => ({:counter 1, :dependency "32", :timestamp "20141121T134500.000Z"} {:counter 2, :dependency "35", :timestamp "20141121T134500.000Z"})

  (api/new-fact! service :avg_time 15 {:company "bbva" :campaign "ventas" :channel "web"})
  (api/new-fact! service :avg_time 15 {:company "bbva" :campaign "ventas" :channel "mail"})

  (api/get-timeseries service
         :avg_time
         :campaign
         {:company "bbva" :campaign "ventas"}
         (tcoerce/from-date #inst "2012-01-01")
         (tcoerce/from-date #inst "2020-01-01"))

  (api/get-timeseries service
         :avg_time
         :channel
         {:company "bbva" :campaign "ventas" :channel "web"}
         (tcoerce/from-date #inst "2012-01-01")
         (tcoerce/from-date #inst "2020-01-01"))

License

Copyright © 2014 Guillermo Winkler

Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.