Permalink
Browse files

Merge branch 'enterprise'

  • Loading branch information...
2 parents 96c7ec2 + 44e863a commit 0de75408a4ac5208eb4c8f6dee9df03d031969d4 @paulasmuth committed Mar 1, 2013
Showing with 1,638 additions and 96 deletions.
  1. +27 −69 README.md
  2. +23 −8 fnordmetric-doc/config.yml
  3. +0 −1 fnordmetric-doc/src/classic_event_handlers.md
  4. +17 −0 fnordmetric-doc/src/enterprise_deployment.md
  5. +4 −0 fnordmetric-doc/src/enterprise_examples.md
  6. +85 −0 fnordmetric-doc/src/enterprise_http_api.md
  7. +51 −0 fnordmetric-doc/src/enterprise_index.md
  8. +92 −0 fnordmetric-doc/src/enterprise_tcp_udp_api.md
  9. +4 −0 fnordmetric-doc/src/enterprise_usage.md
  10. +21 −16 fnordmetric-doc/src/index.md
  11. +4 −0 fnordmetric-doc/web/assets/documentation.css
  12. +3 −2 fnordmetric-doc/web/documentation.erb
  13. +1 −0 fnordmetric-enterprise/.gitignore
  14. +4 −0 fnordmetric-enterprise/TODO.md
  15. +34 −0 fnordmetric-enterprise/build.sbt
  16. +3 −0 fnordmetric-enterprise/project/plugins.sbt
  17. +36 −0 fnordmetric-enterprise/src/AbstractBucket.scala
  18. +14 −0 fnordmetric-enterprise/src/AbstractInstruction.scala
  19. +137 −0 fnordmetric-enterprise/src/Benchmark.scala
  20. +23 −0 fnordmetric-enterprise/src/BucketFactory.scala
  21. +14 −0 fnordmetric-enterprise/src/ErrorInstruction.scala
  22. +188 −0 fnordmetric-enterprise/src/FnordMetric.scala
  23. +39 −0 fnordmetric-enterprise/src/HTTPServer.scala
  24. +39 −0 fnordmetric-enterprise/src/InstructionFactory.scala
  25. +31 −0 fnordmetric-enterprise/src/MeanBucket.scala
  26. +211 −0 fnordmetric-enterprise/src/Metric.scala
  27. +33 −0 fnordmetric-enterprise/src/MetricFactory.scala
  28. +75 −0 fnordmetric-enterprise/src/RingBuffer.scala
  29. +17 −0 fnordmetric-enterprise/src/SampleInstruction.scala
  30. +36 −0 fnordmetric-enterprise/src/Scheduler.scala
  31. +15 −0 fnordmetric-enterprise/src/StorageAdapter.scala
  32. +22 −0 fnordmetric-enterprise/src/SumBucket.scala
  33. +106 −0 fnordmetric-enterprise/src/SwapFile.scala
  34. +27 −0 fnordmetric-enterprise/src/TCPHandler.scala
  35. +51 −0 fnordmetric-enterprise/src/TCPServer.scala
  36. +49 −0 fnordmetric-enterprise/src/UDPServer.scala
  37. +22 −0 fnordmetric-enterprise/src/ValueAtInstruction.scala
  38. +31 −0 fnordmetric-enterprise/src/ValuesInInstruction.scala
  39. +31 −0 fnordmetric-enterprise/src/WebSocket.scala
  40. +18 −0 fnordmetric-enterprise/src/WebSocketHandler.scala
View
@@ -1,9 +1,8 @@
FnordMetric
===========
-FnordMetric is a highly configurable (and pretty fast) realtime app/event
-tracking thing based on [ruby eventmachine](http://rubyeventmachine.com/) and [redis](http://redis.io/). You define your own
-plotting and counting functions as ruby blocks! [See it in action! (RailsCasts)](http://railscasts.com/episodes/378-fnordmetric)
+FnordMetric is a framework for collecting and visualizing timeseries data. It enables
+you to build beautiful real-time analytics dashboards within minutes.
Documentation: [fnordmetric.io](http://fnordmetric.io/)
@@ -12,85 +11,43 @@ Documentation: [fnordmetric.io](http://fnordmetric.io/)
[ ![Screenshot](https://github.com/paulasmuth/fnordmetric/raw/v1.0-alpha/doc/preview3.png) ](http://github.com/paulasmuth/fnordmetric)
+### FnordMetric Enterprise
-Getting Started
----------------
+FnordMetric Enterprise is a JVM-based timeseries database. It's a key-value store
+(much like redis or memcached) where each key holds a "measurement". There are
+different measurement methods like sum, mean, min/max, 90th percentile, etc. You
+continuously add data to these keys/measurements which is aggregated and periodically
+persisted.
-FnordMetric is based on ruby eventmachine and needs to run in a seperate ruby process.
-The preferred way to start it is to create a ruby file (where you put your DSL statements)
-and execute it (more about that in [the documentation](http://fnordmetric.io/documentation))
+FnordMetric Enterprise features disk persistence, a HTTP, TCP and UDP API, native
+WebSockets support, CSV/JSON Export and a turnkey-ready HTML5 visualization solution
+(FnordMetric UI). FnordMetric Enterprise can be used as a drop-in replacement for
+StatsD+Graphite (it is a lot faster, see Benchmarks).
-Save this to `my_fnordmetric.rb`
+[Getting Started with FnordMetric Enterprise](http://fnordmetric.io/documentation/enterprise_index)
-```ruby
-require "fnordmetric"
-FnordMetric.namespace :myapp do
+### FnordMetric UI
- # render a timeseries graph
- widget 'Sales',
- :title => "Sales per Minute",
- :gauges => [:sales_per_minute],
- :type => :timeline,
- :width => 100,
- :autoupdate => 1
+FnordMetric UI is a HTML5 API that lets you plug realtime data and charts into any webpage
+without writing a single line of code. It gives you maximum flexiblitiy as you have full
+control over layout and style with HTML and CSS.
-end
+FnordMetric UI uses WebSockets to communicate with a backend server. There are two backend
+implementations: FnordMetric Classic (ruby + redis) and FnordMetric Enterprise (JVM). You can use
+FnordMetric UI as a white label solution to power your custom realtime analytics apps.
-FnordMetric.standalone
-```
+[Getting Started with FnordMetric UI](http://fnordmetric.io/documentation/ui_index)
-In this case we created one timeseries chart on the dashboard "Sales" that will display
-the number of sales_per_minute and auto-refresh every second.
-You should now be able to start the dashboard on http://localhost:4242/ (default) by running:
+### FnordMetric Classic
- $ ruby my_fnordmetric.rb
+FnordMetric Classic is powered by ruby and redis. It offers a ruby DSL for processing data
+streams and building beautiful web dashboards from a collection of turnkey-ready UI widgets.
+You define your own plotting and aggregation methods as ruby blocks. [See it in action! (RailsCasts)](http://railscasts.com/episodes/378-fnordmetric)
+[Getting Started with FnordMetric Classic](http://fnordmetric.io/documentation/classic_index)
-Now we can start sending data to FnordMetric. The canonical way to submit data is the HTTP API.
-This will report a single sale:
-
- curl -X POST -d '{ "_type": "_incr", "value": 1, "gauge": "sales_per_minute" }' http://localhost:4242/events
-
-There are various other ways to submit events to FnordMetric (more information in [the documentation](http://fnordmetric.io/documentation)).
-
-
-### HTML5 API
-
-FnordMetric offers a HTML5 / JavaScript API (called _FnordMetric UI_) that allows
-you to plug real-time data and charts into any website without having to write code.
-This is achieved by including a JavaScript library and using data-* attributes on
-html elements to declare the widgets.
-
-```html
-<link href='http://localhost:4242/fnordmetric-ui.css' type='text/css' rel='stylesheet' />
-<script src='http://localhost:4242/fnordmetric-ui.js' type='text/javascript'></script>
-
-<span
- data-fnordmetric="counter"
- data-gauge="sales_per_minute"
- data-autoupdate="1"
- data-unit=""
- >0</span>
-
-<script>
- FnordMetric.setup({
- "address": "localhost:4242",
- "namespace": "myapp"
- });
-</script>
-```
-
-
-Installation
-------------
-
- gem install fnordmetric
-
-or in your Gemfile:
-
- gem 'fnordmetric', '>= 1.0.0'
Documentation
@@ -99,6 +56,7 @@ Documentation
You can find the full FnordMetric Documentation at http://fnordmetric.io/
+
Contributors
------------
View
@@ -5,7 +5,7 @@ sitemap:
fnordmetric_ui:
-
- title: "Introduction"
+ title: "Getting Started"
url: "/ui_index"
-
title: "API Reference"
@@ -41,12 +41,27 @@ sitemap:
url: "/classic_hacking"
- #fnordmetric_enterprise:
- # -
- # title: "Getting Started"
- # url: "/ui_index"
- # -
- # title: "HTTP API"
- # url: "/ui_index"
+ fnordmetric_enterprise:
+ -
+ title: "Introduction"
+ url: "/enterprise_index"
+ -
+ title: "Deployment"
+ url: "/enterprise_deployment"
+ -
+ title: "HTTP API"
+ url: "/enterprise_http_api"
+ -
+ title: "TCP/UDP API"
+ url: "/enterprise_tcp_udp_api"
+ -
+ title: "Configuration"
+ url: "/enterprise_usage"
+ -
+ title: "Clients"
+ url: "/enterprise_usage"
+ -
+ title: "Examples"
+ url: "/enterprise_examples"
@@ -325,7 +325,6 @@ _Example:_
=> 1360623178
-<br />
#### Manipulating gauges
These methods allow manipulation of gauges. The time / bucket to modify
@@ -0,0 +1,17 @@
+Deployment
+----------
+
+### Deploy on Ubuntu
+
+upstart script etc
+
+
+### Deploy on EC2
+
+ready made ami
+
+
+
+### Puppet rules
+
+puppet rules
@@ -0,0 +1,4 @@
+Examples
+--------
+
+here be dragons
@@ -0,0 +1,85 @@
+HTTP API
+--------
+
+FIXPAUL: blah blah
+
+### GET /metric/:key
+
+Retrieves a metric's value(s) at a specified point in time or in a specified time interval
+
+Parameters:
+
+<table>
+ <tr>
+ <th>key <i>(mandatory)</i></th>
+ <td>
+ key of this metric (e.g. my-counter-sum-15)
+ </td>
+ </tr>
+ <tr>
+ <th>since / until</th>
+ <td>
+ when the since and until parameters are set, all values in the supplied time
+ interval are returned. values can be a timestamp or a timespec like... since/until
+ are mutually exclusive with at
+ </td>
+ </tr>
+ <tr>
+ <th>at</th>
+ <td>
+ if set, returns a single value at this point in time. content can be a timestamp
+ or a timespec like -3m... mutually exclusive with since/until
+ </td>
+ </tr>
+ <tr>
+ <th>format</th>
+ <td>
+ response format csv, json or xml (also use HTTP-Accept)
+ </td>
+ </tr>
+</table>
+<br />
+
+Examples:
+
+ >> GET /metric/total_sales_in_euro-sum-30?at=-5m&format=json
+ << HTTP/1.1 200 OK
+ << ...
+ << { "value": 23426 }
+
+ >> GET /metric/total_sales_in_euro-sum-30?since=-2min&until=now&format=json
+ << HTTP/1.1 200 OK
+ << ...
+ << { "values": { 1360804571: 4233, 1360804581: 4636, 1360804591: 3621, ... } }
+
+
+### POST /metric/:key
+
+Sample/add a value to a metric.
+
+Parameters:
+
+<table>
+ <tr>
+ <th>key <i>(mandatory)</i></th>
+ <td>
+ key of this metric (e.g. my-counter-sum-15)
+ </td>
+ </tr>
+ <tr>
+ <th>value <i>(mandatory)</i></th>
+ <td>
+ the value to add/sample to this metric
+ </td>
+ </tr>
+</table>
+<br />
+
+Examples:
+
+ >> POST /metric/total_sales_in_euro-sum-30?value=351
+ << HTTP/1.1 201 CREATED
+
+
+
+
@@ -0,0 +1,51 @@
+FnordMetric Enterprise
+----------------------
+
+FnordMetric Enterprise is a JVM-based timeseries database. It's a key-value store
+(much like redis or memcached) where each key holds a "measurement". There are
+different measurement methods like sum, mean, min/max, 90th percentile, etc. You
+continuously add data to these keys/measurements which is aggregated and periodically
+persisted.
+
+FnordMetric Enterprise features disk persistence, a HTTP, TCP and UDP API, native
+WebSockets support, CSV/JSON Export and a turnkey-ready HTML5 visualization solution
+(FnordMetric UI). FnordMetric Enterprise can be used as a drop-in replacement for
+StatsD+Graphite (it is a lot faster, see Benchmarks).
+
+
+### Semantics
+
+There are three basic operations: `add_sample`, `value_at` and `values_in` that
+add a sample to an ongoing measurement, retrieve the measurement value at a
+specified time, or retrieve all aggregated measurement values in a specified time
+interval.
+
+The measurement method and flush_interval are implicitly specified by the key;
+all keys have to be postfixed with "$method-$flush_timeout". For example if
+you want a key "response_time" to operate in average/mean mode and flush every 60
+seconds, use the key `response_time-mean-60`, for a key "total_clicks" that
+aggregates/sums a value and flushes every hour, you could use "total_clicks.sum-3600"
+
+
+### In-memory vs. disk storage
+
+FnordMetric Enterprise stores the values as 64bit double precision floats.
+
+With an example flush timeout of 10 seconds, one metric uses 0.065 MB of
+memory per day or 0.4 MB per week. The default ring buffer size is x,.
+
+That means with only 4GB of ram, you could access the last month of data of
+2500 counters/measurements with 10 second granularity all from the in-memory
+ringbuffers (without ever hitting a HDD).
+
+Requests that can not be served from memory require one sequential disk read.
+
+
+### FnordMetric Enterprise vs. StatsD
+
++ allows for flush intervals as low as one second
++ rendered in the browser, interactive
++ much much more scalable
++ highly customizable with css
++ requires only a single deployment
++ i18n (proper timezones in graphs due to in browser rendering etc)
Oops, something went wrong.

0 comments on commit 0de7540

Please sign in to comment.