Initial implementation

.gitignore

@@ -0,0 +1,17 @@
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in study.gemspec
+gemspec

LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2012 Christoph Olszowka
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

README.md

@@ -0,0 +1,105 @@
+# Study
+
+A simple utility for collecting statistics from your application, stuffing
+them into Redis and then forwarding them to munin, so it makes nice graphs for you.
+
+For example, collect and graph data on:
+
+  * How many signed in users were active on your site?
+  * How many and which background jobs have been processed? How many failed, how many succeeded?
+  * How many 404s have been triggered
+  * Whatever else you might come up with :)
+
+**Disclaimer: This is still in very early development and should not be considered usable at all**
+
+## Usage
+
+Install using `gem` (or add it to your `Gemfile`:)
+
+    gem install study
+
+Now you need to do some configuration. If you're on Rails, this will probably go into `config/initializers/study.rb`,
+otherwise make sure you have this loaded on startup of your Ruby application.
+
+Configure an app name. This will be the default category for your graphs in munin.
+
+    Study.app_name = 'Widgets'
+
+Configure some graphs. This is required since in order to avoid accidental typos on data collection
+the graph's previous existince is checked and will raise an exception if the graph has not been defined
+before.
+
+The most basic definition looks like this:
+
+    Study.define_graph 'processed_jobs'
+
+Now, in your code when you have performed some action that you'd like to be counted, for example you have
+finished processing a Resque job successfully, call:
+
+    Study :processed_jobs
+
+This will increment the counter for `processed_jobs.total` by 1.
+
+If you want your graph to have more elements than just a generic total, you can pass a second argument,
+i.e. holding the job type:
+
+  Study :processed_jobs, 'MailDeliveryJob'
+
+This will increment both `processed_jobs.MailDeliveryJob` and `processed_jobs.total` by 1.
+
+## Plugging the data into munin
+
+Now that you collect data, the remaining step is to teach munin how to fetch them.
+
+Unfortunately, automating this is still TODO, altough the basic requirements are in place.
+
+A munin-plugin is basically a shell script that either returns the keys and values or, when called with
+the argument `config`, gives the configuration.
+
+A basic plugin may look like this right now (**Note that this will be made easier soon and is just
+here to give you a basic idea how this stuff works**)
+
+    #!/usr/bin/env ruby
+
+    require 'study'
+    require 'PATH/TO/STUDY/CONFIG.rb'
+
+    munin = Study::Munin.new(Study.find_graph('widgets'))
+
+    if ARGV[0] == 'config'
+      puts munin.config
+    else
+      puts munin.data
+    end
+
+This will fetch the graph 'widgets' and either print it's config or the data currently stored in redis.
+**If the graph is not configured as `absolute` the counters will all be reset to 0 after fetching this.**
+
+## Advanced graph configuration
+
+You can pass a block to the define_graph method and configure the graph to your liking (see the API for `Study::Graph`):
+
+    Study.define_graph 'processed_jobs' do |g|
+      g.title = 'Some Fancy Title'
+      g.vlabel = 'Vertical Label used by Munin graphs'
+      g.description = 'Some lenghty description of your graph'
+      g.category = 'Override default category, which is the app_name'
+      g.absolute = true # Make graph collect absolute numbers instead of relative
+    end
+
+## Todo / Ideas
+
+  * Improve the README...
+  * Allow for more sophisticated graphs (i.e. fetching current status live, how 
+    many jobs in queue, how many records of type XYZ and so on)
+  * Other output possibilities than Munin (the basic abstraction should be there, munin is separate from everything else)
+
+
+## Contributing
+
+1. Fork it and `bundle install`
+2. Make sure you have redis running and available on localhost with the default port (so `Redis.new` gets a correct connection)
+3. Create your feature branch (`git checkout -b my-new-feature`)
+4. Commit your changes (`git commit -am 'Added some feature'`)
+5. Push to the branch (`git push origin my-new-feature`)
+6. Create new Pull Request

Rakefile

@@ -0,0 +1,11 @@
+#!/usr/bin/env rake
+require "bundler/gem_tasks"
+
+require 'rake/testtask'
+Rake::TestTask.new(:spec) do |test|
+  test.libs << 'lib' << 'spec'
+  test.test_files = FileList['spec/**/*_spec.rb']
+  test.verbose = true
+end
+
+task :default => :spec

lib/study.rb

@@ -0,0 +1,81 @@
+require "study/version"
+require 'redis'
+
+def Study(graph, scope=nil)
+  Study.find_graph(graph).increment(scope)
+end
+
+module Study
+  class UnknownGraphError < StandardError; end;
+  class ConfigurationError < StandardError; end;
+  class DuplicateGraphError < StandardError; end;
+
+  autoload :Graph, File.join(File.dirname(__FILE__), 'study/graph')
+  autoload :Munin, File.join(File.dirname(__FILE__), 'study/munin')
+
+  class << self
+    # Returns an array of all graphs
+    def graphs
+      _configured_graphs.map {|key, graph| graph }
+    end
+
+    # Allows you to retrieve a Study::Graph by the given name.
+    #
+    # Will raise Study::UnknownGraphError if the graph cannot be found.
+    def find_graph(name)
+      _configured_graphs[name.to_sym] || raise(UnknownGraphError, "No Study graph called '#{name}' is defined!")
+    end
+
+    # Returns the redis connection.
+    def redis
+      @redis ||= Redis.new
+    end
+    attr_writer :redis
+
+    # CAUTION! Will kill all stored data for the current scope ("study.APP_NAME.*")
+    def purge!
+      redis.keys(scope + "*").each do |key|
+         redis.del key
+      end
+    end
+
+    # Returns the configured application name. Used for report group names and redis key scopes.
+    #
+    # Will raise Study::ConfigurationError if not specified.
+    def app_name
+      @app_name || raise(Study::ConfigurationError, "Whoops, no app_name specified. Please set one with Study.app_name = 'widgets'")
+    end
+    attr_writer :app_name
+
+    # The scope for the redis keys that study uses.
+    def scope
+      @scope ||= "study.#{app_name}"
+    end
+
+    # Allows to define a new graph with given name, returns it after initializing. 
+    #
+    # If a block is given, the new Study::Graph instance will be yielded to it for further 
+    # configuration, i.e.:
+    #
+    # Study.define_graph :succeeded_jobs do |graph|
+    #   graph.title = 'Succeeded Jobs'
+    # end
+    # 
+    # Will raise Study::DuplicateGraphError if the name is already in use.
+    #
+    def define_graph(name)
+      raise DuplicateGraphError, "There already is a graph called '#{name}'!" if _configured_graphs[name.to_sym]
+      graph = Study::Graph.new(name)
+      _configured_graphs[name.to_sym] = graph
+      yield(graph) if block_given?
+      graph
+    end
+
+    private
+
+      # Internal storage for graphs, based on a hash for quick retrieval by name
+      def _configured_graphs
+        @configured_graphs ||= {}
+      end
+  end
+end

lib/study/graph.rb

@@ -0,0 +1,76 @@
+module Study
+  class Graph
+    attr_reader :name
+    attr_writer :title, :category, :vlabel
+    attr_accessor :description, :absolute
+
+    def initialize(name)
+      @name = name.to_s.freeze
+      @absolute = false
+    end
+
+    def title
+      @title || name
+    end
+
+    def vlabel
+      @vlabel || title
+    end
+
+    def category
+      @category || Study.app_name
+    end
+
+    def increment(scope=nil)  
+      # Always increment total. If there's no scope, this is the only value in the graph
+      Study.redis.incr make_scope_key('total')
+
+      Study.redis.incr make_scope_key(scope) if scope
+    end
+
+    def values
+      data = {}
+      keys.each do |key|
+        data[key.split('.').last] = read(key)
+      end
+      data
+    end
+
+    def keys
+      Study.redis.keys("#{graph_base_scope}\.*")
+    end
+
+    private
+
+      def read(key)
+        if absolute
+          get key
+        else
+          getset key
+        end
+      end
+
+      # Read the value and reset it to 0. Redis 2.4 will get 0 on getset when null, but earlier versions 
+      # may return nil, so enforce a proper return value
+      def getset(key)
+        Study.redis.getset(key, 0).to_i || 0
+      end
+
+      # For absolute graphs, just get the key or return 0 without resetting the value
+      def get(key)
+        Study.redis.get(key).to_i || 0
+      end
+
+      def make_scope_key(scope)
+         if scope.nil?
+           graph_base_scope
+         else
+           "#{graph_base_scope}.#{scope}"
+         end
+      end
+
+      def graph_base_scope
+        @graph_base_scope ||= Study.scope + ".#{name}"
+      end
+  end
+end

lib/study/munin.rb

@@ -0,0 +1,30 @@
+module Study
+  class Munin  
+    attr_reader :graph
+    def initialize(graph)
+      @graph = graph
+    end
+
+    def config
+      report = []
+
+      report << "graph_title #{graph.title}"
+      report << "graph_vlabel #{graph.vlabel}"
+      report << "graph_category #{graph.category}"
+      report << "graph_info #{graph.description}" if graph.description
+
+      graph.keys.each do |scope|
+        key = scope.split('.').last
+        report << "#{key}.label #{key.capitalize}"
+      end
+
+      report.join("\n")
+    end
+
+    def data
+      graph.values.map do |key, value|
+        "#{key}.value #{value}"
+      end.join("\n")
+    end
+  end
+end

lib/study/version.rb

@@ -0,0 +1,3 @@
+module Study
+  VERSION = "0.0.1"
+end