Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

First commit

  • Loading branch information...
commit 7f48742370479abdd7262cd65c589d2244280473 1 parent e3761af
Jose Miguel Pérez authored
2  LICENSE
View
@@ -1,4 +1,4 @@
-Copyright (c) 2009 Jose Miguel Perez
+Copyright (c) 2010 Jose Miguel Perez
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
192 README.rdoc
View
@@ -1,17 +1,185 @@
= trackoid
-Description goes here.
+Trackoid is an analytics tracking system made specifically for MongoDB using Mongoid as ORM.
-== Note on Patches/Pull Requests
-
-* Fork the project.
-* Make your feature addition or bug fix.
-* Add tests for it. This is important so I don't break it in a
- future version unintentionally.
-* Commit, do not mess with rakefile, version, or history.
- (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
-* Send me a pull request. Bonus points for topic branches.
+= Requirements
-== Copyright
+Trackoid requires Mongoid, which obviously in turn requires MongoDB. Although you can only use Trackoid in Rails projects using Mongoid, it can easily be ported to MongoMapper or other ORM. You can also port it to work directly using MongoDB.
-Copyright (c) 2010 Jose Miguel Perez. See LICENSE for details.
+Please feel free to fork and port to other libraries. However, Trackoid really requires MongoDB since it is build from scratch to take advantage of several MongoDB features (please let me know if you dare enough to port Trackoid into CouchDB or similar, I will be glad to know).
+
+= Using Trackoid to track analytics information for models
+
+Given the most obvious use for Trackoid, consider this example:
+
+ Class WebPage
+ include Mongoid::Document
+ include Mongoid::Tracking
+
+ ...
+
+ track :visits
+ end
+
+This class models a web page, and by using `track :visits` we add a `visits` field to track... well... visits. :-) Later, in out controller we can do:
+
+ def view
+ @page = WebPage.find(params[:webpage_id])
+
+ @page.visits.inc # Increment a visit to this page
+ end
+
+That is, dead simple. Later in our views we can use the `visits` field to show the visit information to the users:
+
+ <h1><%= @page.visits.today %> visits to this page today</h1>
+ <p>The page had <%= @page.visits.yesterday %> visits yesterday</p>
+
+Of course, you can also show visits in a time range:
+
+ <h1>Visits on last 7 days</h1>
+ <ul>
+ <% @page.visits.last_days(7).reverse.each_with_index do |i,d| %>
+ <li><%= (DateTime.now - i).to_s %> : <%= d %></li>
+ <% end %>
+ </ul>
+
+== Not only visits...
+
+Of course, you can use Trackoid to track all actions who require numeric analytics in a date frame.
+
+=== Prevent login to a control panel with a maximum login attemps
+
+You can track invalid logins so you can prevent login for a user when certain invalid login had been made. Imagine your login controller:
+
+ # User model
+ class User
+ include Mongoid::Document
+ include Mongoid::Tracking
+
+ track :failed_logins
+ end
+
+ # User controller
+ def login
+ user = User.find(params[:email])
+
+ # Stop login if failed attemps > 3
+ redirect(root_path) if user.failed_logins.today > 3
+
+ # Continue with the normal login steps
+ if user.authenticate(params[:password])
+ redirect_back_or_default(root_path)
+ else
+ user.failed_logins.inc
+ end
+ end
+
+Note that additionally you have the full failed login history for free. :-)
+
+ # All failed login attemps, ever.
+ @user.failed_logins.sum
+
+ # Failed logins this month.
+ @user.failed_logins.this_month
+
+
+=== Automatically saving a history of document changes
+
+You can combine Trackoid with the power of callbacks to automatically track certain operations, for example modification of a document. This way you have a history of document changes.
+
+ class User
+ include Mongoid::Document
+ include Mongoid::Tracking
+
+ field :name
+ track :changes
+
+ after_update :track_changes
+
+ protected
+ def track_changes
+ self.changes.inc
+ end
+ end
+
+
+=== Track temperature history for a nuclear plant
+
+Imagine you need a web service to track the temperature of all rooms of a nuclear plant. Now you have a simple method to do this:
+
+ # Room temperature
+ class Room
+ include Mongoid::Document
+ include Mongoid::Tracking
+
+ track :temperature
+
+ end
+
+
+ # Temperature controller
+ def set_temperature_for_room
+ @room = Room.find(params[:room_number])
+
+ @room.temperature.set(current_temperature)
+ end
+
+So, you don't need only to increment or decrement a value, yuo can also set an specific value. Now it's easy to know the maximum temperature of the last 30 days for a room:
+
+ @room.temperature.last_days(30).max
+
+
+= How does it works?
+
+Trakoid works by embedding date tracking information into models. The date tracking information is limited by a granularity of days for now. As the project evolves and we test performance, my idea is to add finer granularity of hours and perhaps, minutes.
+
+== Scalability and performance
+
+Trackoid is made from the ground up to take advantage of the great scalability features of MongoDB. Trackoid uses "upsert" operations, bypassing Mongoid controllers so that it can be used in a distributed system without data loses. This is perfect for a cloud application.
+
+The problem with a distributed system for tracking analytical information is the atomicity of operations. Imagine you must increment visits information from several servers at the same time and how you would do it. With an SQL model, this is somewhat easy because the tradittional approaches for doing this only require INSERT or UPDATE operations that are atomic by nature. But for a Document Oriented Database like MongoDB you need some kind of special operations. MongoDB uses "upsert" commands, which stands for "update or insert". That is, modify this and create if not exists.
+
+The problem with Mongoid, and with all other ORM for that matter, is that they are not made with those operations in mind. If you store an Array or Hash into a Mongoid document, you read or save it as a whole, you can not increment or store only a value without reading/writting the full Array.
+
+Trackoid issues "upsert" commands directly to the MongoDB driver, with the following structure:
+
+
+ collection.update( {_id:ObjectID}, {$inc: {visits.2010.05.30: 1} }, true )
+
+This way, the collection can receive multiple incremental operations without requiring additional logic for locking or something. The only drawback is that you will not have realtime data in your model. For example:
+
+ v = @page.visits.today # v is now "5" if there was 5 visits today
+ @page.visits.inc # Increment visits today
+ @page.visits.today == v+1 # Visits is now incremented in our local copy
+ # of the object, but we need to reload for it
+ # to reflect the realtime visits to the page
+ # since there could be another processes
+ # updating visits
+
+In practice, we don't need visits information so fine grained, but it's good to take this into account.
+
+== Embedding tracking information into models
+
+Tracking analytics data in SQL databases was historicaly saved into her own table, perhaps called `site_visits` with a relation to the sites table and each row saving an integer for each day.
+
+ Table "site_visits"
+
+ SiteID Date Visits
+ ------ ---------- ------
+ 1234 2010-05-01 34
+ 1234 2010-05-02 25
+ 1234 2010-05-03 45
+
+With this schema, it's easy to get visits for a website using single SQL statements. However, for complex queries this can be easily become cumbersome. Also this doesn't work so well for systems using a generic SQL DSL like ActiveRecord since for really taking advantage of some queries you need to use SQL language directly, one option that isn't neither really interesting nor available.
+
+Trackoid uses an embedding approach to tackle this. For the above examples, Trackoid would embedd a ruby Hash into the Site model. This means the tracking information is already saved "inside" the Site, and we don't have to reach the database for any date querying! Moreover, since the data retrieved with the accessor methods like "last_days", "this_month" and the like, are already arrays, we could use Array methods like sum, count, max, min, etc...
+
+== Memory implications
+
+Since storing all tracking information with the model implies we add additional information that can grow, and grow, and grow... You can be wondering yourself if this is a good idea. Yes, it's is, or at least I think so. Let me convice you...
+
+MongoDB stores information in BSON format as a binary representation of a JSON structure. So, BSON stores integers like integers, not like string representations of ASCII characters. This is important to calculate the space used for analytic information.
+
+A year full of statistical data takes only 2.8Kb, if you store integers. If your statistical data includes floats, a year full of information takes 4.3Kb. I said "a year full of data" because Trackoid does not store information for days without data.
+
+For comparison, this README is already 8.5Kb in size.
2  VERSION
View
@@ -1 +1 @@
-0.0.0
+0.1.0
5 lib/trackoid.rb
View
@@ -0,0 +1,5 @@
+require 'rubygems'
+
+gem "mongoid", ">= 1.9.0"
+
+require 'trackoid/tracking'
28 lib/trackoid/tracker.rb
View
@@ -13,9 +13,9 @@ def add(how_much = 1, date = DateTime.now)
raise "Can not update a recently created object" if @owner.new_record?
update_data(data_for(date) + how_much, date)
-
- # tc.collection.update( tc._selector, { "$inc" => {"stats.2010.1.1" => 5} }, :upsert => false )
- { (how_much > 0 ? "$inc" : "$dec") => update_hash(date, how_much.abs) }
+ @owner.collection.update( @owner._selector,
+ { (how_much > 0 ? "$inc" : "$dec") => update_hash(how_much.abs, date) },
+ :upsert => true)
end
def inc(date = DateTime.now)
@@ -30,28 +30,34 @@ def set(how_much, date = DateTime.now)
raise "Can not update a recently created object" if @owner.new_record?
update_data(how_much, date)
-
- { "$set" => update_hash(date, how_much) }
+ @owner.collection.update( @owner._selector,
+ { "$set" => update_hash(how_much, date) },
+ :upsert => true)
end
# Access methods
def today
- [data_for(Date.today)]
+ data_for(Date.today)
end
def yesterday
- [data_for(Date.today - 1)]
+ data_for(Date.today - 1)
end
def last_days(how_much = 7)
- return today unless how_much > 0
+ return [today] unless how_much > 0
- date = DateTime.now
- values = []
+ date, values = DateTime.now, []
(date - how_much.abs + 1).step(date) {|d| values << data_for(d) }
values
end
+ def on(date)
+ date = DateTime.parse(date) if date.is_a?(String)
+ return date.collect {|d| data_for(d)} if date.is_a?(Range)
+ data_for(date)
+ end
+
# Private methods
private
def data_for(date)
@@ -82,7 +88,7 @@ def year_literal(d); "#{d.year}"; end
def month_literal(d); "#{d.year}.#{d.month}"; end
def date_literal(d); "#{d.year}.#{d.month}.#{d.day}"; end
- def update_hash(date, num)
+ def update_hash(num, date)
{
"#{@for}.#{date_literal(date)}" => num
}
4 lib/trackoid/tracking.rb
View
@@ -23,8 +23,8 @@ def track(name)
name_sym = "#{name}_data".to_sym
field name_sym, :type => Hash, :default => {}
- # Shoul we index this field?
- index name_sym
+ # Shoul we make an index for this field?
+ # index name_sym
define_method("#{name}") do
Tracker.new(self, name_sym)
1  spec/spec.opts
View
@@ -1 +1,2 @@
--color
+--format nested
19 spec/spec_helper.rb
View
@@ -1,9 +1,26 @@
$LOAD_PATH.unshift(File.dirname(__FILE__))
$LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
+require 'rubygems'
+
+gem 'mocha', '>= 0.9.8'
+
+require 'mocha'
+require 'mongoid'
require 'trackoid'
require 'spec'
require 'spec/autorun'
+Mongoid.configure do |config|
+ name = "trackoid_test"
+ host = "localhost"
+ port = "27017"
+ # config.master = Mongo::Connection.new(host, port, :logger => Logger.new(STDOUT)).db(name)
+ config.master = Mongo::Connection.new.db(name)
+end
+
Spec::Runner.configure do |config|
-
+ config.mock_with :mocha
+ config.before :suite do
+ Mongoid.master.collections.each(&:drop)
+ end
end
170 spec/trackoid_spec.rb
View
@@ -1,7 +1,171 @@
require File.expand_path(File.dirname(__FILE__) + '/spec_helper')
-describe "Trackoid" do
- it "fails" do
- fail "hey buddy, you should probably rename this file and start specing for real"
+class Test
+ include Mongoid::Document
+ include Mongoid::Tracking
+
+ field :name # Dummy field
+ track :visits
+end
+
+describe Mongoid::Tracking do
+
+ it "should raise error when used in a class not of class Mongoid::Document" do
+ lambda {
+ class NotMongoidClass
+ include Mongoid::Tracking
+ end
+ }.should raise_error
+ end
+
+ it "should not not raise when used in a class of class Mongoid::Document" do
+ lambda {
+ class MongoidedDocument
+ include Mongoid::Document
+ include Mongoid::Tracking
+ end
+ }.should_not raise_error
+ end
+
+ describe "when creating a new field with stats" do
+
+ before(:all) do
+ @mock = Test.new
+ end
+
+ it "should deny access to the underlying mongoid field" do
+ lambda { @mock.visits_data }.should raise_error NoMethodError
+ lambda { @mock.visits_data = {} }.should raise_error NoMethodError
+ end
+
+ it "should create a method for accesing the stats" do
+ @mock.respond_to?(:visits).should == true
+ end
+
+ it "should create a method for accesing the stats of the proper class" do
+ @mock.visits.class.should == Mongoid::Tracking::Tracker
+ end
+
+ it "should not update stats when new record" do
+ lambda { @mock.inc }.should raise_error
+ end
+
+ it "shold create an empty hash as the internal representation" do
+ @mock.visits.send(:_original_hash).should == {}
+ end
+
+ it "should give 0 for today stats" do
+ @mock.visits.today.should == 0
+ end
+
+ it "should give 0 for last 7 days stats" do
+ @mock.visits.last_days.should == [0, 0, 0, 0, 0, 0, 0]
+ end
+
+ it "should give today stats for last 0 days stats" do
+ @mock.visits.last_days(0).should == [@mock.visits.today]
+ end
+
+ end
+
+ describe "when using a model in the database" do
+
+ before(:all) do
+ Test.delete_all
+ Test.create(:name => "test")
+ @object_id = Test.first.id
+ end
+
+ before do
+ @mock = Test.find(@object_id)
+ end
+
+ it "should increment visits stats for today" do
+ @mock.visits.inc
+ @mock.visits.today.should == 1
+ end
+
+ it "should increment another visits stats for today for a total of 2" do
+ @mock.visits.inc
+ @mock.visits.today.should == 2
+ end
+
+ it "should also work for yesterday" do
+ @mock.visits.inc(DateTime.now - 1)
+ @mock.visits.yesterday.should == 1
+ end
+
+ it "should also work for yesterday if adding another visit (for a total of 2)" do
+ @mock.visits.inc(DateTime.now - 1)
+ @mock.visits.yesterday.should == 2
+ end
+
+ it "then, the visits of today + yesterday must be the same" do
+ @mock.visits.last_days(2).should == [2, 2]
+ end
+
+ it "should have 4 visits for this test" do
+ @mock.visits.last_days(2).sum.should == 4
+ end
+
+ it "should correctly handle the last 7 days" do
+ @mock.visits.last_days.should == [0, 0, 0, 0, 0, 2, 2]
+ end
+
+ end
+
+ context "testing accessor operations without reloading models" do
+
+ before(:all) do
+ Test.delete_all
+ Test.create(:name => "test")
+ @object_id = Test.first.id
+ end
+
+ before do
+ @mock = Test.find(@object_id)
+ end
+
+ it "'set' operator must work" do
+ @mock.visits.set(5)
+ @mock.visits.today.should == 5
+ Test.find(@object_id).visits.today.should == 5
+ end
+
+ it "'set' operator must work on arbitrary days" do
+ @mock.visits.set(5, Date.parse("2010-05-01"))
+ @mock.visits.on(Date.parse("2010-05-01")).should == 5
+ Test.find(@object_id).visits.on(Date.parse("2010-05-01")).should == 5
+ end
+
+ it "'add' operator must work" do
+ @mock.visits.add(5)
+ @mock.visits.today.should == 10 # Remember 5 set on previous test
+ Test.find(@object_id).visits.today.should == 10
+ end
+
+ it "'add' operator must work on arbitrary days" do
+ @mock.visits.add(5, Date.parse("2010-05-01"))
+ @mock.visits.on(Date.parse("2010-05-01")).should == 10
+ Test.find(@object_id).visits.on(Date.parse("2010-05-01")).should == 10
+ end
+
+ it "on() accessor must work on dates as String" do
+ # We have data for today as previous tests populated the visits field
+ @mock.visits.on("2010-05-01").should == 10
+ end
+
+ it "on() accessor must work on dates as Date ancestors" do
+ # We have data for today as previous tests populated the visits field
+ @mock.visits.on(Date.parse("2010-05-01")).should == 10
+ end
+
+ it "on() accessor must work on dates as Ranges" do
+ # We have data for today as previous tests populated the visits field
+ @mock.visits.on(Date.parse("2010-04-30")..Date.parse("2010-05-02")).should == [0, 10, 0]
+ end
+
+
end
+
end
56 trackoid.gemspec
View
@@ -0,0 +1,56 @@
+# Generated by jeweler
+# DO NOT EDIT THIS FILE DIRECTLY
+# Instead, edit Jeweler::Tasks in Rakefile, and run the gemspec command
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+ s.name = %q{trackoid}
+ s.version = "0.1.0"
+
+ s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+ s.authors = ["Jose Miguel Perez"]
+ s.date = %q{2010-05-30}
+ s.description = %q{Trackoid is an easy but powerful analytics tracker using MongoDB and Mongoid}
+ s.email = %q{josemiguel@perezruiz.com}
+ s.extra_rdoc_files = [
+ "LICENSE",
+ "README.rdoc"
+ ]
+ s.files = [
+ ".document",
+ ".gitignore",
+ "LICENSE",
+ "README.rdoc",
+ "Rakefile",
+ "VERSION",
+ "lib/trackoid.rb",
+ "lib/trackoid/tracker.rb",
+ "lib/trackoid/tracking.rb",
+ "spec/spec.opts",
+ "spec/spec_helper.rb",
+ "spec/trackoid_spec.rb"
+ ]
+ s.homepage = %q{http://github.com/twoixter/trackoid}
+ s.rdoc_options = ["--charset=UTF-8"]
+ s.require_paths = ["lib"]
+ s.rubygems_version = %q{1.3.7}
+ s.summary = %q{Trackoid is an easy but powerful scalable analytics tracker using MongoDB and Mongoid}
+ s.test_files = [
+ "spec/spec_helper.rb",
+ "spec/trackoid_spec.rb"
+ ]
+
+ if s.respond_to? :specification_version then
+ current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+ s.specification_version = 3
+
+ if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
+ s.add_development_dependency(%q<rspec>, [">= 1.2.9"])
+ else
+ s.add_dependency(%q<rspec>, [">= 1.2.9"])
+ end
+ else
+ s.add_dependency(%q<rspec>, [">= 1.2.9"])
+ end
+end
+
Please sign in to comment.
Something went wrong with that request. Please try again.