Permalink
Browse files

Remote metrics: use Web API to push/retrieve metrics on remote host.

Initial support for multi-value metrics.

* Added: Vanity.playground.collecting. You want this to be true only in
production environment. When false, disables collecting of metric and
experiment data.
* Added: Metric.last_update_at.
* Removed: Metric.created_at, derived from experiment and never used.

Fixed MongoDB flushdb to drop collections.

.rvmrc no longer setups Ruby/creates gemset.

Apparently on Linux, Passenger uses UNIX sockets, where on OS X it
uses TCP.
  • Loading branch information...
1 parent e20e9ca commit 10b1523d049040b9e45f9d851c556ad35c047493 @assaf assaf committed Jul 8, 2010
View
@@ -1,5 +1,3 @@
-export rvm_install_on_use_flag=1
-export rvm_gemset_create_on_use_flag=1
export RUBYOPT="rubygems"
export RUBYLIB="."
rvm 1.8.7@vanity
View
@@ -5,17 +5,24 @@ access to collected metrics and experiment data:
vanity upgrade
+
Connection adapters! We have a new way for managing connections which extends
to multiple adapters (not just Redis). The easiest is to use the configuration
file config/vanity.yml. For example:
- development:
+ production:
adapter: redis
- test:
- adapter: mock
- production:
+ worker:
adapter: mongodb
+Redis is carried forward and we get MongoDB adapter. Instead of using mock
+adapter or calling playground.test!, turn collection off when running outside
+production. Under Rails, collection is turned off in all environment except
+production. You can control it using:
+
+ Vanity.playground.collection = true/false.
+
+
This release switches from older vendored version of the Redis gem to the new
2.0.x gem that is now required as a separate dependency. If your own code
relies on the Redis gem, watch out for subtle incompatibilities introduced in
@@ -30,11 +37,20 @@ To switch around:
rvm 1.9.2@vanity
* Added: Adapter API, see Vanity::Adapters::AbstractAdapter and
- Vanity::Adapters::RedisAdapter.
+Vanity::Adapters::RedisAdapter.
* Added: MongoDB support.
* Added: Upgrade command.
+* Added: Metric.last_update_at.
+* Added: Vanity.playground.collecting. You want this to be true only in
+production environment. When false, disables collecting of metric and
+experiment data.
+* Added: Remote metrics. Push data to remote service.
+* Added: Partial support for multi-series metrics. Laying the ground for the
+future.
* Change: Vanity.playground.redis and redis= methods are deprecated, use
- connection and establish_connection instead.
+connection and establish_connection instead.
+* Removed: Metric.created_at, derived from experiment and never used.
+
== 1.3.0 (2010-03-01)
View
@@ -1,11 +1,12 @@
source "http://rubygems.org/"
gem "garb"
-gem "rack"
+gem "redis"
gem "redis-namespace"
group :development do
gem "jekyll"
gem "rake"
+ gem "RedCloth"
gem "yard"
end
@@ -15,7 +16,9 @@ group :test do
gem "mongo"
gem "passenger"
gem "rails", "2.3.8"
+ gem "rack", "1.1.0"
gem "shoulda"
gem "sqlite3-ruby", "1.2.5" # 1.3.0 doesn't like Ruby 1.9.1
gem "timecop"
+ gem "webmock"
end
View
@@ -1,5 +1,7 @@
require "rake/testtask"
+# -- Building stuff --
+
spec = Gem::Specification.load(File.expand_path("vanity.gemspec", File.dirname(__FILE__)))
desc "Build the Gem"
@@ -24,35 +26,7 @@ task :push=>["test:rubies", "build"] do
end
-# Run the test suit.
-
-task :default=>:test
-desc "Run all tests"
-Rake::TestTask.new do |task|
- task.test_files = FileList['test/*_test.rb']
- if Rake.application.options.trace
- #task.warning = true
- task.verbose = true
- elsif Rake.application.options.silent
- task.ruby_opts << "-W0"
- else
- task.verbose = true
- end
- task.ruby_opts << "-I."
-end
-
-# These are all the adapters we're going to test with.
-ADAPTERS = %w{redis mongodb mock}
-
-desc "Test using different back-ends"
-task "test:adapters", :adapter do |t, args|
- adapters = args.adapter ? [args.adapter] : ADAPTERS
- adapters.each do |adapter|
- puts "** Testing #{adapter} adapter"
- sh "rake test ADAPTER=#{adapter} #{'--trace' if Rake.application.options.trace}"
- end
-end
-
+# -- Testing stuff --
# Ruby versions we're testing with.
RUBIES = %w{1.8.7 1.9.1 1.9.2}
@@ -70,6 +44,7 @@ task "test:rubies", :ruby do |t, args|
rubies = args.ruby ? [args.ruby] : RUBIES
rubies.each do |ruby|
puts "** Setup #{ruby}"
+ sh "env rvm_install_on_use_flag=1 rvm_gemset_create_on_use_flag=1 rvm use #{ruby}@vanity"
sh "rvm #{ruby}@vanity rake test:setup"
puts
puts "** Test using #{ruby}"
@@ -91,9 +66,40 @@ task "test:setup" do
end
end
+# These are all the adapters we're going to test with.
+ADAPTERS = %w{redis mongodb}
+
+desc "Test using different back-ends"
+task "test:adapters", :adapter do |t, args|
+ adapters = args.adapter ? [args.adapter] : ADAPTERS
+ adapters.each do |adapter|
+ puts "** Testing #{adapter} adapter"
+ sh "rake test ADAPTER=#{adapter} #{'--trace' if Rake.application.options.trace}"
+ end
+end
+
+# Run the test suit.
+
+task :default=>:test
+desc "Run all tests"
+Rake::TestTask.new do |task|
+ task.test_files = FileList['test/**/*_test.rb']
+ if Rake.application.options.trace
+ #task.warning = true
+ task.verbose = true
+ elsif Rake.application.options.silent
+ task.ruby_opts << "-W0"
+ else
+ task.verbose = true
+ end
+ task.ruby_opts << "-I."
+end
+
task(:clobber) { rm_rf "tmp" }
+# -- Documenting stuff --
+
begin
require "yard"
YARD::Rake::YardocTask.new(:yardoc) do |task|
@@ -106,8 +112,13 @@ end
desc "Jekyll generates the main documentation (sans API)"
task(:jekyll) { sh "jekyll", "doc", "html" }
+file "html/vanity-api-#{spec.version}.zip"=>:yardoc do |t|
+ Dir.chdir "html" do
+ sh "zip vanity-api-#{spec.version}.zip -r api"
+ end
+end
desc "Create documentation in docs directory (including API)"
-task :docs=>[:jekyll, :yardoc]
+task :docs=>[:jekyll, :yardoc, "html/vanity-api-#{spec.version}.zip"]
desc "Remove temporary files and directories"
task(:clobber) { rm_rf "html" ; rm_rf ".yardoc" }
@@ -117,6 +128,8 @@ task :publish=>[:clobber, :docs] do
end
+# -- Misc --
+
task :report do
$LOAD_PATH.unshift "lib"
require "vanity"
@@ -14,10 +14,13 @@ Vanity.playground.establish_connection "redis://db.example.com"
Configuration options are:
-|_. name |_. Is all about ... |_. Default |
-| load_path | Directory containing experiment files | experiments |
-| logger | This should be obvious | default/Rails |
+|_. name |_. Is all about ... |_. Default |
+| load_path | Directory containing experiment files | experiments |
+| logger | This should be obvious | default/Rails |
+| collecting | False if you won't want data collected | true |
Database connection information is loaded from @config/vanity.yml@, based on the current environment (@RACK_ENV@ or @RAILS_ENV@). If your application has no configuration file and does not establish a connection directly, Vanity will automatically connect to the Redis instance running on @localhost@ at port 6379.
-When "running under Rails":rails.html, Vanity defaults to using the Rails logger, locates the load_path relative to Rails root and will use the @config/vanity.yml@ configuration file, if present.
+You want Vanity to collect information in production environment, but there's no point collecting data in other environments, and data collection just makes setting up the environment harder. You can turn collection on off by setting @Vanity.playground.collection@. In environments that have collection off, you don't have to specify any connection configuration.
+
+When "running under Rails":rails.html, Vanity defaults to using the Rails logger, locates the load_path relative to Rails root, uses the @config/vanity.yml@ configuration file (if present) and turns collection on only in production mode.
View
@@ -31,16 +31,12 @@ end
If you have a @config/vanity.yml@ file, Vanity will read the configuration for the current environment. For example:
<pre>
-development:
+staging:
adapter: redis
-test:
- adapter: mock
+ host: staging.internal
production:
- adapter: redis
- host: db.local
-mongodb:
adapter: mongo
- host: db.local
+ host: live.internal
database: vanity
</pre>
@@ -53,25 +49,7 @@ config.after_initialize do
end
</pre>
-
-h3(#test). Test Environment
-
-As your tests run, any metric and experiment data is stored in Redis. You might not appreciate tests filling up Redis with useless data, and there are two things you can do about it.
-
-Easiest it to tell Vanity to avoid accessing the database altogether. Just add this to @config/environments/test.rb@:
-
-<pre>
-require "vanity"
-Vanity.playground.test!
-</pre>
-
-A bit more convoluted (but you'll learn a couple of things along the way) is to flush the database after each test, or at least once before loading the test suite. Of course, flushing the database will also clear any data you might need in development mode (most people develop and test on the same machine), so you'll want the test environment to use a different database. Like this:
-
-<pre>
-require "vanity"
-Vanity.playground.establish_connection "redis://localhost/15" # database 15
-Vanity.playground.connection.flushdb # Do this before running tests
-</pre>
+There's generally no need to collect metric and experiment data outside production environment. Under Rails, Vanity turns collection on only if the environment name is "production". You can control this from @config/environments@ by setting @Vanity.playground.collecting@ to true/false. When collection is off, Vanity doesn't connect to the database server, so there's no need to set a database configuration for these environments.
h3(#dashboard). Dashboard
View
@@ -1,6 +1,7 @@
require "date"
require "time"
require "logger"
+require "cgi"
# All the cool stuff happens in other places.
# @see Vanity::Helper
@@ -25,6 +26,7 @@ module Version
require "vanity/metric/base"
require "vanity/metric/active_record"
require "vanity/metric/google_analytics"
+require "vanity/metric/remote"
# Experiments.
require "vanity/experiment/base"
require "vanity/experiment/ab_test"
@@ -30,25 +30,20 @@ def disconnect!
def reconnect!
end
- # Empty the database.
+ # Empty the database. This is used during tests.
def flushdb
end
# -- Metrics --
- # Store when metric was created (do not write over existing value).
- def set_metric_created_at(metric, time)
- fail "Not implemented"
- end
-
- # Return when metric was created.
- def get_metric_created_at(metric)
+ # Return when metric was last updated.
+ def get_metric_last_update_at(metric)
fail "Not implemented"
end
- # Track metric for given time instance.
- def metric_track(metric, time, count = 1)
+ # Track metric data.
+ def metric_track(metric, timestamp, identity, values)
fail "Not implemented"
end
@@ -45,24 +45,22 @@ def flushdb
# -- Metrics --
- def set_metric_created_at(metric, time)
- @metrics[metric] ||= {}
- @metrics[metric][:created_at] ||= time
- end
-
- def get_metric_created_at(metric)
- @metrics[metric] && @metrics[metric][:created_at]
+ def get_metric_last_update_at(metric)
+ @metrics[metric] && @metrics[metric][:last_update_at]
end
- def metric_track(metric, time, count = 1)
+ def metric_track(metric, timestamp, identity, values)
@metrics[metric] ||= {}
- @metrics[metric][time.to_date] ||= 0
- @metrics[metric][time.to_date] += count
+ current = @metrics[metric][timestamp.to_date] ||= []
+ values.each_with_index do |v,i|
+ current[i] = (current[i] || 0) + v || 0
+ end
+ @metrics[metric][:last_update_at] = Time.now
end
def metric_values(metric, from, to)
hash = @metrics[metric] || {}
- (from.to_date..to.to_date).map { |date| hash[date] || 0 }
+ (from.to_date..to.to_date).map { |date| hash[date] || [] }
end
def destroy_metric(metric)
@@ -52,36 +52,32 @@ def to_s
URI::Generic.build(:scheme=>"mongo", :userinfo=>userinfo, :host=>@options[:host], :port=>@options[:port], :path=>"/#{@options[:database]}").to_s
end
- def databse
- @databse
- end
-
def flushdb
- @metrics.remove
- @experiments.remove
- @participants.remove
+ @metrics.drop
+ @experiments.drop
+ @participants.drop
end
# -- Metrics --
- def set_metric_created_at(metric, time)
- @metrics.insert :_id=>metric, :created_at=>time
- end
-
- def get_metric_created_at(metric)
+ def get_metric_last_update_at(metric)
record = @metrics.find_one(:_id=>metric)
- record && record["created_at"]
+ record && record["last_update_at"]
end
- def metric_track(metric, time, count = 1)
- @metrics.update({ :_id=>metric }, { "$inc"=>{ "data.#{time.to_date}"=>count } })
+ def metric_track(metric, timestamp, identity, values)
+ inc = {}
+ values.each_with_index do |v,i|
+ inc["data.#{timestamp.to_date}.#{i}"] = v
+ end
+ @metrics.update({ :_id=>metric }, { "$inc"=>inc, "$set"=>{ :last_update_at=>Time.now } }, :upsert=>true)
end
def metric_values(metric, from, to)
record = @metrics.find_one(:_id=>metric)
data = record && record["data"] || {}
- (from.to_date..to.to_date).map { |date| data[date.to_s] || 0 }
+ (from.to_date..to.to_date).map { |date| (data[date.to_s] || {}).values }
end
def destroy_metric(metric)
Oops, something went wrong.

0 comments on commit 10b1523

Please sign in to comment.