Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

New hash syntax #34

Merged
merged 6 commits into from
Feb 7, 2014
Merged

New hash syntax #34

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
44328fa
REVISE: needs fixing, doesn't work at all yet, but somewhat close
Soleone Feb 6, 2014
cd33e1b
Use optional hash for metric arguments instead of argument list.
wvanbergen Feb 7, 2014
293a329
Some more tests and a small fix.
wvanbergen Feb 7, 2014
8841d20
Update README.
wvanbergen Feb 7, 2014
4918143
Handle nil argument properly for StatsD.measure.
wvanbergen Feb 7, 2014
cc130c5
Remove valid_optional_arguments constant
wvanbergen Feb 7, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,15 +81,15 @@ StatsD.increment('GoogleBase.insert')
StatsD.increment('GoogleBase.insert', 10)
# you can also specify a sample rate, so only 1/10 of events
# actually get to statsd. Useful for very high volume data
StatsD.increment('GoogleBase.insert', 1, 0.1)
StatsD.increment('GoogleBase.insert', 1, sample_rate: 0.1)
```

#### StatsD.gauge

A gauge is a single numerical value value that tells you the state of the system at a point in time. A good example would be the number of messages in a queue.

``` ruby
StatsD.gauge('GoogleBase.queued', 12, 1.0)
StatsD.gauge('GoogleBase.queued', 12, sample_rate: 1.0)
```

Normally, you shouldn't update this value too often, and therefore there is no need to sample this kind metric.
Expand All @@ -100,7 +100,7 @@ A set keeps track of the number of unique values that have been seen. This is a

``` ruby
# Submit the customer ID to the set. It will only be counted if it hasn't been seen before.
StatsD.set('GoogleBase.customers', "12345", 1.0)
StatsD.set('GoogleBase.customers', "12345", sample_rate: 1.0)
```

Because you are counting unique values, the results of using a sampling value less than 1.0 can lead to unexpected, hard to interpret results.
Expand Down Expand Up @@ -174,7 +174,6 @@ end
You can instrument class methods, just like instance methods, using the metaprogramming methods. You simply have to configure the instrumentation on the singleton class of the Class you want to instrument.

```ruby
AWS::S3::Base.singleton_class.extend StatsD::Instrument
AWS::S3::Base.singleton_class.statsd_measure :request, 'S3.request'
```

Expand All @@ -189,6 +188,15 @@ passed.
GoogleBase.statsd_count :insert, lamdba{|object, args| object.class.to_s.downcase + "." + args.first.to_s + ".insert" }
```

### Tags

The Datadog implementation support tags, which you can use to slice and dice metrics in their UI. You can specify a list of tags as an option, either standalone tag (e.g. `"mytag"`), or key value based, separated by a colon: `"env:production"`.

``` ruby
StatsD.increment('my.counter', tags: ['env:production', 'unicorn'])
GoogleBase.statsd_count :insert, 'GoogleBase.insert', tags: ['env:production']
```

## Reliance on DNS

Out of the box StatsD is set up to be unidirectional fire-and-forget over UDP. Configuring the StatsD host to be a non-ip will trigger a DNS lookup (ie synchronous round trip network call) for each metric sent. This can be particularly problematic in clouds that have a shared DNS infrastructure such as AWS.
Expand Down
72 changes: 48 additions & 24 deletions lib/statsd/instrument.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@ def self.generate_metric_name(metric_name, callee, *args)
metric_name.respond_to?(:call) ? metric_name.call(callee, args).gsub('::', '.') : metric_name.gsub('::', '.')
end

def statsd_measure(method, name, sample_rate = StatsD.default_sample_rate)
def statsd_measure(method, name, *metric_options)
add_to_method(method, name, :measure) do |old_method, new_method, metric_name, *args|
define_method(new_method) do |*args, &block|
StatsD.measure(StatsD::Instrument.generate_metric_name(metric_name, self, *args), nil, sample_rate) { send(old_method, *args, &block) }
StatsD.measure(StatsD::Instrument.generate_metric_name(metric_name, self, *args), nil, *metric_options) { send(old_method, *args, &block) }
end
end
end

def statsd_count_success(method, name, sample_rate = StatsD.default_sample_rate)
def statsd_count_success(method, name, *metric_options)
add_to_method(method, name, :count_success) do |old_method, new_method, metric_name|
define_method(new_method) do |*args, &block|
begin
Expand All @@ -52,13 +52,13 @@ def statsd_count_success(method, name, sample_rate = StatsD.default_sample_rate)
result
ensure
suffix = truthiness == false ? 'failure' : 'success'
StatsD.increment("#{StatsD::Instrument.generate_metric_name(metric_name, self, *args)}.#{suffix}", 1, sample_rate)
StatsD.increment("#{StatsD::Instrument.generate_metric_name(metric_name, self, *args)}.#{suffix}", 1, *metric_options)
end
end
end
end

def statsd_count_if(method, name, sample_rate = StatsD.default_sample_rate)
def statsd_count_if(method, name, *metric_options)
add_to_method(method, name, :count_if) do |old_method, new_method, metric_name|
define_method(new_method) do |*args, &block|
begin
Expand All @@ -70,16 +70,16 @@ def statsd_count_if(method, name, sample_rate = StatsD.default_sample_rate)
truthiness = (yield(result) rescue false) if block_given?
result
ensure
StatsD.increment(StatsD::Instrument.generate_metric_name(metric_name, self, *args), sample_rate) if truthiness
StatsD.increment(StatsD::Instrument.generate_metric_name(metric_name, self, *args), *metric_options) if truthiness
end
end
end
end

def statsd_count(method, name, sample_rate = StatsD.default_sample_rate)
def statsd_count(method, name, *metric_options)
add_to_method(method, name, :count) do |old_method, new_method, metric_name|
define_method(new_method) do |*args, &block|
StatsD.increment(StatsD::Instrument.generate_metric_name(metric_name, self, *args), 1, sample_rate)
StatsD.increment(StatsD::Instrument.generate_metric_name(metric_name, self, *args), 1, *metric_options)
send(old_method, *args, &block)
end
end
Expand Down Expand Up @@ -130,39 +130,62 @@ def remove_from_method(method, name, action)
end

# glork:320|ms
def self.measure(key, milli = nil, sample_rate = default_sample_rate, tags = nil)
def self.measure(key, value = nil, *metric_options)
if value.is_a?(Hash) && metric_options.empty?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if you want/can extract these 4 lines in to a method. you have pretty much the same code again in increment. might not be worth it, but maybe consider it if more public methods are gonna be added and it might potentially be copy/pasted there again.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to implement it, but it actually gets more messy, because the value not being set means something different depending on the case. The code actually ends up being way more complicated.

metric_options = [value]
value = nil
end

result = nil
ms = milli || 1000 * Benchmark.realtime do
ms = value || 1000 * Benchmark.realtime do
result = yield
end

collect(key, ms, :ms, sample_rate, tags)
collect(:ms, key, ms, hash_argument(metric_options))
result
end

# gorets:1|c
def self.increment(key, delta = 1, sample_rate = default_sample_rate, tags = nil)
collect(key, delta, :incr, sample_rate, tags)
def self.increment(key, value = 1, *metric_options)
if value.is_a?(Hash) && metric_options.empty?
metric_options = [value]
value = 1
end

collect(:incr, key, value, hash_argument(metric_options))
end

# gaugor:333|g
# guagor:1234|kv|@1339864935 (statsite)
def self.gauge(key, value, sample_rate_or_epoch = default_sample_rate, tags = nil)
collect(key, value, :g, sample_rate_or_epoch, tags)
def self.gauge(key, value, *metric_options)
collect(:g, key, value, hash_argument(metric_options))
end

# histogram:123.45|h
def self.histogram(key, value, sample_rate_or_epoch = default_sample_rate, tags = nil)
collect(key, value, :h, sample_rate_or_epoch, tags)
def self.histogram(key, value, *metric_options)
collect(:h, key, value, hash_argument(metric_options))
end

# uniques:765|s
def self.set(key, value, sample_rate_or_epoch = default_sample_rate, tags = nil)
collect(key, value, :s, sample_rate_or_epoch, tags)
def self.set(key, value, *metric_options)
collect(:s, key, value, hash_argument(metric_options))
end

private

def self.hash_argument(args)
return {} if args.length == 0
return args.first if args.length == 1 && args.first.is_a?(Hash)

order = [:sample_rate, :tags]
hash = {}
args.each_with_index do |value, index|
hash[order[index]] = value
end

return hash
end

def self.invalidate_socket
@socket = nil
end
Expand All @@ -175,12 +198,13 @@ def self.socket
@socket
end

def self.collect(k, v, op, sample_rate = default_sample_rate, tags = nil)
def self.collect(type, k, v, options = {})
return unless enabled
sample_rate = options[:sample_rate] || StatsD.default_sample_rate
return if sample_rate < 1 && rand > sample_rate

command = generate_packet(k, v, op, sample_rate, tags)
write_packet(command)
packet = generate_packet(type, k, v, sample_rate, options[:tags])
write_packet(packet)
end

def self.write_packet(command)
Expand All @@ -202,9 +226,9 @@ def self.clean_tags(tags)
end
end

def self.generate_packet(k, v, op, sample_rate = default_sample_rate, tags = nil)
def self.generate_packet(type, k, v, sample_rate = default_sample_rate, tags = nil)
command = "#{self.prefix + '.' if self.prefix}#{k}:#{v}"
case op
case type
when :incr
command << '|c'
when :ms
Expand Down
36 changes: 23 additions & 13 deletions test/statsd_instrumentation_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ class StatsDInstrumentationTest < Test::Unit::TestCase
def test_statsd_count_if
ActiveMerchant::Gateway.statsd_count_if :ssl_post, 'ActiveMerchant.Gateway.if'

StatsD.expects(:increment).with('ActiveMerchant.Gateway.if', 1).once
StatsD.expects(:increment).with('ActiveMerchant.Gateway.if').once
ActiveMerchant::Gateway.new.purchase(true)
ActiveMerchant::Gateway.new.purchase(false)

Expand All @@ -63,7 +63,7 @@ def test_statsd_count_if_with_method_receiving_block
result == 'true'
end

StatsD.expects(:collect).with('ActiveMerchant.Base.post_with_block', 1, :incr, 1.0, nil).once
StatsD.expects(:collect).with(:incr, 'ActiveMerchant.Base.post_with_block', 1, {}).once
assert_equal 'true', ActiveMerchant::Base.new.post_with_block { 'true' }
assert_equal 'false', ActiveMerchant::Base.new.post_with_block { 'false' }

Expand All @@ -75,7 +75,7 @@ def test_statsd_count_if_with_block
result[:success]
end

StatsD.expects(:increment).with('ActiveMerchant.Gateway.block', 1).once
StatsD.expects(:increment).with('ActiveMerchant.Gateway.block').once
ActiveMerchant::UniqueGateway.new.purchase(true)
ActiveMerchant::UniqueGateway.new.purchase(false)

Expand All @@ -99,8 +99,8 @@ def test_statsd_count_success_with_method_receiving_block
result == 'successful'
end

StatsD.expects(:collect).with('ActiveMerchant.Base.post_with_block.success', 1, :incr, 1.0, nil).once
StatsD.expects(:collect).with('ActiveMerchant.Base.post_with_block.failure', 1, :incr, 1.0, nil).once
StatsD.expects(:collect).with(:incr, 'ActiveMerchant.Base.post_with_block.success', 1, {}).once
StatsD.expects(:collect).with(:incr, 'ActiveMerchant.Base.post_with_block.failure', 1, {}).once

assert_equal 'successful', ActiveMerchant::Base.new.post_with_block { 'successful' }
assert_equal 'not so successful', ActiveMerchant::Base.new.post_with_block { 'not so successful' }
Expand All @@ -113,10 +113,10 @@ def test_statsd_count_success_with_block
result[:success]
end

StatsD.expects(:increment).with('ActiveMerchant.Gateway.success', 1, StatsD.default_sample_rate)
StatsD.expects(:increment).with('ActiveMerchant.Gateway.success', 1)
ActiveMerchant::UniqueGateway.new.purchase(true)

StatsD.expects(:increment).with('ActiveMerchant.Gateway.failure', 1, StatsD.default_sample_rate)
StatsD.expects(:increment).with('ActiveMerchant.Gateway.failure', 1)
ActiveMerchant::UniqueGateway.new.purchase(false)

ActiveMerchant::UniqueGateway.statsd_remove_count_success :ssl_post, 'ActiveMerchant.Gateway'
Expand All @@ -125,7 +125,7 @@ def test_statsd_count_success_with_block
def test_statsd_count
ActiveMerchant::Gateway.statsd_count :ssl_post, 'ActiveMerchant.Gateway.ssl_post'

StatsD.expects(:increment).with('ActiveMerchant.Gateway.ssl_post', 1, StatsD.default_sample_rate)
StatsD.expects(:increment).with('ActiveMerchant.Gateway.ssl_post', 1)
ActiveMerchant::Gateway.new.purchase(true)

ActiveMerchant::Gateway.statsd_remove_count :ssl_post, 'ActiveMerchant.Gateway.ssl_post'
Expand All @@ -135,7 +135,7 @@ def test_statsd_count_with_name_as_lambda
metric_namer = lambda { |object, args| object.class.to_s.downcase + ".insert." + args.first.to_s }
ActiveMerchant::Gateway.statsd_count(:ssl_post, metric_namer)

StatsD.expects(:increment).with('gatewaysubclass.insert.true', 1, StatsD.default_sample_rate)
StatsD.expects(:increment).with('gatewaysubclass.insert.true', 1)
GatewaySubClass.new.purchase(true)

ActiveMerchant::Gateway.statsd_remove_count(:ssl_post, metric_namer)
Expand All @@ -144,7 +144,7 @@ def test_statsd_count_with_name_as_lambda
def test_statsd_count_with_method_receiving_block
ActiveMerchant::Base.statsd_count :post_with_block, 'ActiveMerchant.Base.post_with_block'

StatsD.expects(:collect).with('ActiveMerchant.Base.post_with_block', 1, :incr, 1.0, nil)
StatsD.expects(:collect).with(:incr, 'ActiveMerchant.Base.post_with_block', 1, {})
assert_equal 'block called', ActiveMerchant::Base.new.post_with_block { 'block called' }

ActiveMerchant::Base.statsd_remove_count :post_with_block, 'ActiveMerchant.Base.post_with_block'
Expand All @@ -153,7 +153,7 @@ def test_statsd_count_with_method_receiving_block
def test_statsd_measure_with_nested_modules
ActiveMerchant::UniqueGateway.statsd_measure :ssl_post, 'ActiveMerchant::Gateway.ssl_post'

StatsD.expects(:measure).with('ActiveMerchant.Gateway.ssl_post', nil, StatsD.default_sample_rate)
StatsD.expects(:measure).with('ActiveMerchant.Gateway.ssl_post', nil)
ActiveMerchant::UniqueGateway.new.purchase(true)

ActiveMerchant::UniqueGateway.statsd_remove_measure :ssl_post, 'ActiveMerchant::Gateway.ssl_post'
Expand All @@ -171,7 +171,7 @@ def test_statsd_measure
def test_statsd_measure_with_method_receiving_block
ActiveMerchant::Base.statsd_measure :post_with_block, 'ActiveMerchant.Base.post_with_block'

StatsD.expects(:collect).with('ActiveMerchant.Base.post_with_block', is_a(Float), :ms, 1.0, nil)
StatsD.expects(:collect).with(:ms, 'ActiveMerchant.Base.post_with_block', is_a(Float), {})
assert_equal 'block called', ActiveMerchant::Base.new.post_with_block { 'block called' }

ActiveMerchant::Base.statsd_remove_measure :post_with_block, 'ActiveMerchant.Base.post_with_block'
Expand All @@ -181,7 +181,17 @@ def test_instrumenting_class_method
ActiveMerchant::Gateway.singleton_class.extend StatsD::Instrument
ActiveMerchant::Gateway.singleton_class.statsd_count :sync, 'ActiveMerchant.Gateway.sync'

StatsD.expects(:increment).with('ActiveMerchant.Gateway.sync', 1, StatsD.default_sample_rate)
StatsD.expects(:increment).with('ActiveMerchant.Gateway.sync', 1)
ActiveMerchant::Gateway.sync

ActiveMerchant::Gateway.singleton_class.statsd_remove_count :sync, 'ActiveMerchant.Gateway.sync'
end

def test_statsd_count_with_tags
ActiveMerchant::Gateway.singleton_class.extend StatsD::Instrument
ActiveMerchant::Gateway.singleton_class.statsd_count :sync, 'ActiveMerchant.Gateway.sync', :tags => ['t']

StatsD.expects(:increment).with('ActiveMerchant.Gateway.sync', 1, :tags => ['t'])
ActiveMerchant::Gateway.sync

ActiveMerchant::Gateway.singleton_class.statsd_remove_count :sync, 'ActiveMerchant.Gateway.sync'
Expand Down
37 changes: 28 additions & 9 deletions test/statsd_test.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,37 +16,56 @@ def setup
end

def test_statsd_measure_with_explicit_value
StatsD.expects(:collect).with('values.foobar', 42, :ms, 1.0, nil)
StatsD.expects(:collect).with(:ms, 'values.foobar', 42, {})
StatsD.measure('values.foobar', 42)
end

def test_statsd_measure_with_explicit_value_and_sample_rate
StatsD.expects(:collect).with('values.foobar', 42, :ms, 0.1, nil)
StatsD.measure('values.foobar', 42, 0.1)
StatsD.expects(:collect).with(:ms, 'values.foobar', 42, :sample_rate => 0.1)
StatsD.measure('values.foobar', 42, :sample_rate => 0.1)
end

def test_statsd_measure_with_benchmarked_value
Benchmark.stubs(:realtime).returns(1.12)
StatsD.expects(:collect).with('values.foobar', 1120.0, :ms, 1.0, nil)
StatsD.measure('values.foobar', nil) do
StatsD.expects(:collect).with(:ms, 'values.foobar', 1120.0, {})
StatsD.measure('values.foobar') do
#noop
end
end


def test_statsd_measure_with_benchmarked_value_and_options
Benchmark.stubs(:realtime).returns(1.12)
StatsD.expects(:collect).with(:ms, 'values.foobar', 1120.0, :sample_rate => 1.0)
StatsD.measure('values.foobar', :sample_rate => 1.0) do
#noop
end
end

def test_statsd_increment_with_hash_argument
StatsD.expects(:collect).with(:incr, 'values.foobar', 1, :tags => ['test'])
StatsD.increment('values.foobar', :tags => ['test'])
end

def test_statsd_increment_with_multiple_arguments
StatsD.expects(:collect).with(:incr, 'values.foobar', 12, :sample_rate => nil, :tags => ['test'])
StatsD.increment('values.foobar', 12, nil, ['test'])
end

def test_statsd_gauge
StatsD.expects(:collect).with('values.foobar', 12, :g, 1.0, nil)
StatsD.expects(:collect).with(:g, 'values.foobar', 12, {})
StatsD.gauge('values.foobar', 12)
end

def test_statsd_set
StatsD.expects(:collect).with('values.foobar', 12, :s, 1.0, nil)
StatsD.expects(:collect).with(:s, 'values.foobar', 12, {})
StatsD.set('values.foobar', 12)
end

def test_statsd_histogram_on_datadog
StatsD.stubs(:implementation).returns(:datadog)
StatsD.expects(:collect).with('values.hg', 12.33, :h, 0.2, ['tag_123', 'key-name:value123'])
StatsD.histogram('values.hg', 12.33, 0.2, ['tag_123', 'key-name:value123'])
StatsD.expects(:collect).with(:h, 'values.hg', 12.33, :sample_rate => 0.2, :tags => ['tag_123', 'key-name:value123'])
StatsD.histogram('values.hg', 12.33, :sample_rate => 0.2, :tags => ['tag_123', 'key-name:value123'])
end

def test_collect_respects_enabled
Expand Down