Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

DNS bottleneck #5

Closed
minaguib opened this Issue Apr 19, 2011 · 4 comments

Comments

Projects
None yet
6 participants

(reported this bug in another statsd client, which reinh/statsd also exhibits same behavior):

Currently, the DNS service is queried once per sent packet if the supplied host is a hostname and not an IP address.

Ideally, the hostname is resolved only once (ever, or per configurable X seconds) and the resulting IP address is what's used in sending the UDP packet.

Now that I think of it, I believe (though untested) that if the DNS service is down or slow, the sending of the UDP packet in the current code will block while the resolving library tries to resolve the hostname to an IP address - so the whole benefit of UDP being fire-and-forget is overshadowed by the blocking DNS operation before the sending.

Ideally, DNS resolution (wherever it's done - by this code or lower) is done with an appropriate, configurable timeout - and not done too often.

Owner

reinh commented Apr 19, 2011

That is unfortunate. I may not have much control over DNS resolution but I will look into it. You could of course use an IP instead of a hostname.

mdp commented Aug 24, 2011

You could also do this in the library with something along the lines of:

def initialize(host, port)
  unless host.match(/^[^a-zA-Z].+$/)
    host = Resolv.getaddress(host)
  end

jeremy added a commit to jeremy/statsd-ruby that referenced this issue Dec 4, 2011

Resolve non-IP hosts to an IP immediately rather than incur a DNS loo…
…kup for each socket send

Addresses issue #5 - thanks @mdp!

ktheory commented Dec 19, 2011

This patch turns statsd into a badly behaving DNS client.

If the DNS of the hostname changes to a different IP address, one expects clients to pick up the change within the TTL. This patch seems to cache DNS for the lifetime of the process.

OTOH, I'm not observing that "DNS service is queried once per sent packet". In ruby 1.8.7 on OS X Lion, DNS works as expected: the first sent stat is slow, and subsequent stats are much faster for the duration of the TTL:

irb -rstatsd -rbenchmark
irb(main):001:0> Benchmark.realtime { Statsd.new("google.com").increment("foo") }
=> 0.0229549407958984 # Slow for DNS
irb(main):002:0> Benchmark.realtime { Statsd.new("google.com").increment("foo") }
=> 0.000710964202880859 # Too fast for a DNS request

Edit June 22 2014 0.00071 seconds (0.7ms) is not too fast for a DNS request. 🐳

Contributor

jeremy commented Dec 19, 2011

@ktheory indeed - pragmatic attempt to speed up the common case where DNS never changes, but this shouldn't be the client's job. I'll revert.

jeremy added a commit to jeremy/statsd-ruby that referenced this issue Dec 19, 2011

Revert "Resolve non-IP hosts to an IP immediately rather than incur a…
… DNS lookup for each socket send"

This reverts commit 8bcb43d addressing issue #5.

Thanks @ktheory for pointing out it's both incorrect & unnecessary!

Conflicts:

	lib/statsd.rb
	spec/statsd_spec.rb

@raggi raggi closed this in 98eadcf Jul 2, 2012

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment