New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement experimental metric tracking time spent within PowerDNS per query #5774

Merged
merged 5 commits into from Oct 25, 2017

Conversation

Projects
None yet
5 participants
@ahupowerdns
Member

ahupowerdns commented Oct 6, 2017

Short description

With this commit, PowerDNS provides metrics on the difference between the time spent waiting for authoritative servers, and the amount of time elapsed between arrival of query
and sending out the response. This metric should be seen as experimental until operational experience proves its relevance.

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled and tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)
  • checked that this code was merged to master

ahupowerdns added some commits Oct 6, 2017

Implement experimental metric tracking time spent within PowerDNS per…
… query

With this commit, PowerDNS provides metrics on the difference between the time spent waiting for authoritative servers, and the amount of time elapsed between arrival of query
and sending out the response. This metric should be seen as experimental until operational experience proves its relevance.
as expected, the 'delta' latency between time passed and time spent o…
…n network sometimes goes negative (because we use timers like that). Protect ourselves against messing up the stats when that happens (it is very rare, but it happened to me)

@Habbie Habbie added this to the rec-4.1.0 milestone Oct 23, 2017

@aerique aerique requested review from Habbie and rgacogne and removed request for Habbie Oct 24, 2017

@rgacogne

One issue worth fixing before merging, IMHO, looks good otherwise.

Show outdated Hide outdated pdns/pdns_recursor.cc Outdated
@@ -1253,6 +1253,28 @@ static void startDoResolve(void *p)
newLat = min(newLat,(uint64_t)(((uint64_t) g_networkTimeoutMsec)*1000)); // outliers of several minutes exist..
g_stats.avgLatencyUsec=(1-1.0/g_latencyStatSize)*g_stats.avgLatencyUsec + (float)newLat/g_latencyStatSize;
// no worries, we do this for packet cache hits elsewhere
auto ourtime = 1000.0*spent-sr.d_totUsec/1000.0; // in msec

This comment has been minimized.

@rgacogne

rgacogne Oct 25, 2017

Member

It looks like we are not updating SyncRes::d_totUsec when we forward a cache-only (RD=0) query to a hard-coded server in SyncRes::doResolve(). This would look like we spent a lot of time in the recursor when we didn't.

@rgacogne

rgacogne Oct 25, 2017

Member

It looks like we are not updating SyncRes::d_totUsec when we forward a cache-only (RD=0) query to a hard-coded server in SyncRes::doResolve(). This would look like we spent a lot of time in the recursor when we didn't.

This comment has been minimized.

@ahupowerdns

ahupowerdns Oct 25, 2017

Member

nice! solved.

@ahupowerdns

ahupowerdns Oct 25, 2017

Member

nice! solved.

@rgacogne

LGTM!

@@ -536,6 +536,10 @@ int SyncRes::doResolve(const DNSName &qname, const QType &qtype, vector<DNSRecor
boost::optional<Netmask> nm;
res=asyncresolveWrapper(remoteIP, d_doDNSSEC, qname, qtype.getCode(), false, false, &d_now, nm, &lwr);
d_totUsec += lwr.d_usec;
accountAuthLatency(lwr.d_usec, remoteIP.sin4.sin_family);

This comment has been minimized.

@pieterlexis

pieterlexis Oct 25, 2017

Member

I think this incidentally solves #5207 and #5187?

@pieterlexis

pieterlexis Oct 25, 2017

Member

I think this incidentally solves #5207 and #5187?

This comment has been minimized.

@rgacogne

rgacogne Oct 25, 2017

Member

I don't think it does?

@rgacogne

rgacogne Oct 25, 2017

Member

I don't think it does?

This comment has been minimized.

@pieterlexis

pieterlexis Oct 25, 2017

Member

ah, this has indeed nothing to do with the speed tracking per-ns

@pieterlexis

pieterlexis Oct 25, 2017

Member

ah, this has indeed nothing to do with the speed tracking per-ns

@aerique aerique merged commit fc724ef into PowerDNS:master Oct 25, 2017

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment