New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

dnsdist: do not iterate over hash map for chashed based query distribution #6939

Merged
merged 1 commit into from Sep 25, 2018

Conversation

Projects
None yet
3 participants
@chbruyand
Member

chbruyand commented Sep 7, 2018

Short description

This changes how consistent hashing based query distribution is made to use std::lower_bound instead of iterating over hashes map.

Fix #6932

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled this code
  • tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)
@pavel-odintsov

This comment has been minimized.

pavel-odintsov commented Sep 7, 2018

Nice! Thank you for doing it!

@pieterlexis pieterlexis requested a review from rgacogne Sep 17, 2018

@pavel-odintsov

This comment has been minimized.

pavel-odintsov commented Sep 21, 2018

Thank you so much for writing this fix! I prepared binary package with this patch and it works very well.

Before this patch, each lookup to dsndist consumed about 200ms:

Got answer from 1.1.1.1:53, relayed to 127.0.0.1:33684, took 130457 usec
Got query for habr.com.|A from 127.0.0.1:42693, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:42693, took 146052 usec
Got query for habr.com.|A from 127.0.0.1:48349, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:48349, took 172195 usec
Got query for habr.com.|A from 127.0.0.1:39512, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:39512, took 183957 usec
Got query for habr.com.|A from 127.0.0.1:54989, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:54989, took 215102 usec
Got query for habr.com.|A from 127.0.0.1:35290, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:35290, took 162011 usec
Got query for habr.com.|A from 127.0.0.1:48629, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:48629, took 144442 usec

After applying patch, lookup time decreased drastically :

Got answer from 1.0.0.1:53, relayed to 127.0.0.1:46597, took 2125.57 usec
Got query for google.com.|A from 127.0.0.1:38102, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:38102, took 3426.38 usec
Got query for yandex.ru.|A from 127.0.0.1:58340, relayed to third
Got answer from 1.0.0.1:53, relayed to 127.0.0.1:58340, took 3099.86 usec
Got query for google.com.|A from 127.0.0.1:35976, relayed to second
Got answer from 1.1.1.1:53, relayed to 127.0.0.1:35976, took 5190.43 usec
Got query for yandex.ru.|A from 127.0.0.1:35590, relayed to third
Got answer from 1.0.0.1:53, relayed to 127.0.0.1:35590, took 3663.64 usec

Direct lookup to 1.1.1.1 or 1.0.0.1 from my machine is about 2ms.

@rgacogne rgacogne added this to the dnsdist-1.3.x milestone Sep 25, 2018

@rgacogne

LGTM!

@rgacogne rgacogne merged commit 628b164 into PowerDNS:master Sep 25, 2018

4 checks passed

LGTM analysis: C/C++ No alert changes
Details
LGTM analysis: JavaScript No alert changes
Details
LGTM analysis: Python No alert changes
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@pavel-odintsov

This comment has been minimized.

pavel-odintsov commented Sep 26, 2018

Thanks!

@chbruyand chbruyand deleted the chbruyand:dnsdist-chashed-cpu branch Dec 5, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment