New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rec-4.1.x: Reduce systemcall usage in protobuf logging #7430

Merged
merged 2 commits into from Jan 30, 2019

Conversation

Projects
None yet
2 participants
@rgacogne
Copy link
Member

rgacogne commented Jan 29, 2019

Short description

This is mostly a backport of #7428 to rel/rec-4.1.x, but it also backports some protobuf regression tests as well.

Checklist

I have:

  • read the CONTRIBUTING.md document
  • compiled this code
  • tested this code
  • included documentation (including possible behaviour changes)
  • documented the code
  • added or modified regression test(s)
  • added or modified unit test(s)
  • checked that this code was merged to master
Reduce systemcall usage in protobuf logging
Since Spectre/Meltdown, system calls have become more expensive.  In
addition, relevant versions of glibc turn out to implement pthread_cond_wait
and pthread_cond_signal in such a way that they use multiple system calls always.
There is an optimization in glibc to improve this but it is disabled.

This new setup changes our protobuf logging so it amortizes system calls so we perform
far less than one call per message.

Note that our previous RemoteLogger was configured in terms of how many
*messages* it would buffer. Our new code is configured in terms of how many
*bytes*. I have multiplied the configured numbers by 100 elsewhere (recursor
config, dnsdist config) to sort of maintain parity.

In addition, the old RemoteLogger would buffer messages while there was no
connection available. We no longer do this.

Finally new, every 'reconnectTimeout' seconds we will flush our buffers
opportunistically to not keep people waiting.

@rgacogne rgacogne added the rec label Jan 29, 2019

@rgacogne rgacogne added this to the rec-4.1.x milestone Jan 29, 2019

@rgacogne rgacogne force-pushed the rgacogne:rec41-redo-remotelogger branch from 3aa56d2 to f3ad6bc Jan 29, 2019

@rgacogne rgacogne force-pushed the rgacogne:rec41-redo-remotelogger branch from f3ad6bc to c32ea3c Jan 29, 2019

@ahupowerdns ahupowerdns merged commit c925ec3 into PowerDNS:rel/rec-4.1.x Jan 30, 2019

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@rgacogne rgacogne deleted the rgacogne:rec41-redo-remotelogger branch Jan 30, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment