Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce debug logging overhead #6251

Merged
merged 5 commits into from Oct 25, 2019

Conversation

@michaelkaye
Copy link
Contributor

michaelkaye commented Oct 24, 2019

Pull Request Checklist

  • Pull request is based on the develop branch
  • Pull request includes a changelog file
  • Pull request includes a sign off

I'm not sure how much CPU this would likely save us, but I believe that the following three commits are safe and can avoid some formatting when not needed:

  • encode_canonical_json is not a trivial operation for debug logging (we do it twice - before and after we sign the event)
  • log.debug("%s", repr(obj)) => log.debug("%r", obj) will save calling repr() unless needed
  • log.debug("%s, %s" % (obj1,obj2)) => log.debug("%s, %s", obj1, obj2) will save string formatting unless needed

Hopefully in the worst case they're just noop changes.

michaelkaye added 5 commits Oct 24, 2019
I believe that string formatting ~10-15 sized events will
take a proportion of CPU time.
This avoids calculating __repr__ unless we are going to log.
Using % will cause the string to be generated even if debugging
is off.
@michaelkaye michaelkaye requested a review from matrix-org/synapse-core Oct 24, 2019
@erikjohnston erikjohnston merged commit 44ab048 into develop Oct 25, 2019
16 of 18 checks passed
16 of 18 checks passed
buildkite/synapse Build #5114 failed (22 minutes, 23 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-monolith Failed (exit status 1)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 37 seconds)
Details
buildkite/synapse/check-style Passed (1 minute, 44 seconds)
Details
buildkite/synapse/isort Passed (15 seconds)
Details
buildkite/synapse/mypy Passed (23 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (42 seconds)
Details
buildkite/synapse/packaging Passed (18 seconds)
Details
buildkite/synapse/pipeline Passed (3 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (17 minutes, 3 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (7 minutes, 1 second)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite-slash-old-deps Passed (8 minutes, 46 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (5 minutes, 55 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (18 minutes, 59 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (17 minutes, 41 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (6 minutes, 50 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-workers Passed (17 minutes, 18 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-sqlite-slash-monolith Passed (13 minutes, 34 seconds)
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.