Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic opentracing support #5544

Merged
merged 74 commits into from Jul 11, 2019

Conversation

@JorikSchellekens
Copy link
Contributor

commented Jun 24, 2019

I've set up opentracing for synapse. The configuration options as they stand ignore the whitelists. This is coming later. Jaeger comes with a bunch of config options but I assume we don't want to expose them.

I've written a custom scope manager which keeps a track of the currently active span in opentracing. This scope manager ties in with logcontexts where we generally create a new logcontext for each new scope. There is an exception to this when the current log context has no scope. In this case the scope is simply attached to the context and detached again on close. Otherwise the lifetime of a scope and it's logcontext are exactly the same. (This goes against what we discussed @erikjohnston but opening a new context with the scope in site.py (line 239) causes the logcontext to be dropped prematurely. If anyone sees a nicer way shout.)

JorikSchellekens added some commits Jun 21, 2019

Configure and initialise tracer
Includes config options for the tracer and sets up JaegerClient.
Scope manager using LogContexts
We piggy-back our tracer scopes by using log context.
The current log context gives us the current scope. If new scope is
created we create a stack of scopes in the context.
Trace federation requests on the way in and out.
The span is created in _started_processing and closed in
_finished_processing because we need a meaningful log context.
Create logcontext for new scope.
Instead of having a stack of scopes in a logcontext we create a new
context for a new scope if the current logcontext already has a scope.

@JorikSchellekens JorikSchellekens requested a review from matrix-org/synapse-core Jun 24, 2019

@JorikSchellekens JorikSchellekens force-pushed the joriks/opentracing branch from 895f68a to 6a6f97a Jun 25, 2019

Merge branch 'develop' into joriks/opentracing
Needed to get black commit because otherwise black
was trying to alter 290ish files.

@JorikSchellekens JorikSchellekens force-pushed the joriks/opentracing branch 3 times, most recently from 79033ed to 923a042 Jun 25, 2019

@JorikSchellekens JorikSchellekens force-pushed the joriks/opentracing branch from a60cc63 to db10eeb Jun 25, 2019

@codecov

This comment has been minimized.

Copy link

commented Jun 25, 2019

Codecov Report

Merging #5544 into develop will decrease coverage by 0.06%.
The diff coverage is 56.78%.

@@             Coverage Diff             @@
##           develop    #5544      +/-   ##
===========================================
- Coverage    63.25%   63.19%   -0.07%     
===========================================
  Files          328      331       +3     
  Lines        35854    36046     +192     
  Branches      5911     5934      +23     
===========================================
+ Hits         22681    22778      +97     
- Misses       11548    11637      +89     
- Partials      1625     1631       +6

@erikjohnston erikjohnston changed the title Joriks/opentracing Add basic opentracing support Jun 25, 2019

synapse/config/tracer.py Outdated Show resolved Hide resolved
synapse/util/tracerutils.py Outdated Show resolved Hide resolved

@JorikSchellekens JorikSchellekens force-pushed the joriks/opentracing branch from f14d57f to 14376e7 Jul 3, 2019

@JorikSchellekens JorikSchellekens force-pushed the joriks/opentracing branch 3 times, most recently from d38b7db to 7cd08fb Jul 4, 2019

@JorikSchellekens JorikSchellekens force-pushed the joriks/opentracing branch from 7cd08fb to 386285b Jul 4, 2019

@JorikSchellekens JorikSchellekens requested a review from erikjohnston Jul 4, 2019

@erikjohnston
Copy link
Member

left a comment

aaaaaaaaalmost there

synapse/logging/opentracing.py Outdated Show resolved Hide resolved
synapse/http/matrixfederationclient.py Outdated Show resolved Hide resolved
synapse/logging/opentracing.py Outdated Show resolved Hide resolved
synapse/logging/opentracing.py Outdated Show resolved Hide resolved
@erikjohnston
Copy link
Member

left a comment

🎉

@JorikSchellekens JorikSchellekens merged commit 38a6d3e into develop Jul 11, 2019

17 checks passed

buildkite/synapse Build #2751 passed (21 minutes, 13 seconds)
Details
buildkite/synapse/check-sample-config Passed (1 minute, 17 seconds)
Details
buildkite/synapse/check-style Passed (1 minute, 22 seconds)
Details
buildkite/synapse/isort Passed (20 seconds)
Details
buildkite/synapse/newspaper-newsfile Passed (16 seconds)
Details
buildkite/synapse/packaging Passed (18 seconds)
Details
buildkite/synapse/pipeline Passed (9 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-postgres-9-dot-5 Passed (17 minutes, 11 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite Passed (4 minutes, 45 seconds)
Details
buildkite/synapse/python-3-dot-5-slash-sqlite-slash-old-deps Passed (7 minutes, 41 seconds)
Details
buildkite/synapse/python-3-dot-6-slash-sqlite Passed (4 minutes, 25 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-11 Passed (17 minutes, 38 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-postgres-9-dot-5 Passed (19 minutes, 38 seconds)
Details
buildkite/synapse/python-3-dot-7-slash-sqlite Passed (4 minutes, 31 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-monolith Passed (5 minutes, 37 seconds)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-postgres-9-dot-6-slash-workers Soft failed (exit status 1)
Details
buildkite/synapse/sytest-python-3-dot-5-slash-sqlite-slash-monolith Passed (4 minutes, 27 seconds)
Details

anoadragon453 added a commit that referenced this pull request Jul 22, 2019

Merge tag 'v1.2.0rc1' into develop
v1.2.0rc1

Features
--------

- Add support for opentracing. ([\#5544](#5544), [\#5712](#5712))
- Add ability to pull all locally stored events out of synapse that a particular user can see. ([\#5589](#5589))
- Add a basic admin command app to allow server operators to run Synapse admin commands separately from the main production instance. ([\#5597](#5597))
- Add `sender` and `origin_server_ts` fields to `m.replace`. ([\#5613](#5613))
- Add default push rule to ignore reactions. ([\#5623](#5623))
- Include the original event when asking for its relations. ([\#5626](#5626))
- Implement `session_lifetime` configuration option, after which access tokens will expire. ([\#5660](#5660))
- Return "This account has been deactivated" when a deactivated user tries to login. ([\#5674](#5674))
- Enable aggregations support by default ([\#5714](#5714))

Bugfixes
--------

- Fix 'utime went backwards' errors on daemonization. ([\#5609](#5609))
- Various minor fixes to the federation request rate limiter. ([\#5621](#5621))
- Forbid viewing relations on an event once it has been redacted. ([\#5629](#5629))
- Fix requests to the `/store_invite` endpoint of identity servers being sent in the wrong format. ([\#5638](#5638))
- Fix newly-registered users not being able to lookup their own profile without joining a room. ([\#5644](#5644))
- Fix bug in #5626 that prevented the original_event field from actually having the contents of the original event in a call to `/relations`. ([\#5654](#5654))
- Fix 3PID bind requests being sent to identity servers as `application/x-form-www-urlencoded` data, which is deprecated. ([\#5658](#5658))
- Fix some problems with authenticating redactions in recent room versions. ([\#5699](#5699), [\#5700](#5700), [\#5707](#5707))
- Ignore redactions of m.room.create events. ([\#5701](#5701))

Updates to the Docker image
---------------------------

- Base Docker image on a newer Alpine Linux version (3.8 -> 3.10). ([\#5619](#5619))
- Add missing space in default logging file format generated by the Docker image. ([\#5620](#5620))

Improved Documentation
----------------------

- Add information about nginx normalisation to reverse_proxy.rst. Contributed by @skalarproduktraum - thanks! ([\#5397](#5397))
- --no-pep517 should be --no-use-pep517 in the documentation to setup the development environment. ([\#5651](#5651))
- Improvements to Postgres setup instructions. Contributed by @Lrizika - thanks! ([\#5661](#5661))
- Minor tweaks to postgres documentation. ([\#5675](#5675))

Deprecations and Removals
-------------------------

- Remove support for the `invite_3pid_guest` configuration setting. ([\#5625](#5625))

Internal Changes
----------------

- Move logging code out of `synapse.util` and into `synapse.logging`. ([\#5606](#5606), [\#5617](#5617))
- Add a blacklist file to the repo to blacklist certain sytests from failing CI. ([\#5611](#5611))
- Make runtime errors surrounding password reset emails much clearer. ([\#5616](#5616))
- Remove dead code for persiting outgoing federation transactions. ([\#5622](#5622))
- Add `lint.sh` to the scripts-dev folder which will run all linting steps required by CI. ([\#5627](#5627))
- Move RegistrationHandler.get_or_create_user to test code. ([\#5628](#5628))
- Add some more common python virtual-environment paths to the black exclusion list. ([\#5630](#5630))
- Some counter metrics exposed over Prometheus have been renamed, with the old names preserved for backwards compatibility and deprecated. See `docs/metrics-howto.rst` for details. ([\#5636](#5636))
- Unblacklist some user_directory sytests. ([\#5637](#5637))
- Factor out some redundant code in the login implementation. ([\#5639](#5639))
- Update ModuleApi to avoid register(generate_token=True). ([\#5640](#5640))
- Remove access-token support from `RegistrationHandler.register`, and rename it. ([\#5641](#5641))
- Remove access-token support from `RegistrationStore.register`, and rename it. ([\#5642](#5642))
- Improve logging for auto-join when a new user is created. ([\#5643](#5643))
- Remove unused and unnecessary check for FederationDeniedError in _exception_to_failure. ([\#5645](#5645))
- Fix a small typo in a code comment. ([\#5655](#5655))
- Clean up exception handling around client access tokens. ([\#5656](#5656))
- Add a mechanism for per-test homeserver configuration in the unit tests. ([\#5657](#5657))
- Inline issue_access_token. ([\#5659](#5659))
- Update the sytest BuildKite configuration to checkout Synapse in `/src`. ([\#5664](#5664))
- Add a `docker` type to the towncrier configuration. ([\#5673](#5673))
- Convert `synapse.federation.transport.server` to `async`. Might improve some stack traces. ([\#5689](#5689))
- Documentation for opentracing. ([\#5703](#5703))

@JorikSchellekens JorikSchellekens deleted the joriks/opentracing branch Jul 23, 2019

@JorikSchellekens JorikSchellekens restored the joriks/opentracing branch Jul 23, 2019

@JorikSchellekens JorikSchellekens deleted the joriks/opentracing branch Jul 23, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.