-
Notifications
You must be signed in to change notification settings - Fork 25.3k
Do not automatically start tracing transactions in TaskManager #130607
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Do not automatically start tracing transactions in TaskManager #130607
Conversation
This changes TaskManager to only create new tracing spans if a trace parent (and hence a tracing transaction) already exists. Tracing transactions are automatically started in the RestController for external requests. To trace detached, internal transport actions, a transaction has to be explicitly started using Tracer.startTrace. This helps to prevent everlasting tracing transaction for internal self-rescheduling work that is using transport actions. Such transactions have caused memory issues with the APM java agent in the past. Relates to ES-10969
Pinging @elastic/es-core-infra (Team:Core/Infra) |
server/src/main/java/org/elasticsearch/common/util/concurrent/ThreadContext.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All makes sense to me - I have left comments, some of which suggest maybe breaking this up into separate PRs because there's a lot of stuff going on here.
server/src/main/java/org/elasticsearch/common/util/concurrent/ThreadContext.java
Outdated
Show resolved
Hide resolved
@DaveCTurner this PR contains only the remaining key changes and should be ready for another review |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great stuff LGTM now
This changes TaskManager to only create new tracing spans if a trace parent (and hence a tracing transaction) already exists.
Tracing transactions are automatically started in the RestController for external requests. To trace detached, internal transport actions, a transaction has to be explicitly started using Tracer.startTrace.
This helps to prevent everlasting tracing transaction for internal self-rescheduling work that is using transport actions. Such transactions have caused memory issues with the APM java agent in the past.
Relates to ES-10969