-
Notifications
You must be signed in to change notification settings - Fork 714
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tokio-trace: how to use Spans to represent concurrency? #80
Comments
Ah, I just noticed |
I don't think The case you describe does sound like a parent / child relationship. The work in the spawned task is entirely in service of the spawner. Being able to represent this as a parent / child was an original design goal. Thinking about it, I wonder if it was lost a bit over iterations. IIRC, my thinking was that the spawner and the spawned task would get linked via a manual step. There would be an API to say that span A is a child of span B. However, parent / child was removed from -core. My thought was that it could be entirely handled by the subscriber by observing how spans are entered / exited. Perhaps this isn't sufficient. There are a few questions in my mind:
|
@jsgf this is an excellent question, thanks! The answer right now is that this is something that the core With that said, here are some of my thoughts that have fallen out of experimenting with subscriber implementations:
In the use-case you described, representing the parent future with a span and having each spawned future correspond to a new span inside the spawning future's span definitely seems like what I would do. I would probably do something like this (using the use futures::future::{self, Future};
use tokio_trace_futures::Instrument;
fn parent_task(how_many: usize) -> impl Future<Item=(), Error=()> {
future::lazy(move || {
info!("spawning subtasks...");
let subtasks = (1..=how_many).map(|i| {
debug!(message = "creating subtask;", number = i);
subtask(i)
}).collect::<Vec<_>>();
future::join_all(subtasks)
})
.map(|result| {
debug!("all subtasks completed");
let sum: usize = result.into_iter().sum();
info!(sum = sum);
})
.instrument(span!("parent_task", subtasks = how_many))
}
fn subtask(number: usize) -> impl Future<Item=usize, Error=()> {
future::lazy(move || {
info!("polling subtask...");
Ok(number)
}).instrument(span!("subtask", number = number))
} Running this as an example (using the
which seems about like what I would expect to see from this code. |
As a side note, sorry that the nursery crates I mentioned above are not currently very well documented or discoverable; they're still works in progress (is is what happens when one tries to materialize an entire library ecosystem from scratch. :P). |
Thanks for your explanations. I think I have a clearer idea about how things fit together. Specifically the understanding I have now is:
Does that sound about right? |
My gut is that that using a thread local would be a good default for a subscriber to use if no parent / child relationship is specified, but there should be a way using the instrumentation API to explicitly create a parent / child relationship or to create an unrooted span. I don't think we use span creation to determine parent / child given that spans are not entered when created. Subscribers probably have to observe the first time a span is entered and find the parent then. |
I'm open to adding this.
I don't agree --- in my experience, considering the span in which a span was created to be its parent leads to behaviour more closely in line with the user's expectations. For example, consider code like span!("span_1").enter(|| {
let fut = foo
.and_then(...)
// ...
.instrument(span!("span 2"));
tokio::spawn(fut);
}); Logically, I would expect But, regardless, this is behaviour that ought to be determined by the subscriber. *Or, will be a child of whatever span the worker thread that began executing it was currently inside of when the future was polled for the first time, which could be very surprising indeed. |
When would such a snippet ever happen? Spans are created in future constructors. In practice, it would be something like this: TcpStream::connect(...)
.instrument(span!("child"))
.and_then(|...| {
})
.instrument(span!("parent")) If the parent / child link happens at construction time, it will fail here. |
I think we've put some work into fixing up the docs so that this is a little clearer. I'm closing this ticket for now. |
I've been reading the code for tokio-trace, and I've got some questions about
Span
s and how they're intended to be used.It looks like they're pretty good at representing strictly nested time-periods, as you'd get with a function call tree. But its unclear to me if the direct children of a parent
Span
can overlap with each other (ie, representing long-term concurrent operations).For example, if I have a Future which is managing 10 concurrent connections which it initiates at once and then waits for each of them to produce a result, could I represent that as a parent
Span
with 10 sub-Span
representing each connection (which in turn may have sub-spans, etc)? It seems like that would't work because there's always a notion of "the" currentSpan
.Or is this not the intended level of granularity? Should I be using
Span
just to track specific calls toFuture
/Stream::poll
and derive tracing at that level? I can see how that would be possible, but it seems like it would be hard to reconstruct application-level states from that level of granularity.The example doesn't really help clarify this, because it omits all actual async behaviour.
On a more general level, is there a way to represent a dependency relationship in the trace? For example, if task A is waiting for something from task B, can that be recorded?
The text was updated successfully, but these errors were encountered: