Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Support DOT graph printing from log results
Made log entries per-argument Added :move log entries for move() calls Passed thunk function to :compute logs Passed thunk function and input argument ids to :comm, :move logs Allowed changing timeline between timespans Added incoming thunk ID to :comm and :move timespan keys Added result type and size info to :compute timeline Added logging and graphing docs
- Loading branch information
Showing
7 changed files
with
267 additions
and
40 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,41 @@ | ||
# Logging and Graphing | ||
|
||
Dagger's scheduler keeps track of the important and potentially expensive | ||
actions it does, such as moving data between workers or executing thunks, and | ||
tracks how much time and memory allocations these operations consume. Saving | ||
this information somewhere accessible is disabled by default, but it's quite | ||
easy to turn it on: | ||
|
||
```julia | ||
ctx = Context() | ||
log = Dagger.LocalEventLog() | ||
ctx.log_sink = log | ||
``` | ||
|
||
Now anytime `ctx` is used as the context for a scheduler, the scheduler will | ||
log events into `log`. A `LocalEventLog` logs information in-memory, and does | ||
so on each worker. The default log object is a `NoOpLog`, which doesn't store | ||
events at all. The `FilterLog` exists to allow writing events to a | ||
user-defined location (such as a database, file, or network socket). | ||
|
||
Once sufficient data has been accumulated into a `LocalEventLog`, it can be | ||
gathered to a single host via `Dagger.get_logs!(log)`. The result is a | ||
`Vector` of `Dagger.Timespan` objects, which describe some metadata about an | ||
operation that occured and the scheduler logged. These events may be | ||
introspected directly, or may also be rendered to a DOT-format string: | ||
|
||
```julia | ||
logs = Dagger.get_logs!(log) | ||
str = Dagger.show_plan(logs) | ||
``` | ||
|
||
`Dagger.show_plan` can also be called as `Dagger.show_plan(io::IO, logs)` to | ||
write the graph to a file or other `IO` object. The string generated by this | ||
function may be passed to an external tool like `Graphviz` for rendering. Note | ||
that this method doesn't display input arguments to the DAG (non-`Thunk`s); | ||
you can call `Dagger.show_plan(logs, thunk)`, where `thunk` is the output | ||
`Thunk` of the DAG, to render argument nodes. | ||
|
||
!!! note | ||
`Dagger.get_logs!` clears out the event logs, so that old events don't mix | ||
with new ones from future DAGs. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.