-
-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Merged by Bors] - simple tool to compare traces between executions #4628
Conversation
|
Wait what? How did I miss that Tracy had that?? Awesome! |
While I like tracy, it's an extra tool to install and more complicated to use. I would like to promote more for PR the ability to compare spans before/after, but that's currently something that need a somewhat complicated setup. This tool was made in the idea of I added it in Bevy because it's not a general chrome trace comparison tool, it only supports the traces emitted by Bevy, and we can make it evolve based on the stats we want to monitor. |
bors try |
tryBuild failed: |
Honestly, I agree. It's already clunky to switch branches to run the profiles. The Tracy workflow then involves clicking around in a GUI to open the traces, find the relevant systems in the two traces to compare, noting down the statistics to write into a discussion / issue / PR / whatever. If I could just capture the two profiles and then run a command that spits out the information I want, that's going to be a faster workflow.
I'm in. And you know I'll make use of this so... |
I like the justification here. I'm on board with the design direction here, although I haven't reviewed the code in depth yet. |
tools/spancmp/src/main.rs
Outdated
/// name | ||
name: String, | ||
/// phase | ||
ph: String, | ||
/// timestamp | ||
ts: f32, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/// name | |
name: String, | |
/// phase | |
ph: String, | |
/// timestamp | |
ts: f32, | |
name: String, | |
phase: String, | |
timestamp: f32, |
and fix references. I see no reason to shorten the member names, nor to document with what the member names are/should be.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fields have the same name as in the json for deserialization. They can be annotated with the serde annotation to rename them, but I tend to prefer to stick to the same name as the json being deserialised when not part of the public api
tools/spancmp/src/main.rs
Outdated
second_trace: Option<String>, | ||
} | ||
|
||
const MARGIN_PERCENT: f32 = 0.02; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just hacked this in. It should probably be a CLI argument with a default. :) Later we can base it on the standard deviation of the distributions or something statistically sensible.
new look thanks to @superdump |
Can we use short names for the systems here? We already have a tool for this (get_short_name in bevy_reflect), that I refactored in #4299 |
Almost, but not exactly: here the name is not directly a rust type name, it's a string, and we want to keep part of that string. It's also useful to keep the full type to be able to filter all spans from System names in spans are not easy to read, but most of the info in their name is useful. That's also painful when using other traces tool 🙁 |
@alice-i-cecile in fact, it's possible with some regex I added a flag |
Co-Authored-By: Robert Swain <302146+superdump@users.noreply.github.com>
now with everything aligned and time units thanks to @superdump |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should merge this then continue iterating on it. :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm in agreement with Rob; this looks fine to me and I agree with the direction.
Because there's no user impact I'm happy to take a merge and iterate approach.
bors r+ |
# Objective - Have an easy way to compare spans between executions ## Solution - Add a tool to compare spans from chrome traces ```bash > cargo run --release -p spancmp -- --help Compiling spancmp v0.1.0 Finished release [optimized] target(s) in 1.10s Running `target/release/spancmp --help` spancmp USAGE: spancmp [OPTIONS] <TRACE> [SECOND_TRACE] ARGS: <TRACE> <SECOND_TRACE> OPTIONS: -h, --help Print help information -p, --pattern <PATTERN> Filter spans by name matching the pattern -t, --threshold <THRESHOLD> Filter spans that have an average shorther than the threshold [default: 0] ``` for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration. just displaying a trace ![Screenshot 2022-04-28 at 21 56 21](https://user-images.githubusercontent.com/8672791/165835310-f465c6f2-9e6b-4808-803e-884b06e49292.png) comparing two traces ![Screenshot 2022-04-28 at 21 56 55](https://user-images.githubusercontent.com/8672791/165835353-097d266b-a70c-41b8-a8c1-27804011dc97.png) Co-authored-by: Robert Swain <robert.swain@gmail.com>
Build failed: |
bors r+ |
# Objective - Have an easy way to compare spans between executions ## Solution - Add a tool to compare spans from chrome traces ```bash > cargo run --release -p spancmp -- --help Compiling spancmp v0.1.0 Finished release [optimized] target(s) in 1.10s Running `target/release/spancmp --help` spancmp USAGE: spancmp [OPTIONS] <TRACE> [SECOND_TRACE] ARGS: <TRACE> <SECOND_TRACE> OPTIONS: -h, --help Print help information -p, --pattern <PATTERN> Filter spans by name matching the pattern -t, --threshold <THRESHOLD> Filter spans that have an average shorther than the threshold [default: 0] ``` for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration. just displaying a trace ![Screenshot 2022-04-28 at 21 56 21](https://user-images.githubusercontent.com/8672791/165835310-f465c6f2-9e6b-4808-803e-884b06e49292.png) comparing two traces ![Screenshot 2022-04-28 at 21 56 55](https://user-images.githubusercontent.com/8672791/165835353-097d266b-a70c-41b8-a8c1-27804011dc97.png) Co-authored-by: Robert Swain <robert.swain@gmail.com>
# Objective - Have an easy way to compare spans between executions ## Solution - Add a tool to compare spans from chrome traces ```bash > cargo run --release -p spancmp -- --help Compiling spancmp v0.1.0 Finished release [optimized] target(s) in 1.10s Running `target/release/spancmp --help` spancmp USAGE: spancmp [OPTIONS] <TRACE> [SECOND_TRACE] ARGS: <TRACE> <SECOND_TRACE> OPTIONS: -h, --help Print help information -p, --pattern <PATTERN> Filter spans by name matching the pattern -t, --threshold <THRESHOLD> Filter spans that have an average shorther than the threshold [default: 0] ``` for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration. just displaying a trace ![Screenshot 2022-04-28 at 21 56 21](https://user-images.githubusercontent.com/8672791/165835310-f465c6f2-9e6b-4808-803e-884b06e49292.png) comparing two traces ![Screenshot 2022-04-28 at 21 56 55](https://user-images.githubusercontent.com/8672791/165835353-097d266b-a70c-41b8-a8c1-27804011dc97.png) Co-authored-by: Robert Swain <robert.swain@gmail.com>
# Objective - Have an easy way to compare spans between executions ## Solution - Add a tool to compare spans from chrome traces ```bash > cargo run --release -p spancmp -- --help Compiling spancmp v0.1.0 Finished release [optimized] target(s) in 1.10s Running `target/release/spancmp --help` spancmp USAGE: spancmp [OPTIONS] <TRACE> [SECOND_TRACE] ARGS: <TRACE> <SECOND_TRACE> OPTIONS: -h, --help Print help information -p, --pattern <PATTERN> Filter spans by name matching the pattern -t, --threshold <THRESHOLD> Filter spans that have an average shorther than the threshold [default: 0] ``` for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration. just displaying a trace ![Screenshot 2022-04-28 at 21 56 21](https://user-images.githubusercontent.com/8672791/165835310-f465c6f2-9e6b-4808-803e-884b06e49292.png) comparing two traces ![Screenshot 2022-04-28 at 21 56 55](https://user-images.githubusercontent.com/8672791/165835353-097d266b-a70c-41b8-a8c1-27804011dc97.png) Co-authored-by: Robert Swain <robert.swain@gmail.com>
# Objective - Have an easy way to compare spans between executions ## Solution - Add a tool to compare spans from chrome traces ```bash > cargo run --release -p spancmp -- --help Compiling spancmp v0.1.0 Finished release [optimized] target(s) in 1.10s Running `target/release/spancmp --help` spancmp USAGE: spancmp [OPTIONS] <TRACE> [SECOND_TRACE] ARGS: <TRACE> <SECOND_TRACE> OPTIONS: -h, --help Print help information -p, --pattern <PATTERN> Filter spans by name matching the pattern -t, --threshold <THRESHOLD> Filter spans that have an average shorther than the threshold [default: 0] ``` for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration. just displaying a trace ![Screenshot 2022-04-28 at 21 56 21](https://user-images.githubusercontent.com/8672791/165835310-f465c6f2-9e6b-4808-803e-884b06e49292.png) comparing two traces ![Screenshot 2022-04-28 at 21 56 55](https://user-images.githubusercontent.com/8672791/165835353-097d266b-a70c-41b8-a8c1-27804011dc97.png) Co-authored-by: Robert Swain <robert.swain@gmail.com>
Objective
Solution
for each span, it will display the count, minimum duration, average duration and max duration. It can be filtered by a pattern on the span name or by a minimum average duration.
just displaying a trace
![Screenshot 2022-04-28 at 21 56 21](https://user-images.githubusercontent.com/8672791/165835310-f465c6f2-9e6b-4808-803e-884b06e49292.png)
comparing two traces
![Screenshot 2022-04-28 at 21 56 55](https://user-images.githubusercontent.com/8672791/165835353-097d266b-a70c-41b8-a8c1-27804011dc97.png)