Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENHANCEMENTS] Scaling widths by metrics + callstack categorisation/grouping #11

Open
Zabrane opened this issue Jan 22, 2022 · 4 comments

Comments

@Zabrane
Copy link

Zabrane commented Jan 22, 2022

@Stratus3D I'm really enjoying eflambé and i'm wondering if it's possible to add these two innovations from flame_prof:

High scheduler load

The reductions heatmap and corresponding flamegraph below show what callstacks contributed most to schedular load in the example profile. Colour saturation in the heatmap and widths in the flamegraph are scaled to reductions. Scaling flamegraph widths by metrics measured during sampling is the first of flame_prof's two main innovations.

Crucially, in the flamegraph, the callstacks are categorised by the process status of upto 100 processes running that particular code (the example profile auto-selected the top 100.) Such callstack categorisation/grouping is flame_prof's other main innovation. It has proven extremely useful especially in identifying code causing concurrency bottlenecks or memory issues.

Armed with them, eflambé could be the best profiling tool for Erlang.

@Stratus3D
Copy link
Owner

eFlambé doesn't do heatmaps, it does flamegraphs/callstacks. Aside from being simpler, it allows us to use different flamegraph viewers interchangeably and only have to worry about the file formatting differences. I don't plan on ever changing this and don't have the desire to support heatmaps like flame_prof does.

eFlambé only profiles function calls and uses runtime to set the widths in the flamegraph. We can definitely add an option to use runtime, reductions, or other metrics to set the widths in the flamegraphs.

Scaling flamegraph widths by metrics measured during sampling is the first of flame_prof's two main innovations.

Can you provide a list of metrics that frame_prof can use to set the widths in flamegraphs?

I'm thinking of adding a {scale_by, metric()} option to the list of options that can be passed to the eflambe functions where metric() is one of runtime, reductions, etc... Thoughts?

@Zabrane
Copy link
Author

Zabrane commented Jan 27, 2022

@Stratus3D ok for heatmaps.
Regarding metrics (random order):

  1. reductions
  2. garbage collection (heap block sizes, etc)
  3. group callstacks by the memory used by those processes when the calls were being made.
  4. bin_vheap_size
  5. process message queue lengths
  6. message buffer sizes
  7. stack memory

@Stratus3D
Copy link
Owner

@Zabrane does flame_prof implement all these? And if so, can you link to the relevant source code? Thanks!

Some of these metrics seem easier to implement than others, but I haven't started on this yet.

@Stratus3D
Copy link
Owner

Note to self: look at the source code for https://github.com/kzemek/erlang_pmp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants