Provide a way to dump different formats to different files

[MaXal]

We use svg, traces and collapsed formats at the moment and going adopt jfr and experiment with tree.
Unfortunately, we have to split the one file that async produces into different files by hands.

That requires hardcoding heuristics where to split the file and that heuristics became complicated since formats are changed from time to time and we use different sets of formats for different purposes.

It would be amazing if something like svg=C,svg_file=output.svg,traces=1000,traces_file=traces.txt would be possible.

[apangin]

My intention is to disable multiformat output altogether in favor of one-at-a-time dump option.

Once the profile is collected, you may dump it as many times as you want using different dump options. Right now this is possible with stop command, but I think we need to add a dedicate dump command to avoid confusion.

stop,svg,file=output.svg
stop,tree,file=output.html
etc.

[MaXal]

That would be really cool!
If you could add dumpSVG and dumpJFR to the Java API it would be also of a great help.

[apangin]

Yes, I'll rework Java API, too.

Note that JFR is different from other output formats. As long as it dumps every sample on the go rather than aggregated profile at the end, JFR output must be enabled at the beginning of profiling session. It won't be possible to dump JFR afterwards.

[apangin]

Since the latest release, async-profiler no longer supports multiple formats in -o option.
The recommended way to produce output in different formats is to execute stop command multiple times with different output options.

I decided not to add dumpSVG and other dump method in favor of universal execute() method which handles all available commands and options.

The long term intention is to make the agent produce just one comprehensive format, and convert it to any other format offline.

[aalexand]

@apangin (a maintainer of github.com/google/pprof here) Consider using pprof's https://github.com/google/pprof/blob/master/proto/profile.proto format - it's a protobuf but it's fairly easy to generate a binary protobuf without actual protobuf library dependency. Then pprof can be used to convert to any other format (pprof supports a number of formats; it also has an embedded web UI).

[apangin]

@aalexand Thanks. When I do a research, I'll look at this option, too.

[JigarJoshi]

@apangin Thanks for this very useful tool.

Do you plan to support pprof format output?
Can you point me to JFR file specification please?

[apangin]

@JigarJoshi As I've told earlier, there are no plans to add more formats - just one output format with pluggable converters.

Can you point me to JFR file specification please?

I don't know if it even exists. You can look at JMC and HotSpot sources for the reference. BTW, Flight Recorder has Java API for parsing .jfr files.

[rpulle]

@apangin do we capture color coding data in JFR format? I am planning to use JFR parser to extract JSON.

[apangin]

@rpulle Yes, async-profiler outputs a frame type in JFR format:
https://github.com/jvm-profiling-tools/async-profiler/blob/b8493976b635028cb6238865bd96f2b4157e333a/src/flightRecorder.cpp#L83-L91

[apangin]

But the simplest format for parsing is probably collapsed. With -a (annotate) option it will supplement each Java method with _[j] suffix.

[rpulle]

Thank you. I am trying to make a single parser work for both jcmd JFR and async profiler JFR output. The parser is working, just wanted to include colors.

[rpulle]

@apangin actually there is a tool under jdk bin to parse JFR files. bin/jfr. , however async profiler jfr format cannot be parsed using this tool. Can we make JFR format compliant with this tool?

Error:
jfr print: could not read recording at /Users/rpulle/Downloads/Async-profiler-cpu.jfr. File version 0.9. Only Flight Recorder files of version 1.x and 2.x can be read by this JDK.

[apangin]

@rpulle There are different versions of JFR format. async-profiler uses the version compatible with JDK 8, which can be read by JMC 5.5.x (bundled with Oracle JDK 8).

JDK 11 writes a different version of JFR, which is not compatible with JMC 5.5.

Async-profiler deliberately writes an older JFR, that is compatible with JDK 8 and can be read by any version of Mission Control. Unfortunately, jfr tool does not support it, as it does not support JFR files written by JDK 8.