AP-Agent

Async Profiler Agent is a minimal Java agent that allows you to proxy to Async Profiler via a minimal REST API, making it easy to profile your applications. Simply add it to the start of the JVM and as it uses the AP-Loader, there is no need for Async Profiler up front.

Usage

Download the latest version.

To use the AP-Agent, simply add it to the JVM startup. The agent exposes a REST API for profiling with the following endpoint: http://localhost:8080/profiler/profile.

java -javaagent:/path/to/ap-agent.jar -jar /path/to/my-awesome-app.jar

The endpoint accepts the following parameters:

• event: The type of event to profile (e.g. cpu, itimer, wall)
• output: The desired output format (e.g. flamegraph, hotcold, jfr, pprof, collapsed, fp)
• params: Additional parameters to pass to the flame graph (e.g. simple, title=My Title, threads, reverse)
• duration: The length of time to profile for (in seconds)

Flame Graph

For example, to profile CPU usage for 30 seconds and output the results in Flamegraph format, the following API call would be used: http://localhost:8080/profiler/profile?event=cpu&output=flame&duration=30

Hot/Cold Flame Graph

This type of visualization combines both on-CPU and off-CPU flame graphs. This visualization provides a comprehensive view of the performance data by showing all thread time in one graph and allowing direct comparisons between on-CPU and off-CPU code path durations.

For example, the following API call would be used: http://localhost:8080/profiler/profile?event=cpu&output=hotcold&duration=30

Flame Graph from Collapsed Stack Traces

The collapsed stack trace format is a collection of call stacks, where each line represents a semicolon-separated list of frames followed by a counter. The frames represent the function calls in the stack and the counter indicates how many times that particular stack has been executed.

The format is as follows:

main;run;doSomething;processData;readFile;open;readBytes:5
main;run;doSomething;processData;readFile;open;readBytes:3
main;run;doSomething;processData;readFile;open;readBytes:2
main;run;doSomething;processData;readFile;close:1
main;run;doSomething;processData;writeFile;open;writeBytes:4
main;run;doSomething;processData;writeFile;close:1

To generate a flame graph from the collapsed stack trace format, and share it easily using flamegraph.com, you can use the following command:

curl http://localhost:8080/profiler/profile?event=cpu&output=collapsed&duration=30 | curl --data-binary @- https://flamegraph.com | jq -r '."url"' 
...
...
https://flamegraph.com/share/4672162e-a978-11ed-aa32-fa99570776b6

Finally, you can open the URL in your browser to view the flame graph.

Continuous Profiling a la Bash

We can create a simple bash script to continuously profile our application and output the results to a file.

#!/bin/bash

event=${1:-itimer}
profiling_duration=${2:-30}
results_folder=${3:-profiling_results}

mkdir -p $results_folder

while true; do
    timestamp=$(date +%Y-%m-%d_%H-%M-%S)
    output_file="${event}_profile_$timestamp.html"
    start_time=$(date +%s)
    curl -s "http://localhost:8080/profiler/profile?event=$event&output=flame&duration=$profiling_duration" -o "$results_folder/$output_file"
    end_time=$(date +%s)
    duration=$((end_time - start_time))
    echo "Profile saved to $results_folder/$output_file at $(date) took $duration seconds."
done

Running the script with the cpu event and 60 second duration, we can see the results in the profiling_results folder.

./loop.sh cpu 60 profiling_results

Profile saved to profiling_results/cpu_profile_2023-01-24_16-16-24.html at 04:17:24 took 60 seconds.
Profile saved to profiling_results/cpu_profile_2023-01-24_16-16-24.html at 04:18:24 took 60 seconds.

Firefox Profiler (experimental)

1. Examples
2. Profiling results

Examples

1. Basic example with curl
2. Example using jfrtofp-server
3. Example using the loop.sh script

Basic example with curl

1. Execute the profiler for the cpu event, fp (Firefox Profiler) output, a 60 seconds duration and write the response to profiling_results/firefox-profiler-example.json.gz

curl -s "http://localhost:8080/profiler/profile?event=cpu&output=fp&duration=60" -o profiling_results/firefox-profiler-example.json.gz

2. Visit the Firefox Profiler page

3. Load the output file from step 1, and you'll see the profiling result

Example using jfrtofp-server

1. Execute the profiler for the cpu event, fp (Firefox Profiler) output, a 60 seconds duration and write the response to profiling_results/firefox-profiler-example.json.gz

curl -s "http://localhost:8080/profiler/profile?event=cpu&output=fp&duration=60" -o profiling_results/firefox-profiler-example.json.gz

2. Start the jfrtofp-server, you can follow the steps from the README, with the output file from step 1 as an argument

java -jar jfrtofp-server-all.jar profiling_results/firefox-profiler-example.json.gz

3. The jfrtofp-server will log a message like Navigate to http://localhost:55287/from-url/http%3A%2F%2Flocalhost%3A55287%2Ffiles%firefox-profiler-example.json.gz to launch the profiler view

4. Just click that link, and you will see the profiling result in the Firefox Profiler page

Example using the loop.sh script

1. Continuously profile the application for the cpu event, fp (Firefox Profiler) output, a 60 seconds duration and write the execution results to profiling_results/ folder

./loop.sh cpu 60 profiling_results fp

Profile saved to profiling_results/cpu_profile_2023-01-24_16-16-24.json.gz at 04:17:24 took 60 seconds.
Profile saved to profiling_results/cpu_profile_2023-01-24_16-16-24.json.gz at 04:18:24 took 60 seconds.

2. Visit the Firefox Profiler page
3. Load the output file from step 1, and you'll see the profiling result

Profiling results

Call tree

Flame graph

Go Mode

The agent also supports a GO(lang) mode, which exposes the /debug/pprof/profile endpoint. This is where we can use the go pprof tools.

java -Dap-agent.handler.go-mode=true -javaagent:/path/to/ap-agent.jar -jar /path/to/my-awesome-app.jar

go tool pprof -http :8000 http://localhost:8080/debug/pprof/profile?seconds=30  

Additional Endpoints

In addition, the AP-Agent also supports two additional endpoints:

• /debug/pprof/block: Returns a profiling report of contended locks that are blocking on synchronization primitives. This endpoint can help identify where resources are being locked and where contention is occurring.
• /debug/pprof/allocs: Returns a profiling report of memory allocations performed by the application. This endpoint can help identify where memory is being allocated and what kind of objects are consuming the most memory.

Example using pprof.me

One way to analyze the profiling results generated by the AP-Agent is to use pprof.me. It is a free online tool that allows you to upload profiling data and visualize it, without having to install any additional tools.

curl -s http://localhost:8080/debug/pprof/allocs > allocs.pb.gz
pprofme upload -d "java allocs" allocs.pb.gz
firefox | chrome https://pprof.me/a25a2a9

Can I use the ap-agent as a library?

Yes, you can use the ap-agent as library, just add the following dependency to your project:

<dependency>
    <groupId>io.github.dpsoft</groupId>
    <artifactId>ap-agent</artifactId>
    <version>0.1.3</version>
</dependency>

and then, you can use the API as follows(spring-boot controller example):

@RestController
public class PPROFController {
    private final static Logger log = LoggerFactory.getLogger(PPROFController.class);

    private final AsyncProfiler asyncProfiler = AsyncProfilerLoader.loadOrNull();

    @GetMapping(value = {"/debug/pprof/profile", "/debug/pprof/block", "/debug/pprof/allocs"})
    @ResponseBody
    public void profile(@RequestParam Map<String,String> queryParams, HttpServletRequest request, HttpServletResponse response)  {
        final var operation = Functions.lastSegment(request.getServletPath());
        final var command = Command.from(operation, queryParams);

        ProfilerExecutor
                .with(asyncProfiler, command)
                .run()
                .onSuccess(result -> result.pipeTo(response::getOutputStream))
                .onFailure(cause -> log.error("It has not been possible to execute the profiler command.", cause))
                .andFinallyTry(response::flushBuffer);
    }
}

TODO

• Add support for Context ID

License

This code base is available under the Apache License, version 2.