Allow users to control the clock implementation

[danielmitterdorfer]

We run async profiler and perf sched record to gather CPU and scheduler stack traces respectively. For async profiler we use -o jfr. We then postprocess and merge both output files into a combined view which allows us to see what code was on CPU and the reason why it went off CPU (and at which point).

A key issue is alignment of timestamps. While we can configure a specific clock source with perf sched record (e.g. perf sched record -k CLOCK_MONOTONIC), the clock source in async profiler is not configurable and dependent on the CPU architecture, which defaults to rdtsc on Intel CPUs and OS::nanotime (which is CLOCK_MONOTONIC in Linux) for all others.

In order to align timestamps of multiple data sources we would like to have control over the clock source in async profiler similar to perf sched record via a command line switch. Just being able to force usage of OS::nanotime would be sufficient, there is no need to implement new clock sources. I did a proof of concept by simply setting TSC_SUPPORTED to false which allowed us to align the timestamps.

I'm happy to work on a PR but I first wanted to gauge whether there is interest for this feature.

[apangin]

There is already a JVM option -XX:-UseFastUnorderedTimeStamps to achieve the desired behavior.
With -XX:-UseFastUnorderedTimeStamps, both JDK Flight Recorder and async-profiler will use CLOCK_MONOTONIC instead of rdtsc.

[danielmitterdorfer]

Oh, I was unaware of that. Thank you for the pointer. I'll give this a go early next week and will report back.

[danielmitterdorfer]

I thought a bit more about -XX:-UseFastUnorderedTimeStamps. It would indeed solve the use case I was describing but only if we had full control over the JVM flags and could restart the application. In our case we want to attach to an already running production JVM running inside a Docker container. Also, if the flag were applied, the change would be "global", in the sense that all profiling sessions are forced to use CLOCK_MONOTONIC, whereas if we were able to control this when invoking async profiler, we'd have more fine-grained control.

Given that additional context, would you be open for me to work on a PR that allows to control the clock from the command line when invoking async profiler or is this a feature that you'd rather not include?

[apangin]

I implemented the feature myself: it's quite simple, but requires changes in multiple places that may not be obvious from the first glance.

Use --clock monotonic option to select CLOCK_MONOTONIC over RDTSC.

Be careful when using the option in jfrsync mode: JVM and async-proifler will use different timestamp sources and therefore may produce unexpected results.

[danielmitterdorfer]

I did a test now of the latest master version and the flag works as expected. Thanks very much for including this!

Be careful when using the option in jfrsync mode: JVM and async-proifler will use different timestamp sources and therefore may produce unexpected results.

Thanks for the hint! In our use case that's fine though as we only capture CPU samples via async profiler.