[llvm-exegesis] Add support for MCWriteProcResEntry::StartAtCycle

[RKSimon]

The ability to control resource intervals is currently being discussed:

https://reviews.llvm.org/D150310
https://reviews.llvm.org/D150311
https://reviews.llvm.org/D150312

We will need to add support to llvm-exegesis, initially just to analysis - a later task will be to investigate ways to help determine more accurate resource intervals.

CC @boomanaiden154

[llvmbot]

@llvm/issue-subscribers-tools-llvm-exegesis

[boomanaiden154]

Thanks for the ping!

I'm planning on working on getting this implemented within llvm-exegesis. Hoping I can get patches up soon after the resource interval patches land, at least for the analysis section (after I've gone through how llvm-exegesis does analysis more thoroughly because I've so far only focused on latency benchmarking).

I'm planning on doing some experimentation for automatic benchmarking to find these values, but no guarantees there if experimentation doesn't pan out (although I think we should be able to come up with something).

Looking at uops.info data, they give port usage data (from which we can infer resource/functional unit usage), but it doesn't seem like they collect any data on the dependencies/actual execution on those ports relative to the rest of the instruction, so it might even be hard to utilize their methodology.

Could definitely be missing something here and look forward to further discussion!

(Also, do you have any specific "high value" instructions that you'd want initial efforts to focus on?)

[RKSimon]

On x86, being able to use this for the load-folded vector instructions would be very useful, we currently have a mess of read-advance controls.

Horizontal operations (PHADD/PHSUB/HADDPS etc.) could use this as well

[boomanaiden154]

I've done some preliminary implementation with the kernel mode implementation of nanobench, and it seems like automatically gathering this data might be more difficult than I thought. Looking at the AVX VHSUBPS instruction with the operands being YMM,YMM,M256 (so, load-folded), I'm getting the following cycle-by-cycle graph:

Quite a few of the elements make sense. Uops are dispatched to port 2/3 (the line exactly overlaps in the graph, which is a little bit weird) for the load first, and then later on uops get dispatched to ports 1 and 5 (exact same line in the graph, not sure where the fifth load went). Dispatches on port 1 also seem to coincide with loads finalizing (seen by the L1 cache hits counter). Cycles between port 1 and port 5 dispatch also makes sense, being eight cycles, about the same as a M256, YMM load.

So it definitely seems theoretically possible to automatically capture this data, but how practical it is to process this data into something machine-usable I'm not so sure. The data isn't really noisy (no run-to-run variation on this instruction/setup), but definitely has some weird artifacts (not really sure what to make of some of the dropping in value as cycles increase, definitely need to do some more investigation there and see what is causing that artifact). Seems like with some careful engineering it might be possible to do so though if enough people care about it and we can get the SE pieces in place (especially a kernel module, which is required for this type of cycle by cycle analysis).