Framework for Performance Benchmarking in Analysis

[yuxuanzhuang]

Is your feature request related to a problem?

As part of parallelizing benchmarking work (https://github.com/MDAnalysis/parallelization-benchmarks), we would like to implement a code-level timing/performance framework.

Describe the solution you'd like

• Use time.perf_counter to track the time spent in each component of an analysis. The performance report should be parallel-aware, so that time spent in individual computational groups can be extracted and analyzed.
• Add an enable_timing flag to AnalysisBase.run() so that performance data can be accessed, for example via AnalysisBase.performance.

Describe alternatives you've considered

• Using external tools, such as line_profiler.

Additional context

• In the future, we may want to track not only execution time but also memory consumption, job dispatching, and other related metrics.
• There is an existing timeit-style implementation in PMDA (https://github.com/MDAnalysis/pmda/blob/master/pmda/parallel.py) that could serve as inspiration.

[tylerjereddy]

Obviously, be sure to look at what some other large projects have done on this front. I've seen a few different tools used upstream, but it isn't an area I focus on a lot. The profiling and analysis of free-threaded CPython performance/issues/deadlocks in NumPy is one thing that required fairly fine-grained profiling tools for example.