Integrate mass-benchmarking into the Files API

[jeremyfowers]

As of #112, turnkey is able to benchmark build directories directly via the turnkey cache benchmark BUILD_NAME command.

#112 implements the functionality as a completely new code path. This is a technical debt because now there are 2 ways to do benchmarking (turnkey benchmark being the other), they may produce slightly different results (e.g., turnkey cache benchmark does a better job reporting hardware failures), and any changes to benchmarking policy now need to be repeated in two places.

Ideally, a single code path would be used for benchmarking anything, regardless of whether it is a python script, ONNX file, or build directory. However, at the time of #112 there were blockers that need to be resolved before integration can take place:

1. Configurable verbosity in Analyzer Status #113
2. Change --rebuild to --skip #114

Proposed solution:

1. Merge Directly benchmark build directories #112
2. Resolve the blockers as independent PRs
3. Integrate the logic from Directly benchmark build directories #112 into the mainline benchmarking flow

cc @danielholanda

[jeremyfowers]

Open design question: once we introduce turnkey cache benchmark would we ever deprecate it in favor of turnkey benchmark cache/*?

Once the mass-benchmarking logic is mainlined into the Files API it would be trivial to enable turnkey benchmark cache/*, its just that would be a breaking change pretty soon after introducing turnkey cache benchmark.

[danielholanda]

Potential solution to avoid users taking dependency on the interface:

Print a warning when turnkey cache benchmark is used saying that it is an experimental/beta interface for the feature.

[jeremyfowers]

Potential solution to avoid users taking dependency on the interface:

Print a warning when turnkey cache benchmark is used saying that it is an experimental/beta interface for the feature.

Love this, I'm adding the notice to the turnkey cache benchmark PR now.