Delay benchmark runs if the machine is active

[kayceesrk]

If the machine running the benchmarks is actively being used, there's no point running the benchmarks as all the results would be sabotaged. Consider implementing something similar to this to delay benchmark start.

[moazzammoriani]

@Sudha247 may I work on this?

[Sudha247]

Sure, you can go ahead and work on this. Feel free to post here if you have any questions.

[moazzammoriani]

From what I understand, what @kayceesrk means is that if a benchmark is being run on a machine that does not have all / most of its resources dedicated to the program being benchmarked then is not a fair benchmark?

[kayceesrk]

Not exactly. What the linked snippet (https://github.com/kayceesrk/ocamlbench-scripts/blob/master/run-bench.sh#L211-L227) in the original message tests for is the loadavg in the past 1 min on the machine. If it is more than 0.6, then the script delays the start of the benchmarking by 5 minutes and tries again. Does that help?

[moazzammoriani]

I think. Why specifically something like 0.6 though if you don't mind if I ask? Just because it'd be a good heuristic that would increase the likelihood that the machine gets enough resources to run the benchmarks?

[kayceesrk]

Why specifically something like 0.6 though if you don't mind if I ask?

Trial and error. 0.6 just seemed to do the trick.