Skip to content

Commit

Permalink
Adding check for large standard deviation
Browse files Browse the repository at this point in the history
  • Loading branch information
wathen committed Nov 6, 2019
1 parent c9c808f commit 1c31a9d
Showing 1 changed file with 6 additions and 0 deletions.
6 changes: 6 additions & 0 deletions fitbenchmarking/fitbenchmark_one_problem.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,12 @@ def benchmark(controller, minimizers, num_runs):
chi_sq = np.nan
status = 'failed'
else:
cv = np.std(runtime_list) / np.mean(runtime_list)
if cv > 0.2:
raise Warning('The ratio of the standard deviation and mean'
' is larger than {}, this may indicate caching'
' is used in the timing results.'.format())

chi_sq = misc.compute_chisq(fitted=controller.results,
actual=controller.data_y,
errors=controller.data_e)
Expand Down

0 comments on commit 1c31a9d

Please sign in to comment.