Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change unit test default timeout to 5 seconds #89

Merged
merged 1 commit into from
Jul 10, 2017

Conversation

kidder
Copy link
Member

@kidder kidder commented Jul 9, 2017

No description provided.

@nilsdeppe
Copy link
Member

If this works we can have CMake check an env variable to see if things are running on Travis. We probably also want to have the limit as minimal as possible, and maybe ctest or something else can even print out ones that take longer. Thoughts?

@codecov-io
Copy link

codecov-io commented Jul 9, 2017

Codecov Report

Merging #89 into develop will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           develop      #89   +/-   ##
========================================
  Coverage    99.18%   99.18%           
========================================
  Files           57       57           
  Lines         1834     1834           
========================================
  Hits          1819     1819           
  Misses          15       15

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e73442e...0e40d7e. Read the comment docs.

@nilsdeppe
Copy link
Member

So the longest was 2.81 seconds. I wonder if 4 seconds would be enough. @fmahebert would you mind testing that on your machine? I think as a general rule shorter tests are still better and they should finish locally in under 2 seconds in general. The other reason for this is that running tests on KNL takes about 2x longer because the processors are 2x slower.

Random tests are timing out on Travis in the clang debug coverage build,
usually with a single test somewhere between the 35th and 45th test.
@kidder kidder changed the title Experimental bump of timeout to 60 seconds Change unit test default timeout to 5 seconds Jul 10, 2017
@kidder kidder merged commit db642f6 into sxs-collaboration:develop Jul 10, 2017
@nilsdeppe
Copy link
Member

I object to this change without explicit statement in the dev guide somewhere that the goal should still be under 2 seconds.

@kidder kidder deleted the fix/travis_timeout branch July 10, 2017 16:07
@kidder
Copy link
Member Author

kidder commented Jul 10, 2017

Okay, I will update the dev guide

@fmahebert
Copy link
Contributor

This commit was merged before I had a chance to test on my desktop... so here's a belated report.

I am no longer able to reproduce the tests timing out, with or without this commit. I suspect this could be due to a Linux kernel/system update I made since looking into the timeouts with @nilsdeppe a week ago, but have not been able to confirm this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants