Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define test case sizes #5

Open
jmmv opened this issue Feb 22, 2014 · 1 comment
Labels

Comments

@jmmv
Copy link
Owner

@jmmv jmmv commented Feb 22, 2014

From jmmv@google.com on December 09, 2010 04:48:18

Specifying the test case timeouts is fragile: a test case may run faster or slower depending on the platform, so hardcoding an upper bound per test case is generally a bad idea.

The way we should be doing this is by specifying the test case size declaratively: i.e. allow a test case to define itself as "small", "large" or whatever. Then, the runtime engine can apply different deadlines for every test case size (e.g. 1 minute for small tests, 1 hour for large tests, etc.). We need to have a hard deadline for all tests even if they are set to be of the largest possible type.

Once we have this, the deadlines per size can be either automatically tuned depending on the speed of the running platform and/or we can allow the end user to specify them as part of the configuration of Kyua.

Original issue: http://code.google.com/p/kyua/issues/detail?id=5

@roman-neuhauser

This comment has been minimized.

Copy link

@roman-neuhauser roman-neuhauser commented Dec 7, 2014

a test that stresses I/O and a test that stresses the CPU have independent "speed" values.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.