openstack: throttle job scheduling by default #618
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The test failed because it looked for upload= instead of clone= Signed-off-by: Loic Dachary <ldachary@dachary.org>
ghost
assigned 
| @@ -618,6 +624,9 @@ def schedule_suite(job_config, | |||
| verbose=verbose, | |||
| log_prefix=log_prefix, | |||
| ) | |||
| if throttle: | |||
| log.info("pause between jobs : --throttle " + str(throttle)) | |||
| sleep(int(throttle)) | |||
There was a problem hiding this comment.
This should probably no-op if --dry-run is passed
When scheduling, wait SLEEP seconds between jobs. Useful to avoid bursts that may be too hard on the underlying infrastructure or exceed OpenStack API limits (server creation per minute for instance). http://tracker.ceph.com/issues/12977 Refs: #12977 Signed-off-by: Loic Dachary <loic@dachary.org>
When running a suite with 50 jobs, it will schedule about 100 instance creation within less than a minute. It is likely to exceed the API quotas of the OpenStack provider (number of instance creation per minute) and lead to instance creation failures. Set the teuthology-suite to be called with --throttle 15 by default, that is about 60 / 15 = 4 * 2 = ~8 server creation per minute. http://tracker.ceph.com/issues/12977 Fixes: #12977 Signed-off-by: Loic Dachary <loic@dachary.org>
|
@zmc updated to not wait if dry_run, thanks for catching this :-) |
|
Refer to this link for build results (access rights to CI server needed): |
zmc
added a commit
that referenced
this pull request
Sep 8, 2015
openstack: throttle job scheduling by default
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
http://tracker.ceph.com/issues/12977