-
Notifications
You must be signed in to change notification settings - Fork 205
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add fullstack test for scalability #2777
Conversation
Martchus
commented
Feb 26, 2020
- Fakes isotovideo. Fullstack means here that the worker, scheduler, web UI and web socket are started.
- Spawns a configurable number of workers instances and jobs to test production-like job assignments.
- Allows to reproduce https://progress.opensuse.org/issues/62984
eb277d5
to
c3dd287
Compare
Actually I recommend to rename from "fullstack" to something else as it's not really the full stack anymore with fully mocked isotovideo, right? Maybe "05-worker-scalability.t"? |
695b6b9
to
bfa799c
Compare
bfa799c
to
4e00dae
Compare
4e00dae
to
e76d7a3
Compare
Codecov Report
@@ Coverage Diff @@
## master #2777 +/- ##
==========================================
+ Coverage 92.22% 92.23% +0.01%
==========================================
Files 190 190
Lines 11829 11829
==========================================
+ Hits 10909 10911 +2
+ Misses 920 918 -2
Continue to review full report at Codecov.
|
fd254a1
to
3d2dab9
Compare
# similar to other fullstack tests | ||
my $worker_count = $ENV{SCALABILITY_FULLSTACK_WORKER_COUNT}; | ||
my $job_count = $ENV{SCALABILITY_FULLSTACK_JOB_COUNT} // $worker_count; | ||
plan skip_all => 'set e.g. SCALABILITY_FULLSTACK_WORKER_COUNT=n to run the scalability test for n workers' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shouldn't we set a sensible default for our automatic tests? I don't see you setting values anywhere else, e.g. in circleci.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, so far it is disabled in the CI. That is because the scheduling itself isn't stable enough for this test to be useful to run within the CI yet.
* Never mark a module as running on the final upload to avoid modules being considered 'running' when the test has already finished * Verify whether the result file is actually containing a hash
* Fakes isotovideo. Fullstack means here that the worker, scheduler, web UI and web socket are started. * Spawns a configurable number of workers instances and jobs to test production-like job assignments. * Allows to reproduce https://progress.opensuse.org/issues/62984
3d2dab9
to
4cff1d6
Compare
* Avoid magic number * Be consistent with the clients used for internal communication
Otherwise the test sometimes runs into the error "Can't create listen socket: Address already in use ...".
* Bail out early if the job assignment went completely wrong * Improve coding style
4cff1d6
to
f2160b6
Compare