Implement parallel execution of tests #2
Comments
|
From jmmv@google.com on September 14, 2012 13:27:45 http://code.google.com/p/kyua/wiki/TestersDesign contains a prerequisite before this can happen. |
|
the link from previous comment is effectively 404: the page just says Project "kyua" has moved to another location on the Internet. |
|
It doesn't matter. You can find the document in the GitHub wiki, but me saying that such work was a prerequisite is bogus. In fact, the whole "testers" make this harder than it should and Kyua slower than it could, so I'm actually unwinding that. Regardless, I'm making quick progress on parallelizing Kyua and you can see some of it in the "executor" branch. I'm hoping to have something more stable by the end of this year and possibly release soon in 2015 Q1. |
Merge a variety of supporting (but generic!) changes needed by the executor work, which in turn exists to implement parallel execution of tests as requested in #2. (I've kept these in a private branch for a long time already and the size of the branch is becoming unwieldy. Time to shorten it.)
Say hello to the new parallel execution engine for test cases! This merge brings in the 'executor' work that has taken place over the last 3-4 months. (The history in the branch does not give justice to the real commit activity because I have intensively abused the 'git rebase -i' functionality to build the executor code as a set of well-defined changes. You may notice that, even though the change descriptions build a nice "story", the dates in them make little sense. It's time to stop doing this as explained below.) At this point, only the 'test' command has been converted to use the executor module. In particular, the process to obtain the list of test cases has not yet been integrated into the new executor and thus still relies on the external testers. Also, there is a known regression: the cleanup routine of ATF test cases is not properly handled in all cases. This is properly documented in NEWS and will be addressed later. The new 'executor' code is pure C++ and does not rely on the C-based testers any longer. In other words: the handling of the various test program interfaces has been implemented in C++ again. However, as mentioned above, the code still uses the old testers to perform the 'list' and 'debug' operations on test programs so the C-based testers cannot yet be removed. (Hint: they will at some point.) In other words: this big reshuffling of code is not complete in the sense that the current implementation is not anywhere close to how I would like things to be... but it is a very good start. The reason I'm merging the branch now even with the known regressions and with the acknowledgement that there is still a bunch of integration work to do, is because the code is pretty much fully functional: users can already take advantage of parallel test runs. Lastly, merging this will make things much easier (for me) to do further improvements and fixes as new commits instead of edits to existing commits. Addresses issue #2.
|
The merge of the executor branch has addressed this for the most part. There are still many cleanups to be done in this regard to make the new implementation a shippable product (will file bugs for them), but I consider this feature request done considering that |
This change brings in fixes for... - kyua report-html not properly escaping stdout/stderr: (freebsd/kyua#6). - gettimeofday(2) not necessarily being monotonic, but kyua assuming that it is: (freebsd/kyua#2). - Update configuration to work with Cirrus CI in lieu of Travis CI.
From jmmv@google.com on November 27, 2010 13:33:36
Kyua currently runs tests sequentially, which is an obvious performance problem in today's systems. The scope of this ticket is to make Kyua execute tests in parallel.
This would be trivial if only tests did not have side effects: there are some tests that, e.g. load/unload kernel modules, and running these in parallel can cause problems (if they touch the same module, for example). How do we specify conflicts among tests? In the suite-specific configuration file?
Also, what is the granularity for parallelism? Test programs or test cases? I feel test cases are the correct level (because some of them take a long time to execute), but this needs some more thought.
Original issue: http://code.google.com/p/kyua/issues/detail?id=2
The text was updated successfully, but these errors were encountered: