Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Suggestion to run test suite in parallel #2878

Open
ZyX-I opened this issue May 5, 2018 · 0 comments
Open

Suggestion to run test suite in parallel #2878

ZyX-I opened this issue May 5, 2018 · 0 comments

Comments

@ZyX-I
Copy link

ZyX-I commented May 5, 2018

I recently came to making what Neovim has left of Vim test suite finally run in parallel and it has shown an amazing decrease of time to run the test suite: time dropped from fifty seconds to twelve. What was done is basically

  1. Making Neovim run all tests in Neovim :terminal in headless mode. This part dropped “wait N seconds to avoid that xterm title is messed up” as well as checking for terminal size nonsense (nonsense because tests should not really affect or be affected by that), and especially stopped flooding travis CI output. For debugging, if test did fail it still does dump everything Neovim happened to print to the terminal, but if test did not fail then no need to bother. It also allowed to make tests time out: Neovim acting asynchronously with :terminal job can act as a supervisor. Vim can do just as good job now, even though it has no headless mode.

    Without :terminal it is still possible to do the same thing with screen, though with Vim allowing to both compile without some features and on machines without screen all that has to be optional.

    Note that I did not measure time before this change, fifty seconds was measured without the delay. That is because I was more interested in compressing travis output and not test performance, and locally one of the tests utterly failed (goes infinite loop) because &path contains /usr/include on linux by default and nodejs has a symlink /usr/include/node/deps/uv/include -> ../... Not sure whether Vim has the problem now, but it did have it in Test_expand_star_star.

  2. Since I did not look forward to refactoring all tests to actually be sane and use damn test-specific file names I had to copy all test files (all files in testdir) to build/oldtests/{test_name}, once for each test.

  3. Previous step alone allowed dropping times from fifty seconds to half of that when running parallel, but non-parallel build time got doubled (fifty to minute and fourty). To get that to a sane level (minute and a few seconds) and also get twelve seconds I reported earlier I replaced copying with hardlinking, though still copying directories for obvious reasons.

I would suggest somebody interested to do the same thing for Vim, probably optional and using screen. Tests running four times faster on developer machine would be a good thing for development (for the record I have six-cores AMD FX processor). And also if somebody does refactor tests so that they may be run in parallel without copying files around running them in parallel should be as good as dividing time to run test suite by number of cores.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@ZyX-I and others