Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Fix macOS/Bash 3.2 breakage; eliminate subshells from exec-test, preprocess #210
These changes produce some substantial performance benefits across all platforms, but most especially on Windows. I'm using them in the mainline of my own project as of mbland/go-script-bash#165 (Travis CI results, Coveralls results).
The first commit in this set fixes the three existing tests that failed for Bash 3.2.57(1)-release, the default version for macOS 10.12.3 and other earlier versions of macOS and OS X. This was due to the
Performance impact on Bats's own test suite
The remaining commits methodically eliminate subshells to improve overall test performance, especially those spawned by
Under Bash 3.2.57(1)-release on a MacBook Pro with a 2.9GHz Intel Core i5 CPU and 8GB 1867MHz DDR3 RAM, the timing for the original set of tests (which don't include the four test name-related cases added by one of the later commits in this series) run via
Performance impact on a large Bats test suite
In my https://github.com/mbland/go-script-bash project, I'd already optimized the tests to disable Bats stack trace collection almost as much as possible, bringing the time for the full
Running again at the same commit, but using Bats with these changes included, the suite now runs in:
Performance impact on Windows
The impact is even more dramatic on Windows. On the same MacBook Pro, running Windows 10 under VMware Fusion 8.5.3 with 4GB RAM, and the timing for
Using Bats with these changes included, running the MSYS2-based Bash 4.3.46(2)-release that ships with (what is probably a slightly outdated version of) Git for Windows, the suite now runs in:
Running Bash 4.4.11(2)-release running under Cygwin:
And running Bash 4.3.11(1)-release as part of the Windows Subsystem for Linux (i.e. Ubuntu on Windows 10):
Other versions tested
In addition to the Bash versions mentioned above, I've also tested these changes using the following Bash versions:
All of the go-script-bash tests passed on these other platforms in less than half the time from before. All of the Bats tests passed on every platform with the exception of a few on Alpine Linux; I can perhaps address these failures in the future.
There are other subshells that can likely be eliminated in other parts of Bats, particularly those that happen at startup when
This was referenced
Feb 16, 2017
You may know, how to merge the PR in your own fork?
@Sylvain303 Is there any news on #150 that hasn't been posted to that thread? There hasn't been any response to it since I threw my hat in the ring as a potential maintainer.
@ret2libc I've since done a few more optimizations, posted to my fork as the mbland/bats optimized-20170317 tag. It's effectively "done" in terms of optimizing by eliminating subshells and subprocesses. No improvements as dramatic as in this PR, but still substantial. I've debated posting them to this PR, or opening a new PR, but decided to wait until this one and #150 finally get traction.
Oh, and for anyone who isn't familiar with managing repos with multiple remotes, the Git commands to pull my changes into your repo should be something like:
$ git remote add mbland firstname.lastname@example.org:mbland/bats.git $ git fetch mbland --tags $ git checkout -b optimized optimized-20170317
To see the tag comments with the summaries (whereby
$ git show optimized-20170205 $ git show optimized-20170317
Coming back to your PR, here:
Are changes equivalent?
I'm trying to merge your PR in my internal unittest. I'm using a "python trick like" sourcing detection:
My current bats-preprocess version (with your modifications).
As you changed in
a sub-shell is spawned for:
It would be easier for me to keep source detection feature, than having to rely on some other trick.
Could you also enable issues, on your bats fork, so I can comment there?