Speed up integration tests #83
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The integration tests can be quite slow to run. This is because we are
actually creating a test program as a string, dumping that string into a
temporary file, shelling out to another instance of Ruby/RSpec, running
the file inside of that process, then performing a test on the output.
We have a lot of integration tests, and running the test suite on Travis
can take around 20 minutes. In my opinion, that's way too long for a gem
that isn't terribly large.
This commit attempts to alleviate this pain point by introducing a
development dependency on Zeus. Zeus is a "code preloader" that was
primarily designed to speed up development within a Rails app, as it can
preload your Rails environment so that your tests will run faster. While
Zeus has largely been superceded by Spring, it is still a valuable tool
and can also be used for non-Rails applications.
How does Zeus work? The Zeus server is a persistent process that
establishes an environment by running Ruby code of your choosing, then
awaits a connection from the Zeus client, which will then run some code
within that environment. Zeus employs forking to keep the environment
code in memory so that it does not need to be re-run. (This is not the
best description, but there is more information on the website if you
want to learn more.)
The way that we use Zeus is that we create an environment where
super_diff
is already required and — for tests that need ActiveRecord— where ActiveRecord is already required too. The integration tests will
then re-use this environment (the code that's run in this environment is
specified in the test itself).
After all is said and done, this ends up speeding up all integration
tests by a little over 2x.
How do you get this goodness? I've added a
bin/start-dev
script whichwill run the persistent Zeus server by way of
zeus start
. You willwant to run this command in a separate tab. After this, you can then run
bin/rspec spec/integration/...
and you should see the speed bump.