Skip to content

Speed up integration tests #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 17, 2020
Merged

Speed up integration tests #83

merged 3 commits into from
May 17, 2020

Conversation

mcmire
Copy link
Collaborator

@mcmire mcmire commented May 17, 2020

The integration tests can be quite slow to run. This is because we are
actually creating a test program as a string, dumping that string into a
temporary file, shelling out to another instance of Ruby/RSpec, running
the file inside of that process, then performing a test on the output.
We have a lot of integration tests, and running the test suite on Travis
can take around 20 minutes. In my opinion, that's way too long for a gem
that isn't terribly large.

This commit attempts to alleviate this pain point by introducing a
development dependency on Zeus. Zeus is a "code preloader" that was
primarily designed to speed up development within a Rails app, as it can
preload your Rails environment so that your tests will run faster. While
Zeus has largely been superceded by Spring, it is still a valuable tool
and can also be used for non-Rails applications.

How does Zeus work? The Zeus server is a persistent process that
establishes an environment by running Ruby code of your choosing, then
awaits a connection from the Zeus client, which will then run some code
within that environment. Zeus employs forking to keep the environment
code in memory so that it does not need to be re-run. (This is not the
best description, but there is more information on the website if you
want to learn more.)

The way that we use Zeus is that we create an environment where
super_diff is already required and — for tests that need ActiveRecord
— where ActiveRecord is already required too. The integration tests will
then re-use this environment (the code that's run in this environment is
specified in the test itself).

After all is said and done, this ends up speeding up all integration
tests by a little over 2x.

How do you get this goodness? I've added a bin/start-dev script which
will run the persistent Zeus server by way of zeus start. You will
want to run this command in a separate tab. After this, you can then run
bin/rspec spec/integration/... and you should see the speed bump.

@mcmire mcmire force-pushed the speed-up-integration-tests branch from 820c6d7 to 555a21f Compare May 17, 2020 06:35
@mcmire mcmire force-pushed the speed-up-integration-tests branch 2 times, most recently from 2979a9f to cdb998f Compare May 17, 2020 09:56
The integration tests can be quite slow to run. This is because we are
actually creating a test program as a string, dumping that string into a
temporary file, shelling out to another instance of Ruby/RSpec, running
the file inside of that process, then performing a test on the output.
We have a lot of integration tests, and running the test suite on Travis
can take around 20 minutes. In my opinion, that's way too long for a gem
that isn't terribly large.

This commit attempts to alleviate this pain point by introducing a
development dependency on Zeus. [Zeus] is a "code preloader" that was
primarily designed to speed up development within a Rails app, as it can
preload your Rails environment so that your tests will run faster. While
Zeus has largely been superceded by Spring, it is still a valuable tool
and can also be used for non-Rails applications.

How does Zeus work? The Zeus server is a persistent process that
establishes an environment by running Ruby code of your choosing, then
awaits a connection from the Zeus client, which will then run some code
within that environment. Zeus employs forking to keep the environment
code in memory so that it does not need to be re-run. (This is not the
best description, but there is more information on the website if you
want to learn more.)

The way that we use Zeus is that we create an environment where
`super_diff` is already required and — for tests that need ActiveRecord
— where ActiveRecord is already required too. The integration tests will
then re-use this environment (the code that's run in this environment is
specified in the test itself).

After all is said and done, this ends up speeding up all integration
tests by a little over 2x.

How do you get this goodness? I've added a `bin/start-dev` script which
will run the persistent Zeus server by way of `zeus start`. You will
want to run this command in a separate tab. After this, you can then run
`bin/rspec spec/integration/...` and you should see the speed bump.

[Zeus]: https://github.com/burke/zeus
@mcmire mcmire force-pushed the speed-up-integration-tests branch from cdb998f to dfb5aaa Compare May 17, 2020 10:02
@mcmire mcmire merged commit dfb5aaa into master May 17, 2020
@mcmire mcmire deleted the speed-up-integration-tests branch May 17, 2020 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant