Merged
Conversation
dtognazzini
force-pushed
the
simplifyTesting
branch
from
58fa04f to
9e246b1
Compare
This commit changes the testing combinations to the following: Unit tests - run against various versions of Capybara and Ruby Integration tests - run against various versions of Capybara and Rails using the most recent version of Ruby supported by the version of Rails used. The many versions of Rails are used to verify the ApplicationRouter logic. This could be simplified by extracting an ae_page_objects-rails gem (#83), reducing the integration test suite to use a single Rails version.
dtognazzini
force-pushed
the
simplifyTesting
branch
from
9e246b1 to
f4b9fe9
Compare
added 3 commits
May 12, 2016 15:30
With these changes: Unit tests run against every version of Capybara _at most once_ and against both the minimum and maximum Ruby version _at least once_. Integration tests run against every version of Rails at least once and every version of Capybara at least once. The version of Ruby used in integration tests depends on the version of Rails used. In the current state, this means that the most recent Rails application (4.2) has all of the Capybara version permutations since the Ruby version supported by the Rails application (2.2.5) is supported by all the Capybara versions.
dtognazzini
force-pushed
the
simplifyTesting
branch
from
19d8e88 to
d7dbfa8
Compare
Contributor
|
👍 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The combinatorics involved with testing this gem are too complicated. These are the current dimensions of the build matrix: Ruby version, test suite (unit or integration), Rails version, Capybara version.
Some exclusions are in place so not every combination is being built. Still, there's way too much going on.
Take a look at a recent build before these changes:
https://travis-ci.org/appfolio/ae_page_objects/builds/129808228
After these changes:
https://travis-ci.org/appfolio/ae_page_objects/builds/129874954
The changes in this PR result in the following:
Ruby versions
Only unit tests are run against various Ruby versions. We aim to only test against the oldest supported version (currently 1.9.3) and the most recent supported version (currently 2.2.5). We test against ruby-head but don't care about failures.
Capybara versions
Unit tests run against all supported versions of Capybara using the most recent Ruby version possible.
Integration tests also run against all supported versions of Capybara. These tests run using a Rails application, using the most recent version of Rails possible that supports the Ruby version that the version of Capybara under test supports. Today, all Capybara integration tests are run in the Rails 4.2 application using Ruby 2.2.5.
Rails integration tests
The
ApplicationRouteris tested against various versions of Rails (currently 3.0 to 4.2). These tests use the most recent Ruby and Capybara versions possible for the version of Rails. These tests will move out of this gem when #83 is addressed.Results
Unit tests run against every version of Capybara at most once and against
both the minimum and maximum Ruby version at least once.
Integration tests run against every version of Rails at least once and every
version of Capybara at least once.