This is another attempt at what was started in #232.
We already use Appraisal for dependency regression tests. The .travis.yml file had a gemfiles declaration that competes with Appraisal. Combining the two caused Travis to blow up because Travis tries to run the suite against a particular (generated) gemfile, but the default rake task cleans that same gem file.
I'll pull this in once the Travis build goes green for this PR.
Removed gemfiles from .travis.yml
We already use Appraisal for regression tests. Combining that with
Travis-CI's gemfile management blows up.
Even though this turns the build green, I think I'd rather see Travis-CI manage the gemfiles instead of Appraisal. That would make it easier to see the edge case permutations that fail from within Travis. Maybe we can have Appraisal still generate the gemfiles up front, but I'm now actually skeptical about how much Appraisal actually buys us vs just managing the gemfiles manually. It'd be easy enough to write a small rake task that bundle installs and runs rake for each gemfile if there's a desire to run all the regressions in a local development environment at once.
Just thinking out loud at the moment. Glad to start a discussion.
I agree. The Appraisal gem reminds me of the trend among Rubyists (circa 2009) to use Jeweler to manage gemspecs. I don't need a DSL for a DSL, thank you very much. I think most Rubyists are already comfortable editing Gemfiles vs. having to learn about the appraise method.
In this case, Appraisal seems like it's causing more pain than just managing the three Gemfiles manually. I'd vote 👎 against using it in this project.
We're using Appraisal and Travis-CI and doesn't run into any of the issue. You just need to specify the script which you want Travis to run. See https://github.com/thoughtbot/paperclip/blob/master/.travis.yml for an example.
However, if you don't set that, and I think that's the case why 203447e got merged, then it Travis would run default test task, will run the test against all the versions of rails, which I'm sure that's not the way you want (and why you thought it 'conflicted').
The problem that Appraisal is trying to solve is to allow you to have multiple Gemfiles while not having to maintaining multiple gemfiles. We know you can write and maintain those gemfiles yourself, but wouldn't it be better if you can just specify the variations in one place, and then it generates those gemfiles for you to run on Travis?
We recently made this change in Clearance's Travis + Appraisal setup:
I'm looking forward to our next change, where we test against Rails 4 and 3.2.13 for Ruby 2.0, changing one line in Appraisals declaratively, ignoring generated gemfiles, and letting Appraisal + Travis do the right thing for all the combinations of Ruby and Rails.
@croaky one thing I could see as a downside from doing that would be that the tests aren't running in parallel anymore. If one of the test break in, say, Rails 3.0, then the test for Rails 3.2 will not gen run (because the test process would return $? => 1 and halt the test chain)
Maybe @jferris can weight into this.
@sikachu Dang. That's a good point.