Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Add `rails test` command to run the test suite #9080

Merged
merged 7 commits into from
@sikachu
Collaborator

To run the whole test suite:

$ rails test

To run the test file(s):

$ rails test test/unit/foo_test.rb [test/unit/bar_test.rb ...]

To run the test suite

$ rails test [models,helpers,units,controllers,mailers,...]

For more information, see rails test --help.

This command will eventually replacing rake test:*, and rake test
command will actually invoking rails test instead.


After this got merged, there'll be several things that need to be done:

  • Update guides to cover this :sparkles:new:sparkles: feature.
  • Decide whether we should show a deprecation warning if user run rake test:* (but if you run rake or rake test, there will no deprecation warning.
  • Add a notification for test:prepare event, so that we can drop Rake::Task['test:prepare'].invoke line.
  • Add back uncommitted and recent suites.
  • Add support for -n to specify the test name.
@sikachu
Collaborator

/cc @dhh

@rafaelfranca

Seems good. I think we should update the guides in the scope of this pull request. Unless you think it will need a lot of work.

@sikachu
Collaborator

Yeah, I'm working on the guide right now. Going to submit that as a separate commit, just in case if someone want to merge this in first.

@sikachu
Collaborator

Documentation updated.

Also, add some more stuff to my todo list.

guides/source/testing.md
@@ -755,29 +758,34 @@ end
Rake Tasks for Running your Tests
---------------------------------
-You don't need to set up and run your tests by hand on a test-by-test basis. Rails comes with a number of rake tasks to help in testing. The table below lists all rake tasks that come along in the default Rakefile when you initiate a Rails project.
+You don't need to set up and run your tests by hand on a test-by-test basis. Rails comes with a number of commands to help in testing. The table below lists all commands that come along in the default Rakefile when you initiate a Rails project.
+
+| Tasks | Description |
+| ------------------------------- | ----------- |

Different table indent.

@sikachu Collaborator
sikachu added a note

nice catch. fixed!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
railties/lib/rails/commands/test_runner.rb
((52 lines not shown))
+ puts "-------------------------------------------------------------"
+ end
+ end
+
+ # Create a new +TestRunner+ object with a list of test file paths.
+ def initialize(files)
+ @files = files
+ Rake::Task['test:prepare'].invoke
+ MiniTest::Unit.output = SilentUntilSyncStream.new(MiniTest::Unit.output)
+ end
+
+ # Run the test files by evaluate each of them.
+ def run
+ @files.each do |filename|
+ $0 = filename
+ eval(File.read(filename), nil, filename)

Wouldn't load work?

@sikachu Collaborator
sikachu added a note

Yeah, it does, so I'm going to fix it. I totally forgot about Kernel.load since I was copying the code from rails runner :smile:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@carlosantoniodasilva

Cool stuff @sikachu :+1:

@sikachu
Collaborator

Code updated based on @carlosantoniodasilva's review. Thanks!

@xiejava

good job

@calebthompson

Is this going to be smart enough to run tests/specs for the appropriate test framework for those of us who don't use MiniTest?

@sikachu
Collaborator

@calebthompson no, not for now. We might provide a hook later, but it's not currently in our radar.

Actually, we were thinking about making ./bin/test to run the test (which would be closely to ./bin/rspec) but then it would conflict with /usr/bin/test.

@calebthompson

I feel like this is just going to be cumbersome unless/until rspec and cucumber specifically pick it up. You'll end up with two ways of running tests, and frankly the gems that people prefer to test with are going to win out over Rails conventions here I think.

@calebthompson

With rake, rspec, or cucumber commands being the way people run their tests, over anything rails.

@sikachu
Collaborator

Right. I understand what you mean, and I like that.

However, the goal of this ticket was to fix the slow rake test runner. I think we achieve that. From now on, though, only the sky is the limit. :)

@alindeman

I feel like we do need some way to hook into this. On many projects, I use different frameworks for unit tests, acceptance tests, and JavaScript tests (e.g., rspec/cucumber/jasmine or minitest/cucumber/jasmine). rake test can run them all, and it's a command that everyone expects to run all the tests. If rails test becomes the new standard, but can't be used for the same situations, I expect things to be more confusing for newcomers and experienced folks alike.

@dhh
Owner
@josevalim
Owner

I really don't see the motivation for having rails test running cucumber, rspec and what not. How would it even pick which one to run when I am using both rspec and cucumber? Who uses rake test to run specs today anyway?

Also, given that rspec accepts command line options completely different than rails test would, I just see this bringing more confusion.

@josevalim
Owner

For last, why would you bind your test framework runner to Rails when there isn't a need in the first place? If you guys are having extra free-time, we still have ~400 issues open that would love some attention. :wink: :wink:

@drbrain

the goal of this ticket was to fix the slow rake test runner

What work was done to make the rake test runner faster? This appears to be a complete replacement that doesn't use or touch rake at all.

Since this is a replacement for existing working (if slow) code I would like to see evidence of why it is pointless to try to make rake or the rails test running tasks faster.

Could it be something I can improve in rake?

@sikachu
Collaborator

@drbrain the original problem was that running test via rake had to load the Rails environment twice. (One for rake's environment, then rake would call ruby -Itest ... which is the second.) We kinda want to eliminate that by just load the test files into the runner; hence the usage of load command. On a rainy day, that could mean the test took twice the load time than running the test directly via some command.

@steveklabnik
Collaborator

@drbrain @sikachu

@myronmarston seems to think it's a '(mis?)-use' of Rake that's causing the problems:

https://twitter.com/myronmarston/status/295296519203065856

https://twitter.com/myronmarston/status/295296827350200320

Myron, how should we be improving this?

@myronmarston

Myron, how should we be improving this?

Let me preface my suggestions with two quick comments:

  • Calling rails' use of rake a "misuse" has more to do with twitter-imposed terseness than with the actual way I would categorize it. I do think rails' use of rake is suboptimal and has room for improvement, but that's more nuanced than fits in a tweet.
  • I haven't worked on a rails app in a long time (I've never used rails 3.1 or 3.2, for example, and barely used rails 3.0). The apps I've been working on are very, very far from rails-style CRUD apps, and, as @dhh has stated elsewhere, the more your app is dissimilar from Basecamp, the worse fit rails is for your application. I'm not sure if/when I'll be working on another rails app, so I'm not going to invest lots of time in an extended discussion here.

With that said, I think there's an optimal way to use rake that's quite different from how rails uses it. Specifically, it leverages the fact that rake, like most ruby programs, has two run-time phases:

  • Phase 1 is "task definition time" -- during this phase, rake is evaluating your Rakefile (which involves loading other files required or loaded by your Rakefile).
  • Phase 2 is "task execution time" -- during this phase, rake is executing the specified task(s).

My preferred way to use rake is to make phase 1 as lightweight and minimal as possible. My rule of thumb is to only require files that define rake tasks during this phase, and to defer all other requires to phase 2. Each task becomes responsible (either through the code directly in the task, or via a prerequisite task) for loading the files and libraries it needs to run. This goes hand-in-hand with the way I use bundler, and fits right in with my preferred approach of explicitly loading dependencies at runtime...but I know that DHH prefers the convenience of not having to manually manage dependencies at runtime. (As usual, this is about trade offs. The particular things I value most differ from the things DHH values most and that's fine).

Rails has, for as long as I know, chosen to load the entire rails environment at task definition time, which essentially causes the environment to be double-booted when running a task that spawns a new process that boots the environment. Changing to my preferred approach would be a large change, but if there's desire to move in that direction, a major release (like rails 4) is the right time to do it. As a point of comparison, here's the kind of speed I get out of rake. This is on a large app that I (and an entire team at @seomoz) have been working on full time for 9+ months:

$ time bin/rake -T > /dev/null
bin/rake -T > /dev/null  0.21s user 0.04s system 98% cpu 0.255 total

That's 255 ms wall clock time, which is barely noticeable.

As for the idea of having a more full-featured test runner...I heartily support such a thing (one of my favorite parts of rspec is the super flexible test runner, complete with the binary that supports a plethora of options), but I wonder if it's better suited as an external gem that can be used by any T::U/minitest project? Is there anything about this that needs to be coupled to rails?

@steveklabnik
Collaborator

Calling rails use of rake a "misuse" has more to do with twitter-imposed terseness than with the actual way I would categorize it. I do think rails' use of rake is suboptimal and has room for improvement, but that's more nuanced than fits in a tweet.

Yes, that's why I wanted to make sure you came here, and why I made sure to copy the tweet and the (mis?) part of it. The ? says to me that you weren't trying to make a super big statement, just Twitter. Since you are on the rspec team, I figured that your insight here is important, along with @alindeman's.

@myronmarston

Yes, that's why I wanted to make sure you came here, and why I made sure to copy the tweet and the (mis?) part of it. The ? says to me that you weren't trying to make a super big statement, just Twitter.

Thanks...it's nice to know my tweet was interpreted in the spirit it was intended :).

@dhh
Owner

Prem, before we apply this, please get -n and fixture loading implemented. I don't think it's a full replacement before that happens.

@sikachu
Collaborator

@dhh -n get added to this feature. I'm not sure what you mean by "fixture loading", since I thought Rails doesn't load fixture by default unless you run rake db:fixtures:load ?

@dhh
Owner
dhh commented
@sikachu
Collaborator

Code update with not loading fixture by default. @dhh can you give it another look?

@dhh
Owner
dhh commented

Looks good to me. I think this is good enough to merge and start using.

I say we deprecate the "rake test" tasks. No reason to have two ways to do this.

Also, we should pull in the test:prepare stuff so it doesn't go through Rake.

railties/CHANGELOG.md
@@ -1,11 +1,40 @@
## Rails 4.0.0 (unreleased) ##
+* Rails now generate a `test/test_helper.rb` file with `fixtures :all` commented out by default,
+ since we don't want to force loading all fixtures for user when a single test is run. However,
+ fixtures are still going to be loaded automatically for test suites.
+
+ To force all fixtures to be create in your database, use `rails test -f` to run your test.
@dalibor
dalibor added a note

%s/create/created/g

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@sikachu
Collaborator

Hmm, I just realized that 0ba5229 breaks backward compatibility. I need to fix that before adding the deprecation warning.

@jonleighton jonleighton referenced this pull request in rails/spring
Closed

Consider renaming the `spring test` command #64

@sikachu
Collaborator

Code updated. Backward compatibility fixed. Now the only thing I need is to use AS::Notification ...

@rwjblue rwjblue referenced this pull request in rails/spring
Closed

Issues with testing gems #83

@sikachu
Collaborator

After I've been trying to convert database preparing step to use AS::Notification, it seems like it's not worth doing it anymore. I think it's fine that we do Rake::Task['db:test:prepare'] for now.

The reason is that all the tasks in https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/databases.rake has to be converted into a new class, with tests, so that it can be called by a new subscriber and rake task without having to duplicate the code.

All it's left now is that @rubys is seeing some failing test case after using rails test. I'm looking into that right now.

Prem Sichanu... and others added some commits
Prem Sichanugrist and Chris Toomey Add `rails test` command to run the test suite
To run the whole test suite:

    $ rails test

To run the test file(s):

    $ rails test test/unit/foo_test.rb [test/unit/bar_test.rb ...]

To run the test suite

    $ rails test [models,helpers,units,controllers,mailers,...]

For more information, see `rails test --help`.

This command will eventually replacing `rake test:*`, and `rake test`
command will actually invoking `rails test` instead.
b4df253
@sikachu sikachu Update testing documentation
* Update test invocation to use `rails test` instead.
* Update all the test command previews (since we're now using MiniTest.)
* Mentioning MiniTest instead of Test::Unit.
* Update list of test suites.
9f75f77
@sikachu sikachu Add support for MiniTest flags in TestRunner
Any flags that got set will be passed through to MiniTest::Unit.runner,
such as `-n`, `-s-, and `-v`.
176b57c
@sikachu sikachu Load fixtures only when running suites, or `-f`
* `rails test -f` will run the test suites with all fixtures loaded
* New application will now generated without `fixtures :all` line
  enabled by default.
1a0c58b
@dalibor dalibor Improve wording for rails test command df85dfa
@sikachu
Collaborator

Ok, this should be ready to go. @dhh, and @rubys can you give it another look?

@rubys

I immediately get a failure with a new project. Reproduction instructions:

$ rails new depot
$ cd depot
$ rails generate scaffold Product title
$ rake db:migrate
$ rails test
You have 1 pending migrations:
  20130309000009 CreateUsers
Run `rake db:migrate` to update your database then try again.

Needless to say, running rake db:migrate again doesn't clear up the problem.

@sikachu
Collaborator

@rubys found it. It seems like I was trying to run db:abort_if_pending_migrations in the test environment, which failed terribly. Can you give it another try?

@rubys

Looks good. I'll have to update all of my tests as the output of the test command changed -- not an issue you need to worry about.

One think I did note is that commands like "rake test:controllers" will inform you that these commands are deprecated, but will not actually run the correct command.

Timings of rake test before:

real    0m11.942s
user    0m10.645s
sys 0m0.648s

Timings of rails test after:

real    0m5.279s
user    0m4.404s
sys 0m0.240s
sikachu added some commits
@sikachu sikachu Update Rake tasks to call `rails test` instead
Also, print out deprecation warning for other rake tasks except
`rake test` and `rake` (default)
b51673f
@sikachu sikachu Make sure that `rails test` load test in test env 3ed41e5
@sikachu
Collaborator

@rubys good catch. Thank you. It was actually running your test, but it was swallowing the output (I was doing backticks instead of exec())

Code updated (again) to fix that issue.

@rubys

I noticed two more things.

What appears to be going on is that I used to be able to run individual tests, but now when I try to do so, the helper methods that provide access to fixture data no longer is available. For example, in my cart_test.rb, I make reference to products(:one). This works from "rails test", but not from "rails test test/models/cart_test.rb".

The other thing I noticed is that "rails test " only supports one filename. Ideally it would support multiple. At a minimum it should warn when additional arguments are passed.

@rubys

I tracked down the problem to sikachu@1a0c58b#L2L9

In the process, I discovered the -f parameter. Either uncommenting that one line or adding the -f parameter makes this work. So the question becomes: why is -f required? And why is that fixtures :all line commented out? Is there ever a case where people will want to uncomment that line?

@sikachu
Collaborator

@rubys see comment above from @dhh. We decided not to load all the fixtures by default if you run single test. I think the intention is that you will want to load only fixtures you want explicitly in the test file.

However, if you run the whole suite, rails test will load the fixtures for you like it used to be.

Regarding the support for only one file name: is that really the case? I think I have a test case to cover running multiple files. Can you re-confirming me on that one?

@rubys

My bad. When I specified two files, I neglected to notice that the second file had zero tests in it.

:shipit:

@rafaelfranca rafaelfranca merged commit 90a9715 into from
@sikachu sikachu deleted the branch
@spastorino

@sikachu why are you using bundle exec? I think there's no need to do so

Collaborator

because I thought I have to use it? haha ..

Sure, I'll send in another pull request. Maybe change it to bin/rails ?

@tenderlove
Owner

Why?

@tenderlove
Owner

Rails loads fixtures by default when you run the rake suite tasks. So "rake test" and "rake test:units" will load fixtures. This should do the same. But don't load fixtures when running one specific file or test within that file. In that case, we should have an option like --load-fixtures/-f that does it.

@dhh why?

This change

  • breaks our usage of autotest
  • prevents us from running one test file via ruby
  • eliminates the possibility of focusing on one failing test
  • eliminates the Rake task hooks we use for environment specific setup

Providing rails test seems fine for people that want to use it, but why do we need to remove these hooks?

Being able to run one test with just plain old ruby is an extremely important use case for me. I would like to know the essence of this feature so that I can add support for my usecase again.

@tenderlove
Owner

Just a couple more thoughts on this. When you do something like this:

$ rails generate scaffold LineItems product:references cart:belongs_to

Rails generates a controller test that depends on fixtures, but that file cannot stand on it's own. It depends on fixtures, but does not load them. Someone writing their own controller test can choose whether or not to load fixtures, but as long as we're generating a controller test that depends on fixtures, shouldn't that file be able to run on it's own?

@dhh
Owner
@rubys

today fixtures are not loaded when you do "ruby -Itest test/unit/some_test.rb"

Actually, they do with Rails 3.2.x and Rails 4.0.0.beta1. And did up to this change, apparently in response to your request: 1a0c58b#L2L9

@dhh
Owner
@sikachu
Collaborator

Wait. Let me double-check. I might interpret what @dhh wanted wrong in 1a0c58b#L2L9

We don't want to create records in the database when you start your test, but we definitely want you to be able to use users(:david) in your test.

@rubys

today fixtures are not loaded

Actually, they do with Rails 3.2.x and Rails 4.0.0.beta1.

That's just loading the fixtures.

Ok. Now I'm thoroughly confused :-)

In any case, the problem that Aaron is seeing, and would like reversed, is as follows:

$ ruby -I lib:test test/controllers/line_items_controller_test.rb 
Run options: --seed 26717

# Running tests:

EEEEEEE

Finished tests in 0.020279s, 345.1847 tests/s, 0.0000 assertions/s.

  1) Error:
LineItemsControllerTest#test_should_create_line_item:
NoMethodError: undefined method `line_items' for #<LineItemsControllerTest:0x007f8d5c520d28>
    test/controllers/line_items_controller_test.rb:5:in `block in <class:LineItemsControllerTest>'
@dhh
Owner
@sikachu
Collaborator

@rubys I think I was the one who got it wrong. Load fixture != put them in the database.

So it seems like we didn't even create records in the database when we ran rake test though. Let me fix it.

Thanks @dhh @rubys and @tenderlove, and I'm sorry that I got it wrong.

@dhh
Owner
@sikachu
Collaborator

I've created #9854 to track the progress and put it on 4.0.0 milestone!

@rubys

I've created #9854 to track the progress and put it on 4.0.0 milestone!

:+1:

@tenderlove
Owner

I don't see how any of these changes break existing code. Could you explain further?

@dhh the rake tasks call exec which kills the existing process. Any code that was executing after the rake tasks will no longer execute. Also, it looks like the test tasks have been changed to run bundle exec .... Booting the app runs bundle setup, so running "bundle exec" just adds more overhead.

The actual problem behind the slow tests is that we load the environment twice. @wangjohn (and team MIT) are working to remove the requirement that we have one app per process. Once this requirement is gone, we will have the ability to switch environments in the same process. That means we can fix the rake tasks to have the same speed as rails test but deprecate nothing and break no existing code.

I would like to revert the changes to the rake tasks. There is no reason to deprecate them, so we should not.

TL;DR we can eventually make the rake tasks as fast as rails test so there is no reason to deprecate them

@dhh
Owner

I guess I don't see why we need two ways to run the same thing. I'm fine that we have a transition period, but at some point we should pick that "rails test" is the way to run tests in Rails, and deprecate/extract the rake tasks that do the same time.

@tenderlove
Owner

I guess I don't see why we need two ways to run the same thing. I'm fine that we have a transition period, but at some point we should pick that "rails test" is the way to run tests in Rails, and deprecate/extract the rake tasks that do the same time.

Why rails test? The rake tasks give people a standard place and way to hook pre / post execution code. Is it just the command line interface you want to change?

@dhh
Owner

I actually don't care much whether it's one or the other. What I care about is:

1) "X test" is super duper fast and can be paired with something like spring to be even faster.
2) That Rails has one way to test things.

I find the argument form for rake test a little awkward, so from a CLI perspective, "rails test" seems nicer. But that's not the major point.

@tenderlove
Owner

@dhh these seem like fine requirements. We can eventually make rake test speed equal rails test speed and the rake tasks should work fine with things like spring. The important thing for me is that we have a place to hook setup / teardown code.

rails test could be implemented in terms of Rake, which would give us the same standard hooks.

Quick proof that rake test does not have to be slow:

$ time bin/rails test
.......

Finished tests in 0.202827s, 34.5122 tests/s, 64.0940 assertions/s.

7 tests, 13 assertions, 0 failures, 0 errors, 0 skips

real    0m2.586s
user    0m2.231s
sys 0m0.325s
$ time RAILS_ENV=test rake aarontest
Run options: --seed 11

# Running tests:

.......

Finished tests in 0.177000s, 39.5480 tests/s, 73.4463 assertions/s.

7 tests, 13 assertions, 0 failures, 0 errors, 0 skips

real    0m1.960s
user    0m1.723s
sys 0m0.224s
$

Here's the definition of aarontest:

task :aarontest do
  require 'minitest/autorun'
  $: << '.'
  $: << 'test'
  Dir['test/**/*_test.rb'].each { |x| require x }
end
@dhh
Owner
@tenderlove

Why load and not request? Do we support loading the same file twice?

Collaborator

I think you mean require?

I was using require before, then someone suggest me to use load instead. I don't think we would want to support loading the same file twice anyway.

@tenderlove
Owner

Alright. I don't feel comfortable shipping the singleton removal in 4.0. I think our deadline is too close.

Instead, it is possible to detect that we are inside a rake test task before the application gets loaded. I've added a diff here which reverts the rake tasks but makes them as fast as rails test. I had to add a hack to rails/all in this commit, but it allows us to switch the environment to test before the app is ever loaded.

If we're OK with that hack, I can modify all the test tasks to no longer shell out. People can have fast tests without using rails test.

Here's output from a test app (using @rubys data):

$ time rake test:controllers
Run options: --seed 10096

# Running tests:

.......

Finished tests in 0.125721s, 55.6788 tests/s, 103.4036 assertions/s.

7 tests, 13 assertions, 0 failures, 0 errors, 0 skips

real    0m2.034s
user    0m1.785s
sys 0m0.234s
$
@dhh
Owner

I'd be OK with a hack if we have a clean path to a solid solution. Then we don't have to divert to "rails test" on the API side. So I say let's roll with your speed up for "rake test" and thank prem for his working that provoked this, even if we don't end up keeping it. The goal all along was fast tests, so that's the big deal!

@sikachu
Collaborator

Yeah, as long as we have a fast test runner, I don't mind if at the end we decide to not including rails test at all.

@tenderlove
Owner

@sikachu thank you for the work you've done on this. I really appreciate the effort you put in! :heart:

@dhh ok. Seems good. I will makeitso.

@alindeman

@tenderlove, et al, is there a way that rspec-rails's spec tasks can be invoked in RAILS_ENV='test' without booting the app twice too?

I think most rspec-rails users use the rspec command instead of rake (which has the same performance benefits as the changes here), but I think it would be nice to bring this speedup to those who do use rake spec.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Mar 9, 2013
  1. @sikachu

    Add `rails test` command to run the test suite

    Prem Sichanugrist and Chris Toomey authored sikachu committed
    To run the whole test suite:
    
        $ rails test
    
    To run the test file(s):
    
        $ rails test test/unit/foo_test.rb [test/unit/bar_test.rb ...]
    
    To run the test suite
    
        $ rails test [models,helpers,units,controllers,mailers,...]
    
    For more information, see `rails test --help`.
    
    This command will eventually replacing `rake test:*`, and `rake test`
    command will actually invoking `rails test` instead.
  2. @sikachu

    Update testing documentation

    sikachu authored
    * Update test invocation to use `rails test` instead.
    * Update all the test command previews (since we're now using MiniTest.)
    * Mentioning MiniTest instead of Test::Unit.
    * Update list of test suites.
  3. @sikachu

    Add support for MiniTest flags in TestRunner

    sikachu authored
    Any flags that got set will be passed through to MiniTest::Unit.runner,
    such as `-n`, `-s-, and `-v`.
  4. @sikachu

    Load fixtures only when running suites, or `-f`

    sikachu authored
    * `rails test -f` will run the test suites with all fixtures loaded
    * New application will now generated without `fixtures :all` line
      enabled by default.
  5. @dalibor @sikachu

    Improve wording for rails test command

    dalibor authored sikachu committed
  6. @sikachu

    Update Rake tasks to call `rails test` instead

    sikachu authored
    Also, print out deprecation warning for other rake tasks except
    `rake test` and `rake` (default)
  7. @sikachu
This page is out of date. Refresh to see the latest.
View
126 guides/source/testing.md
@@ -1,8 +1,7 @@
A Guide to Testing Rails Applications
=====================================
-This guide covers built-in mechanisms offered by Rails to test your
-application.
+This guide covers built-in mechanisms in Rails for testing your application.
After reading this guide, you will know:
@@ -38,11 +37,11 @@ Rails creates a `test` folder for you as soon as you create a Rails project usin
```bash
$ ls -F test
-
-fixtures/ functional/ integration/ test_helper.rb unit/
+controllers/ helpers/ mailers/ test_helper.rb
+fixtures/ integration/ models/
```
-The `unit` directory is meant to hold tests for your models, the `functional` directory is meant to hold tests for your controllers and the `integration` directory is meant to hold tests that involve any number of controllers interacting.
+The `models` directory is meant to hold tests for your models, the `controllers` directory is meant to hold tests for your controllers and the `integration` directory is meant to hold tests that involve any number of controllers interacting.
Fixtures are a way of organizing test data; they reside in the `fixtures` folder.
@@ -140,10 +139,9 @@ The default test stub in `test/models/post_test.rb` looks like this:
require 'test_helper'
class PostTest < ActiveSupport::TestCase
- # Replace this with your real tests.
- test "the truth" do
- assert true
- end
+ # test "the truth" do
+ # assert true
+ # end
end
```
@@ -224,34 +222,30 @@ TIP: You can see all these rake tasks and their descriptions by running `rake --
### Running Tests
-Running a test is as simple as invoking the file containing the test cases through Ruby:
+Running a test is as simple as invoking the file containing the test cases through `rails test` command.
```bash
-$ ruby -Itest test/models/post_test.rb
-
-Loaded suite models/post_test
-Started
+$ rails test test/models/post_test.rb
.
-Finished in 0.023513 seconds.
-1 tests, 1 assertions, 0 failures, 0 errors
-```
+Finished tests in 0.009262s, 107.9680 tests/s, 107.9680 assertions/s.
-This will run all the test methods from the test case. Note that `test_helper.rb` is in the `test` directory, hence this directory needs to be added to the load path using the `-I` switch.
+1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
+```
-You can also run a particular test method from the test case by using the `-n` switch with the `test method name`.
+You can also run a particular test method from the test case by running the test and using `-n` switch with the `test method name`.
```bash
-$ ruby -Itest test/models/post_test.rb -n test_the_truth
-
-Loaded suite models/post_test
-Started
+$ rails test test/models/post_test.rb -n test_the_truth
.
-Finished in 0.023513 seconds.
-1 tests, 1 assertions, 0 failures, 0 errors
+Finished tests in 0.009064s, 110.3266 tests/s, 110.3266 assertions/s.
+
+1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
```
+This will run all test methods from the test case. Note that `test_helper.rb` is in the `test` directory, hence this directory needs to be added to the load path using the `-I` switch.
+
The `.` (dot) above indicates a passing test. When a test fails you see an `F`; when a test throws an error you see an `E` in its place. The last line of the output is the summary.
To see how a test failure is reported, you can add a failing test to the `post_test.rb` test case.
@@ -266,17 +260,16 @@ end
Let us run this newly added test.
```bash
-$ ruby unit/post_test.rb -n test_should_not_save_post_without_title
-Loaded suite -e
-Started
+$ rails test test/models/post_test.rb -n test_should_not_save_post_without_title
F
-Finished in 0.102072 seconds.
+
+Finished tests in 0.044632s, 22.4054 tests/s, 22.4054 assertions/s.
1) Failure:
-test_should_not_save_post_without_title(PostTest) [/test/models/post_test.rb:6]:
-<false> is not true.
+test_should_not_save_post_without_title(PostTest) [test/models/post_test.rb:6]:
+Failed assertion, no message given.
-1 tests, 1 assertions, 1 failures, 0 errors
+1 tests, 1 assertions, 1 failures, 0 errors, 0 skips
```
In the output, `F` denotes a failure. You can see the corresponding trace shown under `1)` along with the name of the failing test. The next few lines contain the stack trace followed by a message which mentions the actual value and the expected value by the assertion. The default assertion messages provide just enough information to help pinpoint the error. To make the assertion failure message more readable, every assertion provides an optional message parameter, as shown here:
@@ -292,9 +285,8 @@ Running this test shows the friendlier assertion message:
```bash
1) Failure:
-test_should_not_save_post_without_title(PostTest) [/test/models/post_test.rb:6]:
-Saved the post without a title.
-<false> is not true.
+test_should_not_save_post_without_title(PostTest) [test/models/post_test.rb:6]:
+Saved the post without a title
```
Now to get this test to pass we can add a model level validation for the _title_ field.
@@ -308,13 +300,12 @@ end
Now the test should pass. Let us verify by running the test again:
```bash
-$ ruby unit/post_test.rb -n test_should_not_save_post_without_title
-Loaded suite unit/post_test
-Started
+$ rails test test/models/post_test.rb -n test_should_not_save_post_without_title
.
-Finished in 0.193608 seconds.
-1 tests, 1 assertions, 0 failures, 0 errors
+Finished tests in 0.047721s, 20.9551 tests/s, 20.9551 assertions/s.
+
+1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
```
Now, if you noticed, we first wrote a test which fails for a desired functionality, then we wrote some code which adds the functionality and finally we ensured that our test passes. This approach to software development is referred to as _Test-Driven Development_ (TDD).
@@ -334,18 +325,17 @@ end
Now you can see even more output in the console from running the tests:
```bash
-$ ruby unit/post_test.rb -n test_should_report_error
-Loaded suite -e
-Started
+$ rails test test/models/post_test.rb -n test_should_report_error
E
-Finished in 0.082603 seconds.
+
+Finished tests in 0.030974s, 32.2851 tests/s, 0.0000 assertions/s.
1) Error:
test_should_report_error(PostTest):
-NameError: undefined local variable or method `some_undefined_variable' for #<PostTest:0x249d354>
- /test/models/post_test.rb:6:in `test_should_report_error'
+NameError: undefined local variable or method `some_undefined_variable' for #<PostTest:0x007fe32e24afe0>
+ test/models/post_test.rb:10:in `block in <class:PostTest>'
-1 tests, 0 assertions, 0 failures, 1 errors
+1 tests, 0 assertions, 0 failures, 1 errors, 0 skips
```
Notice the 'E' in the output. It denotes a test with error.
@@ -642,12 +632,9 @@ Here's what a freshly-generated integration test looks like:
require 'test_helper'
class UserFlowsTest < ActionDispatch::IntegrationTest
- fixtures :all
-
- # Replace this with your real tests.
- test "the truth" do
- assert true
- end
+ # test "the truth" do
+ # assert true
+ # end
end
```
@@ -755,23 +742,28 @@ end
Rake Tasks for Running your Tests
---------------------------------
-You don't need to set up and run your tests by hand on a test-by-test basis. Rails comes with a number of rake tasks to help in testing. The table below lists all rake tasks that come along in the default Rakefile when you initiate a Rails project.
+You don't need to set up and run your tests by hand on a test-by-test basis. Rails comes with a number of commands to help in testing. The table below lists all commands that come along in the default Rakefile when you initiate a Rails project.
+
+| Tasks | Description |
+| ------------------------ | ----------- |
+| `rails test` | Runs all unit, functional and integration tests. You can also simply run `rails test` as Rails will run all the tests by default|
+| `rails test controllers` | Runs all the controller tests from `test/controllers`|
+| `rails test functionals` | Runs all the functional tests from `test/controllers`, `test/mailers`, and `test/functional`|
+| `rails test helpers` | Runs all the helper tests from `test/helpers`|
+| `rails test integration` | Runs all the integration tests from `test/integration`|
+| `rails test mailers` | Runs all the mailer tests from `test/mailers`|
+| `rails test models` | Runs all the model tests from `test/models`|
+| `rails test units` | Runs all the unit tests from `test/models`, `test/helpers`, and `test/unit`|
-| Tasks | Description |
-| ------------------------------- | ----------- |
-| `rake test` | Runs all unit, functional and integration tests. You can also simply run `rake` as the _test_ target is the default.|
-| `rake test:controllers` | Runs all the controller tests from `test/controllers`|
-| `rake test:functionals` | Runs all the functional tests from `test/controllers`, `test/mailers`, and `test/functional`|
-| `rake test:helpers` | Runs all the helper tests from `test/helpers`|
-| `rake test:integration` | Runs all the integration tests from `test/integration`|
-| `rake test:mailers` | Runs all the mailer tests from `test/mailers`|
-| `rake test:models` | Runs all the model tests from `test/models`|
-| `rake test:recent` | Tests recent changes|
-| `rake test:uncommitted` | Runs all the tests which are uncommitted. Supports Subversion and Git|
-| `rake test:units` | Runs all the unit tests from `test/models`, `test/helpers`, and `test/unit`|
+There're also some test commands which you can initiate by running rake tasks:
+| Tasks | Description |
+| ------------------------ | ----------- |
+| `rake test` | Runs all unit, functional and integration tests. You can also simply run `rake` as the _test_ target is the default.|
+| `rake test:recent` | Tests recent changes|
+| `rake test:uncommitted` | Runs all the tests which are uncommitted. Supports Subversion and Git|
-Brief Note About `Test::Unit`
+Brief Note About `MiniTest`
-----------------------------
Ruby ships with a boat load of libraries. Ruby 1.8 provides `Test::Unit`, a framework for unit testing in Ruby. All the basic assertions discussed above are actually defined in `Test::Unit::Assertions`. The class `ActiveSupport::TestCase` which we have been using in our unit and functional tests extends `Test::Unit::TestCase`, allowing
View
32 railties/CHANGELOG.md
@@ -19,6 +19,38 @@
*Terence Lee*
+* Rails now generates a `test/test_helper.rb` file with `fixtures :all` commented out by default,
+ since we don't want to force loading all fixtures for user when a single test is run. However,
+ fixtures are still going to be loaded automatically for test suites.
+
+ To force all fixtures to be create in your database, use `rails test -f` to run your test.
+
+ *Prem Sichanugrist*
+
+* Add `rails test` command for running tests
+
+ To run all tests:
+
+ $ rails test
+
+ To run a test suite
+
+ $ rails test [models,helpers,units,controllers,mailers,...]
+
+ To run a selected test file(s):
+
+ $ rails test test/unit/foo_test.rb [test/unit/bar_test.rb ...]
+
+ To run a single test from a test file
+
+ $ rails test test/unit/foo_test.rb -n test_the_truth
+
+ For more information, see `rails test --help`.
+
+ This command will eventually replace `rake test:*` and `rake test` tasks
+
+ *Prem Sichanugrist and Chris Toomey*
+
* Add notice message for destroy action in scaffold generator.
*Rahul P. Chaudhari*
View
11 railties/lib/rails/commands.rb
@@ -5,6 +5,7 @@
"d" => "destroy",
"c" => "console",
"s" => "server",
+ "t" => "test",
"db" => "dbconsole",
"r" => "runner"
}
@@ -16,6 +17,7 @@
generate Generate new code (short-cut alias: "g")
console Start the Rails console (short-cut alias: "c")
server Start the Rails server (short-cut alias: "s")
+ test Running the test file (short-cut alias: "t")
dbconsole Start a console for the database specified in config/database.yml
(short-cut alias: "db")
new Create a new Rails application. "rails new my_app" creates a
@@ -78,6 +80,15 @@
server.start
end
+when 'test'
+ $LOAD_PATH.unshift("./test")
+ require 'rails/commands/test_runner'
+ options = Rails::TestRunner.parse_arguments(ARGV)
+ ENV['RAILS_ENV'] ||= options[:environment] || 'test'
+
+ require APP_PATH
+ Rails::TestRunner.start(ARGV, options)
+
when 'dbconsole'
require 'rails/commands/dbconsole'
Rails::DBConsole.start
View
146 railties/lib/rails/commands/test_runner.rb
@@ -0,0 +1,146 @@
+require 'optparse'
+require 'minitest/unit'
+
+module Rails
+ # Handles all logic behind +rails test+ command.
+ class TestRunner
+ class << self
+ # Creates a new +TestRunner+ object with an array of test files to run
+ # based on the arguments. When no arguments are provided, it runs all test
+ # files. When a suite argument is provided, it runs only the test files in
+ # that suite. Otherwise, it runs the specified test file(s).
+ def start(files, options = {})
+ original_fixtures_options = options.delete(:fixtures)
+ options[:fixtures] = true
+
+ case files.first
+ when nil
+ new(Dir['test/**/*_test.rb'], options).run
+ when 'models'
+ new(Dir['test/models/**/*_test.rb'], options).run
+ when 'helpers'
+ new(Dir['test/helpers/**/*_test.rb'], options).run
+ when 'units'
+ new(Dir['test/{models,helpers,unit}/**/*_test.rb'], options).run
+ when 'controllers'
+ new(Dir['test/controllers/**/*_test.rb'], options).run
+ when 'mailers'
+ new(Dir['test/mailers/**/*_test.rb'], options).run
+ when 'functionals'
+ new(Dir['test/{controllers,mailers,functional}/**/*_test.rb'], options).run
+ when 'integration'
+ new(Dir['test/integration/**/*_test.rb'], options).run
+ else
+ options[:fixtures] = original_fixtures_options
+ new(files, options).run
+ end
+ end
+
+ # Parses arguments and sets them as option flags
+ def parse_arguments(arguments)
+ options = {}
+ orig_arguments = arguments.dup
+
+ OptionParser.new do |opts|
+ opts.banner = "Usage: rails test [path to test file(s) or test suite]"
+
+ opts.separator ""
+ opts.separator "Run a specific test file(s) or a test suite, under Rails'"
+ opts.separator "environment. If the file name(s) or suit name is omitted,"
+ opts.separator "Rails will run all tests."
+ opts.separator ""
+ opts.separator "Specific options:"
+
+ opts.on '-h', '--help', 'Display this help.' do
+ puts opts
+ exit
+ end
+
+ opts.on '-e', '--environment NAME', String, 'Specifies the environment to run this test under' do |e|
+ options[:environment] = e
+ end
+
+ opts.on '-f', '--fixtures', 'Load fixtures in test/fixtures/ before running the tests' do
+ options[:fixtures] = true
+ end
+
+ opts.on '-s', '--seed SEED', Integer, "Sets random seed" do |m|
+ options[:seed] = m.to_i
+ end
+
+ opts.on '-v', '--verbose', "Verbose. Show progress processing files." do
+ options[:verbose] = true
+ end
+
+ opts.on '-n', '--name PATTERN', "Filter test names on pattern (e.g. /foo/)" do |n|
+ options[:filter] = n
+ end
+
+ opts.separator ""
+ opts.separator "Support types of test suites:"
+ opts.separator "-------------------------------------------------------------"
+ opts.separator "* models (test/models/**/*)"
+ opts.separator "* helpers (test/helpers/**/*)"
+ opts.separator "* units (test/{models,helpers,unit}/**/*"
+ opts.separator "* controllers (test/controllers/**/*)"
+ opts.separator "* mailers (test/mailers/**/*)"
+ opts.separator "* functionals (test/{controllers,mailers,functional}/**/*)"
+ opts.separator "* integration (test/integration/**/*)"
+ opts.separator "-------------------------------------------------------------"
+
+ opts.parse! arguments
+ orig_arguments -= arguments
+ end
+ options
+ end
+ end
+
+ # Creates a new +TestRunner+ object with a list of test file paths.
+ def initialize(files, options)
+ @files = files
+
+ Rails.application.load_tasks
+ Rake::Task['db:test:load'].invoke
+
+ if options.delete(:fixtures)
+ if defined?(ActiveRecord::Base)
+ ActiveSupport::TestCase.send :include, ActiveRecord::TestFixtures
+ ActiveSupport::TestCase.fixture_path = "#{Rails.root}/test/fixtures/"
+ ActiveSupport::TestCase.fixtures :all
+ end
+ end
+
+ MiniTest::Unit.runner.options = options
+ MiniTest::Unit.output = SilentUntilSyncStream.new(MiniTest::Unit.output)
+ end
+
+ # Runs test files by evaluating each of them.
+ def run
+ @files.each { |filename| load(filename) }
+ end
+
+ # A null stream object which ignores everything until +sync+ has been set
+ # to true. This is only used to silence unnecessary output from MiniTest,
+ # as MiniTest calls +output.sync = true+ right before it outputs the first
+ # test result.
+ class SilentUntilSyncStream < File
+ # Creates a +SilentUntilSyncStream+ object by giving it a target stream
+ # object that will be assigned to +MiniTest::Unit.output+ after +sync+ is
+ # set to true.
+ def initialize(target_stream)
+ @target_stream = target_stream
+ super(File::NULL, 'w')
+ end
+
+ # Swaps +MiniTest::Unit.output+ to another stream when +sync+ is true.
+ def sync=(sync)
+ if sync
+ @target_stream.sync = true
+ MiniTest::Unit.output = @target_stream
+ end
+
+ super
+ end
+ end
+ end
+end
View
7 railties/lib/rails/generators/rails/app/templates/test/test_helper.rb
@@ -1,4 +1,4 @@
-ENV["RAILS_ENV"] = "test"
+ENV["RAILS_ENV"] ||= "test"
require File.expand_path('../../config/environment', __FILE__)
require 'rails/test_help'
@@ -6,11 +6,12 @@ class ActiveSupport::TestCase
<% unless options[:skip_active_record] -%>
ActiveRecord::Migration.check_pending!
- # Setup all fixtures in test/fixtures/*.yml for all tests in alphabetical order.
+ # Uncomment the `fixtures :all` line below to setup all fixtures in test/fixtures/*.yml
+ # for all tests in alphabetical order.
#
# Note: You'll currently still have to declare fixtures explicitly in integration tests
# -- they do not yet inherit this setting
- fixtures :all
+ # fixtures :all
<% end -%>
# Add more helper methods to be used by all tests here...
View
80 railties/lib/rails/test_unit/testing.rake
@@ -1,6 +1,7 @@
require 'rbconfig'
require 'rake/testtask'
require 'rails/test_unit/sub_test_task'
+require 'active_support/deprecation'
TEST_CHANGES_SINCE = Time.now - 600
@@ -47,7 +48,11 @@ task default: :test
desc 'Runs test:units, test:functionals, test:integration together'
task :test do
- Rake::Task[ENV['TEST'] ? 'test:single' : 'test:run'].invoke
+ if ENV['TEST']
+ exec "bundle exec rails test #{ENV['TEST'].inspect}"
+ else
+ exec 'bundle exec rails test'
+ end
end
namespace :test do
@@ -56,19 +61,8 @@ namespace :test do
end
task :run do
- errors = %w(test:units test:functionals test:integration).collect do |task|
- begin
- Rake::Task[task].invoke
- nil
- rescue => e
- { task: task, exception: e }
- end
- end.compact
-
- if errors.any?
- puts errors.map { |e| "Errors running #{e[:task]}! #{e[:exception].inspect}" }.join("\n")
- abort
- end
+ ActiveSupport::Deprecation.warn "`rake test:run` is deprecated. Please use `rails test`."
+ exec 'bundle exec rails test'
end
# Inspired by: http://ngauthier.com/2012/02/quick-tests-with-bash.html
@@ -83,7 +77,13 @@ namespace :test do
task :db => %w[db:test:prepare test:all]
end
- Rake::TestTask.new(recent: "test:prepare") do |t|
+ # Display deprecation message
+ task :deprecated do
+ task_name = ARGV.first
+ ActiveSupport::Deprecation.warn "`rake #{ARGV.first}` is deprecated with no replacement."
+ end
+
+ Rake::TestTask.new(recent: ["test:deprecated", "test:prepare"]) do |t|
since = TEST_CHANGES_SINCE
touched = FileList['test/**/*_test.rb'].select { |path| File.mtime(path) > since } +
recent_tests('app/models/**/*.rb', 'test/models', since) +
@@ -94,9 +94,9 @@ namespace :test do
t.libs << 'test'
t.test_files = touched.uniq
end
- Rake::Task['test:recent'].comment = "Test recent changes"
+ Rake::Task['test:recent'].comment = "Deprecated; Test recent changes"
- Rake::TestTask.new(uncommitted: "test:prepare") do |t|
+ Rake::TestTask.new(uncommitted: ["test:deprecated", "test:prepare"]) do |t|
def t.file_list
if File.directory?(".svn")
changed_since_checkin = silence_stderr { `svn status` }.split.map { |path| path.chomp[7 .. -1] }
@@ -118,44 +118,20 @@ namespace :test do
t.libs << 'test'
end
- Rake::Task['test:uncommitted'].comment = "Test changes since last checkin (only Subversion and Git)"
-
- Rake::TestTask.new(single: "test:prepare") do |t|
- t.libs << "test"
- end
-
- Rails::SubTestTask.new(models: "test:prepare") do |t|
- t.libs << "test"
- t.pattern = 'test/models/**/*_test.rb'
- end
-
- Rails::SubTestTask.new(helpers: "test:prepare") do |t|
- t.libs << "test"
- t.pattern = 'test/helpers/**/*_test.rb'
- end
+ Rake::Task['test:uncommitted'].comment = "Deprecated; Test changes since last checkin (only Subversion and Git)"
- Rails::SubTestTask.new(units: "test:prepare") do |t|
- t.libs << "test"
- t.pattern = 'test/{models,helpers,unit}/**/*_test.rb'
+ desc "Deprecated; Please use `rails test \"#{ENV['TEST']}\"`"
+ task :single do
+ ActiveSupport::Deprecation.warn "`rake test:single` is deprecated. Please use `rails test \"#{ENV['TEST']}\"`."
+ exec "bundle exec rails test #{test_suit_name}"
end
- Rails::SubTestTask.new(controllers: "test:prepare") do |t|
- t.libs << "test"
- t.pattern = 'test/controllers/**/*_test.rb'
- end
+ [:models, :helpers, :units, :controllers, :functionals, :integration].each do |test_suit_name|
+ desc "Deprecated; Please use `rails test #{test_suit_name}`"
+ task test_suit_name do
+ ActiveSupport::Deprecation.warn "`rake test:#{test_suit_name}` is deprecated. Please use `rails test #{test_suit_name}`."
- Rails::SubTestTask.new(mailers: "test:prepare") do |t|
- t.libs << "test"
- t.pattern = 'test/mailers/**/*_test.rb'
- end
-
- Rails::SubTestTask.new(functionals: "test:prepare") do |t|
- t.libs << "test"
- t.pattern = 'test/{controllers,mailers,functional}/**/*_test.rb'
- end
-
- Rails::SubTestTask.new(integration: "test:prepare") do |t|
- t.libs << "test"
- t.pattern = 'test/integration/**/*_test.rb'
+ exec "bundle exec rails test #{test_suit_name}"
+ end
end
end
View
23 railties/test/application/rake_test.rb
@@ -91,19 +91,9 @@ def test_rake_test_error_output
raise 'models'
RUBY
- app_file "test/controllers/one_controller_test.rb", <<-RUBY
- raise 'controllers'
- RUBY
-
- app_file "test/integration/one_integration_test.rb", <<-RUBY
- raise 'integration'
- RUBY
-
silence_stderr do
output = Dir.chdir(app_path) { `rake test 2>&1` }
assert_match 'models', output
- assert_match 'controllers', output
- assert_match 'integration', output
end
end
@@ -135,6 +125,19 @@ def test_rake_test_uncommitted_fails_with_no_scm
end
end
+ def test_rake_test_deprecation_messages
+ Dir.chdir(app_path){ `rails generate scaffold user name:string` }
+ Dir.chdir(app_path){ `rake db:migrate` }
+
+ %w(run recent uncommitted models helpers units controllers functionals integration).each do |test_suit_name|
+ output = Dir.chdir(app_path) { `rake test:#{test_suit_name} 2>&1` }
+ assert_match /DEPRECATION WARNING: `rake test:#{test_suit_name}` is deprecated/, output
+ end
+
+ assert_match /DEPRECATION WARNING: `rake test:single` is deprecated/,
+ Dir.chdir(app_path) { `rake test:single TEST=test/models/user_test.rb 2>&1` }
+ end
+
def test_rake_routes_calls_the_route_inspector
app_file "config/routes.rb", <<-RUBY
AppTemplate::Application.routes.draw do
View
298 railties/test/application/test_runner_test.rb
@@ -0,0 +1,298 @@
+require 'isolation/abstract_unit'
+require 'active_support/core_ext/string/strip'
+
+module ApplicationTests
+ class TestRunnerTest < ActiveSupport::TestCase
+ include ActiveSupport::Testing::Isolation
+
+ def setup
+ build_app
+ create_schema
+ end
+
+ def teardown
+ teardown_app
+ end
+
+ def test_should_not_display_heading
+ create_test_file
+ run_test_command.tap do |output|
+ assert_no_match /Run options:/, output
+ assert_no_match /Running tests:/, output
+ end
+ end
+
+ def test_run_shortcut
+ create_test_file :models, 'foo'
+ output = Dir.chdir(app_path) { `bundle exec rails t test/models/foo_test.rb` }
+ assert_match /1 tests, 1 assertions, 0 failures/, output
+ end
+
+ def test_run_single_file
+ create_test_file :models, 'foo'
+ assert_match /1 tests, 1 assertions, 0 failures/, run_test_command("test/models/foo_test.rb")
+ end
+
+ def test_run_multiple_files
+ create_test_file :models, 'foo'
+ create_test_file :models, 'bar'
+ assert_match /2 tests, 2 assertions, 0 failures/, run_test_command("test/models/foo_test.rb test/models/bar_test.rb")
+ end
+
+ def test_run_file_with_syntax_error
+ app_file 'test/models/error_test.rb', <<-RUBY
+ require 'test_helper'
+ def; end
+ RUBY
+
+ error_stream = Tempfile.new('error')
+ redirect_stderr(error_stream) { run_test_command('test/models/error_test.rb') }
+ assert_match /SyntaxError/, error_stream.read
+ end
+
+ def test_invoke_rake_test_prepare
+ app_file "lib/tasks/test.rake", <<-RUBY
+ namespace :test do
+ task :prepare do
+ puts "Hello World"
+ end
+ end
+ RUBY
+ create_test_file
+ assert_match /Hello World/, run_test_command
+ end
+
+ def test_run_models
+ create_test_file :models, 'foo'
+ create_test_file :models, 'bar'
+ create_test_file :controllers, 'foobar_controller'
+ run_test_command("models").tap do |output|
+ assert_match /FooTest/, output
+ assert_match /BarTest/, output
+ assert_match /2 tests, 2 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_helpers
+ create_test_file :helpers, 'foo_helper'
+ create_test_file :helpers, 'bar_helper'
+ create_test_file :controllers, 'foobar_controller'
+ run_test_command('helpers').tap do |output|
+ assert_match /FooHelperTest/, output
+ assert_match /BarHelperTest/, output
+ assert_match /2 tests, 2 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_units
+ create_test_file :models, 'foo'
+ create_test_file :helpers, 'bar_helper'
+ create_test_file :unit, 'baz_unit'
+ create_test_file :controllers, 'foobar_controller'
+ run_test_command('units').tap do |output|
+ assert_match /FooTest/, output
+ assert_match /BarHelperTest/, output
+ assert_match /BazUnitTest/, output
+ assert_match /3 tests, 3 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_controllers
+ create_test_file :controllers, 'foo_controller'
+ create_test_file :controllers, 'bar_controller'
+ create_test_file :models, 'foo'
+ run_test_command('controllers').tap do |output|
+ assert_match /FooControllerTest/, output
+ assert_match /BarControllerTest/, output
+ assert_match /2 tests, 2 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_mailers
+ create_test_file :mailers, 'foo_mailer'
+ create_test_file :mailers, 'bar_mailer'
+ create_test_file :models, 'foo'
+ run_test_command('mailers').tap do |output|
+ assert_match /FooMailerTest/, output
+ assert_match /BarMailerTest/, output
+ assert_match /2 tests, 2 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_functionals
+ create_test_file :mailers, 'foo_mailer'
+ create_test_file :controllers, 'bar_controller'
+ create_test_file :functional, 'baz_functional'
+ create_test_file :models, 'foo'
+ run_test_command('functionals').tap do |output|
+ assert_match /FooMailerTest/, output
+ assert_match /BarControllerTest/, output
+ assert_match /BazFunctionalTest/, output
+ assert_match /3 tests, 3 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_integration
+ create_test_file :integration, 'foo_integration'
+ create_test_file :models, 'foo'
+ run_test_command('integration').tap do |output|
+ assert_match /FooIntegration/, output
+ assert_match /1 tests, 1 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_all_suites
+ suites = [:models, :helpers, :unit, :controllers, :mailers, :functional, :integration]
+ suites.each { |suite| create_test_file suite, "foo_#{suite}" }
+ run_test_command('') .tap do |output|
+ suites.each { |suite| assert_match /Foo#{suite.to_s.camelize}Test/, output }
+ assert_match /7 tests, 7 assertions, 0 failures/, output
+ end
+ end
+
+ def test_run_named_test
+ app_file 'test/unit/chu_2_koi_test.rb', <<-RUBY
+ require 'test_helper'
+
+ class Chu2KoiTest < ActiveSupport::TestCase
+ def test_rikka
+ puts 'Rikka'
+ end
+
+ def test_sanae
+ puts 'Sanae'
+ end
+ end
+ RUBY
+
+ run_test_command('test/unit/chu_2_koi_test.rb -n test_rikka').tap do |output|
+ assert_match /Rikka/, output
+ assert_no_match /Sanae/, output
+ end
+ end
+
+ def test_not_load_fixtures_when_running_single_test
+ create_model_with_fixture
+ create_fixture_test :models, 'user'
+ assert_match /0 users/, run_test_command('test/models/user_test.rb')
+ assert_match /3 users/, run_test_command('test/models/user_test.rb -f')
+ end
+
+ def test_load_fixtures_when_running_test_suites
+ create_model_with_fixture
+ suites = [:models, :helpers, [:units, :unit], :controllers, :mailers,
+ [:functionals, :functional], :integration]
+
+ suites.each do |suite, directory|
+ directory ||= suite
+ create_fixture_test directory
+ assert_match /3 users/, run_test_command(suite)
+ Dir.chdir(app_path) { FileUtils.rm_f "test/#{directory}" }
+ end
+ end
+
+ def test_run_different_environment_using_env_var
+ app_file 'test/unit/env_test.rb', <<-RUBY
+ require 'test_helper'
+
+ class EnvTest < ActiveSupport::TestCase
+ def test_env
+ puts Rails.env
+ end
+ end
+ RUBY
+
+ assert_match /development/, Dir.chdir(app_path) { `RAILS_ENV=development bundle exec rails test test/unit/env_test.rb` }
+ end
+
+ def test_run_different_environment_using_e_tag
+ app_file 'test/unit/env_test.rb', <<-RUBY
+ require 'test_helper'
+
+ class EnvTest < ActiveSupport::TestCase
+ def test_env
+ puts Rails.env
+ end
+ end
+ RUBY
+
+ assert_match /development/, run_test_command('-e development test/unit/env_test.rb')
+ end
+
+ def test_generated_scaffold_works_with_rails_test
+ create_scaffold
+ assert_match /0 failures, 0 errors, 0 skips/, run_test_command('')
+ end
+
+ private
+ def run_test_command(arguments = 'test/unit/test_test.rb')
+ Dir.chdir(app_path) { `bundle exec rails test #{arguments}` }
+ end
+
+ def create_model_with_fixture
+ script 'generate model user name:string'
+
+ app_file 'test/fixtures/users.yml', <<-YAML.strip_heredoc
+ vampire:
+ id: 1
+ name: Koyomi Araragi
+ crab:
+ id: 2
+ name: Senjougahara Hitagi
+ cat:
+ id: 3
+ name: Tsubasa Hanekawa
+ YAML
+
+ run_migration
+ end
+
+ def create_fixture_test(path = :unit, name = 'test')
+ app_file "test/#{path}/#{name}_test.rb", <<-RUBY
+ require 'test_helper'
+
+ class #{name.camelize}Test < ActiveSupport::TestCase
+ def test_fixture
+ puts "\#{User.count} users (\#{__FILE__})"
+ end
+ end
+ RUBY
+ end
+
+ def create_schema
+ app_file 'db/schema.rb', ''
+ end
+
+ def redirect_stderr(target_stream)
+ previous_stderr = STDERR.dup
+ $stderr.reopen(target_stream)
+ yield
+ target_stream.rewind
+ ensure
+ $stderr = previous_stderr
+ end
+
+ def create_test_file(path = :unit, name = 'test')
+ app_file "test/#{path}/#{name}_test.rb", <<-RUBY
+ require 'test_helper'
+
+ class #{name.camelize}Test < ActiveSupport::TestCase
+ def test_truth
+ puts "#{name.camelize}Test"
+ assert true
+ end
+ end
+ RUBY
+ end
+
+ def create_scaffold
+ script 'generate scaffold user name:string'
+ Dir.chdir(app_path) { File.exist?('app/models/user.rb') }
+ run_migration
+ end
+
+ def run_migration
+ Dir.chdir(app_path) { `bundle exec rake db:migrate` }
+ end
+ end
+end
Something went wrong with that request. Please try again.