Merge pull request #9080 from sikachu/master-rails-test

Add `rails test` command to run the test suite

Author: rafaelfranca

guides/source/testing.md

@@ -1,8 +1,7 @@
 A Guide to Testing Rails Applications
 =====================================
 
-This guide covers built-in mechanisms offered by Rails to test your
-application.
+This guide covers built-in mechanisms in Rails for testing your application.
 
 After reading this guide, you will know:
 
@@ -38,11 +37,11 @@ Rails creates a `test` folder for you as soon as you create a Rails project usin
 
 ```bash
 $ ls -F test
-
-fixtures/  functional/  integration/  test_helper.rb  unit/
+controllers/    helpers/        mailers/        test_helper.rb
+fixtures/       integration/    models/
 ```
 
-The `unit` directory is meant to hold tests for your models, the `functional` directory is meant to hold tests for your controllers and the `integration` directory is meant to hold tests that involve any number of controllers interacting.
+The `models` directory is meant to hold tests for your models, the `controllers` directory is meant to hold tests for your controllers and the `integration` directory is meant to hold tests that involve any number of controllers interacting.
 
 Fixtures are a way of organizing test data; they reside in the `fixtures` folder.
 
@@ -140,10 +139,9 @@ The default test stub in `test/models/post_test.rb` looks like this:
 require 'test_helper'
 
 class PostTest < ActiveSupport::TestCase
-  # Replace this with your real tests.
-  test "the truth" do
-    assert true
-  end
+  # test "the truth" do
+  #   assert true
+  # end
 end
 ```
 
@@ -224,34 +222,30 @@ TIP: You can see all these rake tasks and their descriptions by running `rake --
 
 ### Running Tests
 
-Running a test is as simple as invoking the file containing the test cases through Ruby:
+Running a test is as simple as invoking the file containing the test cases through `rails test` command.
 
 ```bash
-$ ruby -Itest test/models/post_test.rb
-
-Loaded suite models/post_test
-Started
+$ rails test test/models/post_test.rb
 .
-Finished in 0.023513 seconds.
 
-1 tests, 1 assertions, 0 failures, 0 errors
-```
+Finished tests in 0.009262s, 107.9680 tests/s, 107.9680 assertions/s.
 
-This will run all the test methods from the test case. Note that `test_helper.rb` is in the `test` directory, hence this directory needs to be added to the load path using the `-I` switch.
+1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
+```
 
-You can also run a particular test method from the test case by using the `-n` switch with the `test method name`.
+You can also run a particular test method from the test case by running the test and using `-n` switch with the `test method name`.
 
 ```bash
-$ ruby -Itest test/models/post_test.rb -n test_the_truth
-
-Loaded suite models/post_test
-Started
+$ rails test test/models/post_test.rb -n test_the_truth
 .
-Finished in 0.023513 seconds.
 
-1 tests, 1 assertions, 0 failures, 0 errors
+Finished tests in 0.009064s, 110.3266 tests/s, 110.3266 assertions/s.
+
+1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
 ```
 
+This will run all test methods from the test case. Note that `test_helper.rb` is in the `test` directory, hence this directory needs to be added to the load path using the `-I` switch.
+
 The `.` (dot) above indicates a passing test. When a test fails you see an `F`; when a test throws an error you see an `E` in its place. The last line of the output is the summary.
 
 To see how a test failure is reported, you can add a failing test to the `post_test.rb` test case.
@@ -266,17 +260,16 @@ end
 Let us run this newly added test.
 
 ```bash
-$ ruby unit/post_test.rb -n test_should_not_save_post_without_title
-Loaded suite -e
-Started
+$ rails test test/models/post_test.rb -n test_should_not_save_post_without_title
 F
-Finished in 0.102072 seconds.
+
+Finished tests in 0.044632s, 22.4054 tests/s, 22.4054 assertions/s.
 
   1) Failure:
-test_should_not_save_post_without_title(PostTest) [/test/models/post_test.rb:6]:
-<false> is not true.
+test_should_not_save_post_without_title(PostTest) [test/models/post_test.rb:6]:
+Failed assertion, no message given.
 
-1 tests, 1 assertions, 1 failures, 0 errors
+1 tests, 1 assertions, 1 failures, 0 errors, 0 skips
 ```
 
 In the output, `F` denotes a failure. You can see the corresponding trace shown under `1)` along with the name of the failing test. The next few lines contain the stack trace followed by a message which mentions the actual value and the expected value by the assertion. The default assertion messages provide just enough information to help pinpoint the error. To make the assertion failure message more readable, every assertion provides an optional message parameter, as shown here:
@@ -292,9 +285,8 @@ Running this test shows the friendlier assertion message:
 
 ```bash
   1) Failure:
-test_should_not_save_post_without_title(PostTest) [/test/models/post_test.rb:6]:
-Saved the post without a title.
-<false> is not true.
+test_should_not_save_post_without_title(PostTest) [test/models/post_test.rb:6]:
+Saved the post without a title
 ```
 
 Now to get this test to pass we can add a model level validation for the _title_ field.
@@ -308,13 +300,12 @@ end
 Now the test should pass. Let us verify by running the test again:
 
 ```bash
-$ ruby unit/post_test.rb -n test_should_not_save_post_without_title
-Loaded suite unit/post_test
-Started
+$ rails test test/models/post_test.rb -n test_should_not_save_post_without_title
 .
-Finished in 0.193608 seconds.
 
-1 tests, 1 assertions, 0 failures, 0 errors
+Finished tests in 0.047721s, 20.9551 tests/s, 20.9551 assertions/s.
+
+1 tests, 1 assertions, 0 failures, 0 errors, 0 skips
 ```
 
 Now, if you noticed, we first wrote a test which fails for a desired functionality, then we wrote some code which adds the functionality and finally we ensured that our test passes. This approach to software development is referred to as _Test-Driven Development_ (TDD).
@@ -334,18 +325,17 @@ end
 Now you can see even more output in the console from running the tests:
 
 ```bash
-$ ruby unit/post_test.rb -n test_should_report_error
-Loaded suite -e
-Started
+$ rails test test/models/post_test.rb -n test_should_report_error
 E
-Finished in 0.082603 seconds.
+
+Finished tests in 0.030974s, 32.2851 tests/s, 0.0000 assertions/s.
 
   1) Error:
 test_should_report_error(PostTest):
-NameError: undefined local variable or method `some_undefined_variable' for #<PostTest:0x249d354>
-    /test/models/post_test.rb:6:in `test_should_report_error'
+NameError: undefined local variable or method `some_undefined_variable' for #<PostTest:0x007fe32e24afe0>
+    test/models/post_test.rb:10:in `block in <class:PostTest>'
 
-1 tests, 0 assertions, 0 failures, 1 errors
+1 tests, 0 assertions, 0 failures, 1 errors, 0 skips
 ```
 
 Notice the 'E' in the output. It denotes a test with error.
@@ -642,12 +632,9 @@ Here's what a freshly-generated integration test looks like:
 require 'test_helper'
 
 class UserFlowsTest < ActionDispatch::IntegrationTest
-  fixtures :all
-
-  # Replace this with your real tests.
-  test "the truth" do
-    assert true
-  end
+  # test "the truth" do
+  #   assert true
+  # end
 end
 ```
 
@@ -755,23 +742,28 @@ end
 Rake Tasks for Running your Tests
 ---------------------------------
 
-You don't need to set up and run your tests by hand on a test-by-test basis. Rails comes with a number of rake tasks to help in testing. The table below lists all rake tasks that come along in the default Rakefile when you initiate a Rails project.
+You don't need to set up and run your tests by hand on a test-by-test basis. Rails comes with a number of commands to help in testing. The table below lists all commands that come along in the default Rakefile when you initiate a Rails project.
+
+| Tasks                    | Description |
+| ------------------------ | ----------- |
+| `rails test`             | Runs all unit, functional and integration tests. You can also simply run `rails test` as Rails will run all the tests by default|
+| `rails test controllers` | Runs all the controller tests from `test/controllers`|
+| `rails test functionals` | Runs all the functional tests from `test/controllers`, `test/mailers`, and `test/functional`|
+| `rails test helpers`     | Runs all the helper tests from `test/helpers`|
+| `rails test integration` | Runs all the integration tests from `test/integration`|
+| `rails test mailers`     | Runs all the mailer tests from `test/mailers`|
+| `rails test models`      | Runs all the model tests from `test/models`|
+| `rails test units`       | Runs all the unit tests from `test/models`, `test/helpers`, and `test/unit`|
 
-| Tasks                           | Description |
-| ------------------------------- | ----------- |
-| `rake test`                     | Runs all unit, functional and integration tests. You can also simply run `rake` as the _test_ target is the default.|
-| `rake test:controllers`         | Runs all the controller tests from `test/controllers`|
-| `rake test:functionals`         | Runs all the functional tests from `test/controllers`, `test/mailers`, and `test/functional`|
-| `rake test:helpers`             | Runs all the helper tests from `test/helpers`|
-| `rake test:integration`         | Runs all the integration tests from `test/integration`|
-| `rake test:mailers`             | Runs all the mailer tests from `test/mailers`|
-| `rake test:models`              | Runs all the model tests from `test/models`|
-| `rake test:recent`              | Tests recent changes|
-| `rake test:uncommitted`         | Runs all the tests which are uncommitted. Supports Subversion and Git|
-| `rake test:units`               | Runs all the unit tests from `test/models`, `test/helpers`, and `test/unit`|
+There're also some test commands which you can initiate by running rake tasks:
 
+| Tasks                    | Description |
+| ------------------------ | ----------- |
+| `rake test`              | Runs all unit, functional and integration tests. You can also simply run `rake` as the _test_ target is the default.|
+| `rake test:recent`       | Tests recent changes|
+| `rake test:uncommitted`  | Runs all the tests which are uncommitted. Supports Subversion and Git|
 
-Brief Note About `Test::Unit`
+Brief Note About `MiniTest`
 -----------------------------
 
 Ruby ships with a boat load of libraries. Ruby 1.8 provides `Test::Unit`, a framework for unit testing in Ruby. All the basic assertions discussed above are actually defined in `Test::Unit::Assertions`. The class `ActiveSupport::TestCase` which we have been using in our unit and functional tests extends `Test::Unit::TestCase`, allowing

railties/CHANGELOG.md

@@ -19,6 +19,38 @@
 
     *Terence Lee*
 
+*   Rails now generates a `test/test_helper.rb` file with `fixtures :all` commented out by default,
+    since we don't want to force loading all fixtures for user when a single test is run. However,
+    fixtures are still going to be loaded automatically for test suites.
+
+    To force all fixtures to be create in your database, use `rails test -f` to run your test.
+
+    *Prem Sichanugrist*
+
+*   Add `rails test` command for running tests
+
+    To run all tests:
+
+        $ rails test
+
+    To run a test suite
+
+        $ rails test [models,helpers,units,controllers,mailers,...]
+
+    To run a selected test file(s):
+
+        $ rails test test/unit/foo_test.rb [test/unit/bar_test.rb ...]
+
+    To run a single test from a test file
+
+        $ rails test test/unit/foo_test.rb -n test_the_truth
+
+    For more information, see `rails test --help`.
+
+    This command will eventually replace `rake test:*` and `rake test` tasks
+
+    *Prem Sichanugrist and Chris Toomey*
+
 *   Add notice message for destroy action in scaffold generator.
 
     *Rahul P. Chaudhari*

railties/lib/rails/commands.rb

@@ -5,6 +5,7 @@
   "d"  => "destroy",
   "c"  => "console",
   "s"  => "server",
+  "t"  => "test",
   "db" => "dbconsole",
   "r"  => "runner"
 }
@@ -16,6 +17,7 @@
  generate    Generate new code (short-cut alias: "g")
  console     Start the Rails console (short-cut alias: "c")
  server      Start the Rails server (short-cut alias: "s")
+ test        Running the test file (short-cut alias: "t")
  dbconsole   Start a console for the database specified in config/database.yml
              (short-cut alias: "db")
  new         Create a new Rails application. "rails new my_app" creates a
@@ -78,6 +80,15 @@
     server.start
   end
 
+when 'test'
+  $LOAD_PATH.unshift("./test")
+  require 'rails/commands/test_runner'
+  options = Rails::TestRunner.parse_arguments(ARGV)
+  ENV['RAILS_ENV'] ||= options[:environment] || 'test'
+
+  require APP_PATH
+  Rails::TestRunner.start(ARGV, options)
+
 when 'dbconsole'
   require 'rails/commands/dbconsole'
   Rails::DBConsole.start