Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Capybara Integration with Rails (AKA System Tests) #26703

Merged
merged 37 commits into from Feb 20, 2017

Conversation

eileencodes
Copy link
Member

@eileencodes eileencodes commented Oct 4, 2016

Update: This PR changed a lot over the course of being open. Please refer to the documentation or code in master / 5-1-stable to use system tests

I'm really excited to open this initial PR for bringing system/acceptance test support to Rails through Capybara.

The goal of this PR is for Rails to take on all the setup that is needed in applications to allow Capybara and make system testing default in Rails applications.

Why is the name Rails::SystemTestCase?
I chose this name because this was the name DHH indicated he preferred in the Rails' Basecamp long ago. I'm totally open to changing the name but ultimately decided to put it in ActionPack with the Rails namespace so I could get onto writing code and stop worrying about the name 馃槈

The test framework has been moved it it's own gem under the Rails name and is now ActionSystemTest
Because the gem expects a Module and our test frameworks expect a Class for inheritance I've made the gem and module name ActionSystemTest and the class for inheriting test code from ActionSystemTestCase. Open to other names as well but the Module and Class names can't be the same else Ruby gets confused.

How do I add System tests to my application?
By default new applications include a base system test much like integration tests. Gemfiles generated for new applications will include Capybara and the Selenium driver.

Capybara's selenium driver requires some setup by the programmer, so I've added a layer between Capybara and Rails so that Rails' takes on that work called RailsSeleniumDriver. The default settings are as follows:

  • The server is puma (because this is Rails' default)
  • the browser is chrome (because FF is broken w/ Capybara and Selenium right now),
  • the default port is 28100, and
  • the default screen size is [1400, 1400].

The reason I have chosen Chrome as the default browser is because Firefox doesn't work out of the box. Selenium and the current version of Firefox don't play nicely together so I've set up the driver to default to Chrome, but it can easily be switched to using Firefox. I'd also like to provide support for Safari.

Instead of requiring the programmer to set up the port, browser, server, etc Rails handles that setup and frees up the programmer to work on writing their tests.

The Rails drivers allow the default configuration to be changed by initializing a new driver:

# config/environments/test.rb
config.system_testing.driver = SystemTesting::DriverAdapters::RailsSeleniumDriver.new(
  browser: :firefox
)

What if I don't want to use Selenium?
If the programmer wants to use one of Capybara's default drivers rather than the provided defaults in Rails Selenium configuration they can easily change that in the test environment. I've provided a shim in Rails so setting the Capybara driver is simple. I've named the class CapybaraDriver and it gives access to the 4 Capybara drivers: poltergeist, webrick, selenium (with no setup), and rack_test:

# config/environments/test.rb
config.system_testing.driver = :poltergeist

Each of the Capybara default drivers that requires a server defaults to Puma. Any of the settings can be changed by initializing a new CapbyaraDriver object:

# config/environments/test.rb
config.system_testing.driver = SystemTesting::DriverAdapters::CapybaraDriver.new(
  name: :poltergeist,
  server: :webrick
)

What if I'm making a new application and don't want system testing at all?
It's easy to skip system testing files in the app generator or the scaffold generator by running with the --skip-system-test flag.

What if I already use Capybara and don't want to use Rails?
Because Rails provides a specific test case name to inherit from you can easily just completely skip the Rails version of system testing and use Capybara directly.

I want to try it out!
I made a test app where you can try out system tests in Rails w/ Selenium! https://github.com/eileencodes/system_testing_app


Cool! What's next?

Below is a list of what is done and what's left to do. Once we sort out naming and other issues I'll finish up the items below.

What's done?

  • Generators: Scaffold, individual, and application
  • Basic Documentation
  • Testing adapter settings (this just tests that Rails provides options included, not that Capybara and friends work. The assumption is that actual behavior of Capybara and friends is tested by themselves, Rails tests it's framework for initializing Capybara)
  • Configurable driver adapters w/ RailsSeleniumDriver being the default.
  • Railtie for configuring system test settings
  • Support for screenshots
  • Changelog
  • Fix the railties tests 馃榿
  • Move to it's own gem under the rails name.
  • Rails Guides
  • Other custom but generic helpers we support at Basecamp that we'd like to port over, especially ActiveJob and ActionCable support
  • Testing the custom helpers and assertions
  • Because Capybara and the test suite use 2 different threads, transactions aren't correctly rolled back at the end of the test, leaving data behind. For years DatabaseCleaner has been used to resolve this problem, but it's fixable in Rails. See PR Ensure test threads share a DB connection聽#28083

Future work

  • Support for Safari (last I checked this wasn't quite ready for prime time)

cc/ @dhh @georgeclaghorn

@eileencodes eileencodes added this to the 5.1.0 milestone Oct 4, 2016
@eileencodes eileencodes self-assigned this Oct 4, 2016
@prathamesh-sonpatki
Copy link
Member

Transactions aren't handled in Capybara the same as they are in Rails' other test so we need to handle that. Capybara suggests using the DatabaseCleaner gem but we only need a small portion of that code.

@eileencodes We should also look for https://github.com/amatsuda/database_rewinder as an alternative to DatabaseCleaner.

@nynhex
Copy link

nynhex commented Oct 4, 2016

This is excellent!

Copy link
Member

@alexcameron89 alexcameron89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @eileencodes, I ran through the documentation and commented on some grammar issues.

This PR looks great, and I'm excited for it to be a part of the default Test Suite!


module Rails
# System tests are similar to Integration tests in that they incorporate multiple
# controllers and actions, but can be used to similate a real user experience.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

similate -> simulate

#
# By default Rails supports Capybara with the Selenium Driver. Rails provides
# configuration setup for using the selenium driver with Capybara.
# Additionally Rails can be used as a layer between Capybara and it's other
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's -> its

# The drivers Capybara supports are: +:rack_test+, +:selenium+, +:webkit+,
# and +:poltergeist+.
#
# Rails provides it's own defaults for Capybara with the Selenium driver
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's -> its

# directly.
#
# To set your system tests to use one of Capybara's default drivers add
# the following to yur Rails' configuration test environment:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"to use one of Capybara's default drivers add" -> "to use one of Capybara's default drivers, add"

line 18: yur -> your

# JavaScript testing and doesn't require a server.
#
# The +:poltergeist+ and +:webkit+ drivers are headless, but require some
# extra environment setup. Because the defalt server for Rails is Puma, each
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defalt -> default

# of the Capybara drivers will default to using Puma. Changing the configuration
# to use Webrick is possible by initalizing a new driver object.
#
# The default settings for the <tt>CapybaraDrvier</tt> are:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CapybaraDrvier -> CapybaraDriver

# selenium-webdriver gem is required by this driver.
#
# The <tt>RailsSeleniumDriver</tt> is useful for real browser testing and
# support Chrome and Firefox.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

support -> supports

# @screen_size=[ 1400, 1400 ]
# >
#
# The settings for the <tt>RailsSeleniumDriver</tt> can be changed in the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"in the Rails'" -> "in the Rails" OR "in Rail's"

@pixeltrix
Copy link
Contributor

I assume that this follow's Capybara's opinionated policy of making it hard to access request/response objects and that we should use integration tests for those?

require "system_testing/test_helper"
require "system_testing/driver_adapter"

module Rails
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're shipping this under the actionpack gem, should we use the ActionPack base module? 馃槵

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I explained why I chose this name in the message above and I don't want to move it around 100 times before we have a consensus on name.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually don't see any dependencies for ActionPack in this patch, so I think we could consider making it a complete stand-alone gem.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dhh Rails::SystemTestCase inherits from ActionDispatch::IntegrationTest so we don't have to rewrite all the routing and url helpers handling.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I missed that. Then going with ActionDispatch::SystemTest probably makes sense. But agree no sense in making any changes until we really know if we love that name!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like ActionDispatch::SystemTest

# Rails' configuration file.
def driver=(driver)
@driver = DriverAdapters.lookup(driver)
@driver.call
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be more explicit about what call does, in regard to setting up / registering the adapter? 馃槵

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling your personal @driver gets you a smooth getaway 馃槑

# | Selenium | Firefox | Yes |
# | Webkit | Headless w/ QtWebKit | Yes |
# | Poltergeist | Headless w/ PhantomJS | Yes |
module DriverAdapters
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to provide a "base" class to inherit from, so people know which methods they need to implement when creating adapters? 馃槵

end

def supports_screenshots?
if @name == :rack_test
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't use a conditional to return a boolean.

def supports_screenshots?
  @name == :rack_test
end

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, but it would be @name != :rack_test. (However, it may have been written as is with the intention that more cases / logic would be added.)

@sgrif
Copy link
Contributor

sgrif commented Oct 4, 2016

Is there a specific reason for defaulting to selenium instead of a headless driver such as poltergeist or capybara-webkit?

@dixpac
Copy link
Contributor

dixpac commented Oct 5, 2016

I agree with @sgrif, IMHO in real world apps people will switch to headless driver right away, so maybe capybara-webkit/poltergeist is better default here 馃槃

@eileencodes
Copy link
Member Author

eileencodes commented Oct 5, 2016

@sgrif and @dixpac I chose the Selenium driver as default because the purpose of adding system testing to Rails is for the initial setup to be absolutely zero. Capybara does that through their default, Rack Test, but it's not really a useful demonstration of the merits of system testing since it doesn't support JavaScript testing.

The selenium driver is one that Rails can take on all of the setup without any extra requirements from the programmer. capybara-webkit requires environmental setup that Rails can't take on - installing the Qt libraries. poltergeist requires PhantomJS which does have a gem but I don't know much about whether it's stable. We use Selenium at Basecamp and for now I'm going to leave the default as-is.

If there's a compelling reason to change the default it's as simple as swapping out the pre-set default and ensuring that Rails takes on the setup for that.


@pixeltrix Since this PR doesn't change how Capybara works, but rather provides a layer between Capybara and Rails.Currently, no, the request object isn't defined, but Rails::SystemTestCase does inherit from ActionDispatch::IntegrationTest so perhaps that's something we can easily expose in the future 馃槃

Copy link
Contributor

@kaspth kaspth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks sweet! Especially looking forward to the DatabaseCleaner work around 馃榿

# fill_in 'Name', with: 'Arya'
# click_on 'Create User'
#
# assert_text 'Arya'
Copy link
Contributor

@kaspth kaspth Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

woof! 馃悤

# click_on 'New User'
#
# fill_in 'Name', with: 'Arya'
# click_on 'Create User'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do anything to make the Capybara helpers work better with Rails' integration helpers?

I haven't used Capybara much, but from what I can remember you can't call visit and then use assert_response :success for instance.

Pointing this out because the system test inherits from ActionDispatch::IntegrationTest so users might think our built in assertions would work.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's a great idea for the future, but right now there's still a ton left to do on this PR and I'm not exactly sure how to expose it just yet. I agree it would be great to add that in once we have this PR in master.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, let's revisit some other time 馃憤

# Rails' configuration file.
def driver=(driver)
@driver = DriverAdapters.lookup(driver)
@driver.call
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Calling your personal @driver gets you a smooth getaway 馃槑

# | Rails' Selenium | Chrome | Yes |
# | Rack Test | No JS Support | No |
# | Selenium | Firefox | Yes |
# | Webkit | Headless w/ QtWebKit | Yes |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe it's spelled WebKit.

#
# config.system_testing.driver = SystemTesting::DriverAdapters::CapybaraDriver.new(
# name: :webkit,
# puma: :webrick
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

server: :webrick

private
def type_for_selector(*items)
if items.first.is_a?(Symbol)
items.shift
Copy link
Contributor

@kaspth kaspth Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to shift this arg? What kind of arguments can we see in this method?

#
# assert_none_of_selectors('ul', 'ol')
def assert_none_of_selectors(*items)
options = items.extract_options!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use a double splat here:

def assert_none_of_selectors(*items, **options)
  # ...
end

# Asserts that all of the provided selectors are present on the given page.
#
# assert_all_of_selectors('p', 'td')
def assert_all_of_selectors(*items)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ditto re double splat.

def image_path
path = "tmp/screenshots/failures_#{method_name}.png"
page.save_screenshot(Rails.root.join(path))
path
Copy link
Contributor

@kaspth kaspth Oct 6, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Find it a little odd that image_path both returns a path and saves an image. I mean, wouldn't the two calls to image_path in find_image save two more images?

assert_equal true, Rails::SystemTestCase.driver.supports_screenshots?

Rails::SystemTestCase.driver = :poltergeist
assert_equal true, Rails::SystemTestCase.driver.supports_screenshots?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want this to return booleans over our generally preferred falsey values?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was an oversight, I've fixed this now.

# name: :webkit,
# server: :webrick
# )
class CapybaraDriver
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[sorry, just reposting this comment since it got accidentally marked as "outdated" thanks to the great work being done here :)]

Does it make sense to provide a "base" class to inherit from, so people know which methods they need to implement when creating adapters? 馃槵

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @maclover7 thanks. I know there are a lot of comments to address and getting around to them as I can. In the future, no need to re-comment.

I don't think adding a base class makes sense becasue the Rails provided adapter takes different arguments than the capybara provided adapters. It might make sense as this project evolves but I'll tackle that when that happens.

@@ -0,0 +1,31 @@
require "system_testing/driver_adapters"

module SystemTesting
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should not this class be in one namespacing?

rafaelfranca
rafaelfranca previously requested changes Oct 29, 2016
Copy link
Member

@rafaelfranca rafaelfranca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels to me that this should be a new gem. It have its own namespace, its own railtie it can be skipped and I don't think we should add capybara as dependency of actionpack. I'd create a new gem, inside the rails repository and add it as dependency of Rails.

@port = port
end

def call
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe call it start?

# config.system_testing.driver = SystemTesting::DriverAdapters::RailsSeleniumDriver.new(
# server: :webrick,
# port: 28100,
# screen_size: [ 800, 800 ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our style guide doesn't put spaces inside array literals

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually it is the other way around. I'll make rubocop point that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no support for that in rubocop 馃槩 .


attr_reader :browser, :server, :port, :screen_size

def initialize(browser: :chrome, server: :puma, port: 28100, screen_size: [ 1400,1400 ]) # :nodoc:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing space after the , and remove the space inside the array literal

options.driver ||= Rails::SystemTestCase.default_driver

ActiveSupport.on_load(:system_testing) do
options.each { |k,v| send("#{k}=", v) }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

space after the ,

@eileencodes
Copy link
Member Author

@rafaelfranca David and I talked about the naming etc this week and decided on creating a new gem inside Rails, so I'll be doing that soon. Thanks for the feedback!


s.license = "MIT"

s.author = ["Eileen Uchitell", "David Heinemeier Hansson"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally I'd throw an e on there, but I'll let you be the judge of how to spell your name 馃槃

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed that, still working on a bunch of stuff. I'll let you know when I'm ready for more review.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good 馃憤

# === Driver Features
#
# | | Default Browser | Supports Screenshots? |
# |-----------------|-----------------------|-----------------------|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: saw another table where we put spaces between the rows and pipes, e.g.: ---- | ----. Not sure if we have a preferred style.

@@ -0,0 +1,15 @@
module ActionSystemTest
# Returns the version of the currently loaded Action Cable as a <tt>Gem::Version</tt>.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's be cable cutters: Action System Test, I presume 馃榿

require_relative "gem_version"

module ActionSystemTest
# Returns the version of the currently loaded Action Cable as a <tt>Gem::Version</tt>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cable :)

puts "before: " + has_user_table.to_s
end

task :after_hook do
has_user_table = ActiveRecord::Base.connection.table_exists?('users')
has_user_table = ActiveRecord::Base.connection.data_source_exists?('users')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this change related to system testing?

@eileencodes eileencodes force-pushed the rails_system_tests branch 2 times, most recently from 41ff098 to ab55f1e Compare November 12, 2016 17:16
# click_checkbox_label 'Admin', checked: true
def click_checkbox_label(name, checked: false)
field = find_checkbox(name, checked)
label = find_label_wrapper(field)
Copy link
Contributor

@twalpole twalpole Nov 12, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capybara provides a selector - https://github.com/jnicklas/capybara/blob/master/lib/capybara/selector.rb#L432 - for the label that wraps an input or where the id/for attributes match. so this could be find(:label, for: field)

end

private
def find_checkbox(name, checked)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capybara provides a :checkbox selector - https://github.com/jnicklas/capybara/blob/master/lib/capybara/selector.rb#L298 - which would limit this to only checkboxes if thats whats desired

This adds the required guides for how to write and use system tests in
your application.
This is a major rewrite of what existed previously. After discussing
this feature with DHH I realized that I was looking at the setup all
wrong.

I had originally mentally broken it into "what Rails wants" and "what
Capybara already has".

What happened after looking at it from DHH's angle was that I saw there
was no reason to group settings by Driver but instead the following
groups:

- There will always be a `Driver`
  - This can selenium, poltergeist, or capybara webkit. Capybara already
    provides all of these and there's no reason to break them into a
    category of Rails' usese Selenium like this and Capybara uses it
    like that.
- Only Selenium drivers care about `Browser`
  - Because of this it was weird to set it only in the Rails end.
  - Therefore only `Browser`, and not `Driver` cares about
    `screen_size`.
- Puma is the default `Server` in Rails
  - Therefore there's no reason to explictly support Webkit

Once I looked at it from this angle I was able to abstract all the
settings away from grouping the drivers with their options.

Now all the driver, server, and browser settings are abstracted away and
not part of the public facing API.

This means there's no requirement to initialize new classes to change
the default settings and the public options API is much smaller.

All of Rails preferred defaults are still there (selenium with port
21800 using the chrome browser with a screen size of 1400x1400) but
changing these no longer requires initializing a new class or
understanding which driver you're using underneath (rails defaults or
capybaras defaults respectively). Rails opinions are now simple defaults
instead of doing a them versus us setup with Drivers and explicit
options.

Changing the defaults is simple. Call `driven_by` with different
settings to change the defaults which will on their own initialize new
classes and change the default settings.

Use poltergeist with port 3000 for Puma

```
driven_by :poltergeist, on: 3000
```

Use selenium with the Chrome browser and a screen size of 800x800
```
driven_by :selenium, using: :firefox, screen_size: [ 800, 800 ]
```

The entire setup of how browser and drivers interact with each other are
abstracted away and the only required argument is the driver name.
Update the documentation after rewriting a majority of the functionality
for system testing.
I've renamed the server to `rails_puma` so that it doesn't override
Capybara's default puma server. I've also removed the hard port setting.
Users can simply use `Capybara.server_port` writer to set the port if
they require that.
* Move system tests back into Action Pack
* Rename `ActionSystemTest` to `ActionDispatch::SystemTestCase`
* Remove private base module and only make file for public
`SystemTestCase` class, name private module `SystemTesting`
* Rename `ActionSystemTestCase` to `ApplicationSystemTestCase`
* Update corresponding documentation and guides
* Delete old `ActionSystemTest` files
This serves as self documentation so users know how to change the
driver.
We only want the file name to include the word `failures` if it failed,
not any time the user wants to take a screenshot during a test run.
* Override integration test default host

Integration tests automatically set the default host to
'http://example.com'. This works fine for integration tests because they
are not real browser sessions, but doesn't work fine for system tests
because they are real browser sessions.

We can override this by setting the `host!` in `before_setup. The
`Capybara.always_include_port` will allow the test to look at
`127.0.0.1:port capybara picks` and properly redirect the test.

Any application can override this by setting the `host!` in
their system test helper. Generally though, applications are going to be
using localhost.

In this commit I also moved the setup and teardown into their own module
for tidiness.

* Move teardown settings into system test case

These configuration options can be put into the system test case file
instead of the generated system tests helper file. This is an
implementation detail and therefore shouldn't be generated with the
template.
This renames the system test helper file to be application system test
case to match what the rest of Rails does. In the future we should
consider changing the test_helper to match.
@dhh
Copy link
Member

dhh commented Feb 20, 2017 via email

There were some grammar issues and incorrect information in the system
tests documentation.
@eileencodes eileencodes merged commit 2ee146a into rails:master Feb 20, 2017
@kaspth
Copy link
Contributor

kaspth commented Feb 20, 2017

@eileencodes yeeees, so cool! 鉂わ笍

@vraravam
Copy link

Though I like this, I would like to understand if/whether its easy to separate out the Unit tests from the functional tests as a separate RAILS_ENV? Basically, almost all the code that I am involved in - we try to run these as separate stages in the CI pipeline, thus having a separate RAILS_ENV would make sense so that the test suites can be run independently of each of other?

@pixeltrix
Copy link
Contributor

@vraravam having a separate environment isn't something we'd do by default but you should be able to copy config/environments/test.rb to a new file such as acceptance.rb and have that work. If there's some assumption somewhere that we're running as RAILS_ENV=test please do file a bug report in a new ticket.

@vraravam
Copy link

@pixeltrix - that's how I have been proceeding till now on all my rails codebases. But, I was trying to formalize the existence of such an env from the core framework itself. :)

@pixeltrix
Copy link
Contributor

@vraravam if that's the case tacking it on the end of a PR with lots of comments is pretty much the worst place to discuss it - you'd be better off discussing it here: https://groups.google.com/forum/#!forum/rubyonrails-core

@tetherit
Copy link

Anyone have an example/instructions how to set this up with rspec-rails?

@eileencodes
Copy link
Member Author

@xanview I haven't tried setting it up with Rspec but this basically builds a simple shim between Rails and Capybara to make setup easier. I think you should be able to make the class the same way in Rails and then inherit from that in your test files. If you're having issues with Rspec and suspect it's a Rails bug please open an issue. If you're having issues and suspect it's how you're doing the setup in your app you can ask on the Rails mailing list instead.

@bf4
Copy link
Contributor

bf4 commented Apr 30, 2017

FWIW, in case there is anyone confused that the PR description reads that an ActionSystemTest gem was extracted, it was, (as described in #26703 (comment) ) but then, during review, it was subsequently (re)moved in 1a0ca84

(IMHO it's probably all for the good since otherwise people (like me) might try to get it working on earlier versions of Rails, and, after reading the code and related PRs, that would be quite the rabbithole, so I'm glad it wasn't made its own gem and saved me from putting time into that :))

cc @eileencodes 炉\_(銉)_/炉

@eileencodes
Copy link
Member Author

Yes @bf4 creating a gem specifically for system tests didn't make sense in the end and it was moved back inside Action Pack and named ActionDispatch::SystemTestCase. There was some discussion that happened outside of this PR that isn't in here.

Since this PR was open for 3 months I wouldn't really use it to determine how system tests work based on the comments. Please refer to documentation or the code itself as it is in the 5-1-stable /master branch and not this PR. A lot has changed since I merged it as well.