Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Capybara Integration with Rails (AKA System Tests) #26703

Merged
merged 37 commits into from Feb 20, 2017

Conversation

@eileencodes
Copy link
Member

@eileencodes eileencodes commented Oct 4, 2016

Update: This PR changed a lot over the course of being open. Please refer to the documentation or code in master / 5-1-stable to use system tests

I'm really excited to open this initial PR for bringing system/acceptance test support to Rails through Capybara.

The goal of this PR is for Rails to take on all the setup that is needed in applications to allow Capybara and make system testing default in Rails applications.

Why is the name Rails::SystemTestCase?
I chose this name because this was the name DHH indicated he preferred in the Rails' Basecamp long ago. I'm totally open to changing the name but ultimately decided to put it in ActionPack with the Rails namespace so I could get onto writing code and stop worrying about the name 馃槈

The test framework has been moved it it's own gem under the Rails name and is now ActionSystemTest
Because the gem expects a Module and our test frameworks expect a Class for inheritance I've made the gem and module name ActionSystemTest and the class for inheriting test code from ActionSystemTestCase. Open to other names as well but the Module and Class names can't be the same else Ruby gets confused.

How do I add System tests to my application?
By default new applications include a base system test much like integration tests. Gemfiles generated for new applications will include Capybara and the Selenium driver.

Capybara's selenium driver requires some setup by the programmer, so I've added a layer between Capybara and Rails so that Rails' takes on that work called RailsSeleniumDriver. The default settings are as follows:

  • The server is puma (because this is Rails' default)
  • the browser is chrome (because FF is broken w/ Capybara and Selenium right now),
  • the default port is 28100, and
  • the default screen size is [1400, 1400].

The reason I have chosen Chrome as the default browser is because Firefox doesn't work out of the box. Selenium and the current version of Firefox don't play nicely together so I've set up the driver to default to Chrome, but it can easily be switched to using Firefox. I'd also like to provide support for Safari.

Instead of requiring the programmer to set up the port, browser, server, etc Rails handles that setup and frees up the programmer to work on writing their tests.

The Rails drivers allow the default configuration to be changed by initializing a new driver:

# config/environments/test.rb
config.system_testing.driver = SystemTesting::DriverAdapters::RailsSeleniumDriver.new(
  browser: :firefox
)

What if I don't want to use Selenium?
If the programmer wants to use one of Capybara's default drivers rather than the provided defaults in Rails Selenium configuration they can easily change that in the test environment. I've provided a shim in Rails so setting the Capybara driver is simple. I've named the class CapybaraDriver and it gives access to the 4 Capybara drivers: poltergeist, webrick, selenium (with no setup), and rack_test:

# config/environments/test.rb
config.system_testing.driver = :poltergeist

Each of the Capybara default drivers that requires a server defaults to Puma. Any of the settings can be changed by initializing a new CapbyaraDriver object:

# config/environments/test.rb
config.system_testing.driver = SystemTesting::DriverAdapters::CapybaraDriver.new(
  name: :poltergeist,
  server: :webrick
)

What if I'm making a new application and don't want system testing at all?
It's easy to skip system testing files in the app generator or the scaffold generator by running with the --skip-system-test flag.

What if I already use Capybara and don't want to use Rails?
Because Rails provides a specific test case name to inherit from you can easily just completely skip the Rails version of system testing and use Capybara directly.

I want to try it out!
I made a test app where you can try out system tests in Rails w/ Selenium! https://github.com/eileencodes/system_testing_app


Cool! What's next?

Below is a list of what is done and what's left to do. Once we sort out naming and other issues I'll finish up the items below.

What's done?

  • Generators: Scaffold, individual, and application
  • Basic Documentation
  • Testing adapter settings (this just tests that Rails provides options included, not that Capybara and friends work. The assumption is that actual behavior of Capybara and friends is tested by themselves, Rails tests it's framework for initializing Capybara)
  • Configurable driver adapters w/ RailsSeleniumDriver being the default.
  • Railtie for configuring system test settings
  • Support for screenshots
  • Changelog
  • Fix the railties tests 馃榿
  • Move to it's own gem under the rails name.
  • Rails Guides
  • Other custom but generic helpers we support at Basecamp that we'd like to port over, especially ActiveJob and ActionCable support
  • Testing the custom helpers and assertions
  • Because Capybara and the test suite use 2 different threads, transactions aren't correctly rolled back at the end of the test, leaving data behind. For years DatabaseCleaner has been used to resolve this problem, but it's fixable in Rails. See PR #28083

Future work

  • Support for Safari (last I checked this wasn't quite ready for prime time)

cc/ @dhh @georgeclaghorn

@eileencodes eileencodes added this to the 5.1.0 milestone Oct 4, 2016
@eileencodes eileencodes self-assigned this Oct 4, 2016
@prathamesh-sonpatki
Copy link
Member

@prathamesh-sonpatki prathamesh-sonpatki commented Oct 4, 2016

Transactions aren't handled in Capybara the same as they are in Rails' other test so we need to handle that. Capybara suggests using the DatabaseCleaner gem but we only need a small portion of that code.

@eileencodes We should also look for https://github.com/amatsuda/database_rewinder as an alternative to DatabaseCleaner.

@nynhex
Copy link

@nynhex nynhex commented Oct 4, 2016

This is excellent!

Copy link
Member

@alexcameron89 alexcameron89 left a comment

Hey @eileencodes, I ran through the documentation and commented on some grammar issues.

This PR looks great, and I'm excited for it to be a part of the default Test Suite!

actionpack/lib/system_test_case.rb Outdated

module Rails
# System tests are similar to Integration tests in that they incorporate multiple
# controllers and actions, but can be used to similate a real user experience.

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

similate -> simulate

actionpack/lib/system_testing/driver_adapters.rb Outdated
#
# By default Rails supports Capybara with the Selenium Driver. Rails provides
# configuration setup for using the selenium driver with Capybara.
# Additionally Rails can be used as a layer between Capybara and it's other

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

it's -> its

actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
# The drivers Capybara supports are: +:rack_test+, +:selenium+, +:webkit+,
# and +:poltergeist+.
#
# Rails provides it's own defaults for Capybara with the Selenium driver

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

it's -> its

actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
# directly.
#
# To set your system tests to use one of Capybara's default drivers add
# the following to yur Rails' configuration test environment:

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

"to use one of Capybara's default drivers add" -> "to use one of Capybara's default drivers, add"

line 18: yur -> your

actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
# JavaScript testing and doesn't require a server.
#
# The +:poltergeist+ and +:webkit+ drivers are headless, but require some
# extra environment setup. Because the defalt server for Rails is Puma, each

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

defalt -> default

actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
# of the Capybara drivers will default to using Puma. Changing the configuration
# to use Webrick is possible by initalizing a new driver object.
#
# The default settings for the <tt>CapybaraDrvier</tt> are:

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

CapybaraDrvier -> CapybaraDriver

actionpack/lib/system_testing/driver_adapters/rails_selenium_driver.rb Outdated
# selenium-webdriver gem is required by this driver.
#
# The <tt>RailsSeleniumDriver</tt> is useful for real browser testing and
# support Chrome and Firefox.

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

support -> supports

actionpack/lib/system_testing/driver_adapters/rails_selenium_driver.rb Outdated
# @screen_size=[ 1400, 1400 ]
# >
#
# The settings for the <tt>RailsSeleniumDriver</tt> can be changed in the

This comment has been minimized.

@alexcameron89

alexcameron89 Oct 4, 2016
Member

"in the Rails'" -> "in the Rails" OR "in Rail's"

@pixeltrix
Copy link
Member

@pixeltrix pixeltrix commented Oct 4, 2016

I assume that this follow's Capybara's opinionated policy of making it hard to access request/response objects and that we should use integration tests for those?

actionpack/lib/system_test_case.rb Outdated
require "system_testing/test_helper"
require "system_testing/driver_adapter"

module Rails

This comment has been minimized.

@maclover7

maclover7 Oct 4, 2016
Member

If we're shipping this under the actionpack gem, should we use the ActionPack base module? 馃槵

This comment has been minimized.

@eileencodes

eileencodes Oct 4, 2016
Author Member

I explained why I chose this name in the message above and I don't want to move it around 100 times before we have a consensus on name.

This comment has been minimized.

@dhh

dhh Oct 4, 2016
Member

I actually don't see any dependencies for ActionPack in this patch, so I think we could consider making it a complete stand-alone gem.

This comment has been minimized.

@eileencodes

eileencodes Oct 4, 2016
Author Member

@dhh Rails::SystemTestCase inherits from ActionDispatch::IntegrationTest so we don't have to rewrite all the routing and url helpers handling.

This comment has been minimized.

@dhh

dhh Oct 4, 2016
Member

Ah, I missed that. Then going with ActionDispatch::SystemTest probably makes sense. But agree no sense in making any changes until we really know if we love that name!

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

I like ActionDispatch::SystemTest

actionpack/lib/system_testing/driver_adapter.rb Outdated
# Rails' configuration file.
def driver=(driver)
@driver = DriverAdapters.lookup(driver)
@driver.call

This comment has been minimized.

@maclover7

maclover7 Oct 4, 2016
Member

Should we be more explicit about what call does, in regard to setting up / registering the adapter? 馃槵

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

Calling your personal @driver gets you a smooth getaway 馃槑

actionpack/lib/system_testing/driver_adapters.rb Outdated
# | Selenium | Firefox | Yes |
# | Webkit | Headless w/ QtWebKit | Yes |
# | Poltergeist | Headless w/ PhantomJS | Yes |
module DriverAdapters

This comment has been minimized.

@maclover7

maclover7 Oct 4, 2016
Member

Does it make sense to provide a "base" class to inherit from, so people know which methods they need to implement when creating adapters? 馃槵

actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
end

def supports_screenshots?
if @name == :rack_test

This comment has been minimized.

@fnando

fnando Oct 4, 2016
Contributor

Don't use a conditional to return a boolean.

def supports_screenshots?
  @name == :rack_test
end

This comment has been minimized.

@jonathanhefner

jonathanhefner Oct 5, 2016
Member

I agree, but it would be @name != :rack_test. (However, it may have been written as is with the intention that more cases / logic would be added.)

@sgrif
Copy link
Contributor

@sgrif sgrif commented Oct 4, 2016

Is there a specific reason for defaulting to selenium instead of a headless driver such as poltergeist or capybara-webkit?

@dixpac
Copy link
Contributor

@dixpac dixpac commented Oct 5, 2016

I agree with @sgrif, IMHO in real world apps people will switch to headless driver right away, so maybe capybara-webkit/poltergeist is better default here 馃槃

@eileencodes
Copy link
Member Author

@eileencodes eileencodes commented Oct 5, 2016

@sgrif and @dixpac I chose the Selenium driver as default because the purpose of adding system testing to Rails is for the initial setup to be absolutely zero. Capybara does that through their default, Rack Test, but it's not really a useful demonstration of the merits of system testing since it doesn't support JavaScript testing.

The selenium driver is one that Rails can take on all of the setup without any extra requirements from the programmer. capybara-webkit requires environmental setup that Rails can't take on - installing the Qt libraries. poltergeist requires PhantomJS which does have a gem but I don't know much about whether it's stable. We use Selenium at Basecamp and for now I'm going to leave the default as-is.

If there's a compelling reason to change the default it's as simple as swapping out the pre-set default and ensuring that Rails takes on the setup for that.


@pixeltrix Since this PR doesn't change how Capybara works, but rather provides a layer between Capybara and Rails.Currently, no, the request object isn't defined, but Rails::SystemTestCase does inherit from ActionDispatch::IntegrationTest so perhaps that's something we can easily expose in the future 馃槃

Copy link
Member

@kaspth kaspth left a comment

Looks sweet! Especially looking forward to the DatabaseCleaner work around 馃榿

actionpack/lib/system_test_case.rb Outdated
# fill_in 'Name', with: 'Arya'
# click_on 'Create User'
#
# assert_text 'Arya'

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

woof! 馃悤

actionpack/lib/system_test_case.rb Outdated
# click_on 'New User'
#
# fill_in 'Name', with: 'Arya'
# click_on 'Create User'

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

Should we do anything to make the Capybara helpers work better with Rails' integration helpers?

I haven't used Capybara much, but from what I can remember you can't call visit and then use assert_response :success for instance.

Pointing this out because the system test inherits from ActionDispatch::IntegrationTest so users might think our built in assertions would work.

This comment has been minimized.

@eileencodes

eileencodes Oct 14, 2016
Author Member

I think that's a great idea for the future, but right now there's still a ton left to do on this PR and I'm not exactly sure how to expose it just yet. I agree it would be great to add that in once we have this PR in master.

This comment has been minimized.

@kaspth

kaspth Oct 28, 2016
Member

Great, let's revisit some other time 馃憤

actionpack/lib/system_testing/driver_adapter.rb Outdated
# Rails' configuration file.
def driver=(driver)
@driver = DriverAdapters.lookup(driver)
@driver.call

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

Calling your personal @driver gets you a smooth getaway 馃槑

actionpack/lib/system_testing/driver_adapters.rb Outdated
# | Rails' Selenium | Chrome | Yes |
# | Rack Test | No JS Support | No |
# | Selenium | Firefox | Yes |
# | Webkit | Headless w/ QtWebKit | Yes |

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

I believe it's spelled WebKit.

actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
#
# config.system_testing.driver = SystemTesting::DriverAdapters::CapybaraDriver.new(
# name: :webkit,
# puma: :webrick

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

server: :webrick

actionpack/lib/system_testing/test_helpers/assertions.rb Outdated
private
def type_for_selector(*items)
if items.first.is_a?(Symbol)
items.shift

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

Why do we want to shift this arg? What kind of arguments can we see in this method?

actionpack/lib/system_testing/test_helpers/assertions.rb Outdated
#
# assert_none_of_selectors('ul', 'ol')
def assert_none_of_selectors(*items)
options = items.extract_options!

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

I think we can use a double splat here:

def assert_none_of_selectors(*items, **options)
  # ...
end
actionpack/lib/system_testing/test_helpers/assertions.rb Outdated
# Asserts that all of the provided selectors are present on the given page.
#
# assert_all_of_selectors('p', 'td')
def assert_all_of_selectors(*items)

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

Ditto re double splat.

actionpack/lib/system_testing/test_helpers/screenshot_helper.rb Outdated
def image_path
path = "tmp/screenshots/failures_#{method_name}.png"
page.save_screenshot(Rails.root.join(path))
path

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

Find it a little odd that image_path both returns a path and saves an image. I mean, wouldn't the two calls to image_path in find_image save two more images?

actionpack/test/system_testing/screenshot_helper_test.rb Outdated
assert_equal true, Rails::SystemTestCase.driver.supports_screenshots?

Rails::SystemTestCase.driver = :poltergeist
assert_equal true, Rails::SystemTestCase.driver.supports_screenshots?

This comment has been minimized.

@kaspth

kaspth Oct 6, 2016
Member

Why do we want this to return booleans over our generally preferred falsey values?

This comment has been minimized.

@eileencodes

eileencodes Oct 15, 2016
Author Member

This was an oversight, I've fixed this now.

@eileencodes eileencodes force-pushed the eileencodes:rails_system_tests branch Oct 7, 2016
@eileencodes eileencodes force-pushed the eileencodes:rails_system_tests branch 4 times, most recently Oct 14, 2016
actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
# name: :webkit,
# server: :webrick
# )
class CapybaraDriver

This comment has been minimized.

@maclover7

maclover7 Oct 15, 2016
Member

[sorry, just reposting this comment since it got accidentally marked as "outdated" thanks to the great work being done here :)]

Does it make sense to provide a "base" class to inherit from, so people know which methods they need to implement when creating adapters? 馃槵

This comment has been minimized.

@eileencodes

eileencodes Oct 15, 2016
Author Member

Hey @maclover7 thanks. I know there are a lot of comments to address and getting around to them as I can. In the future, no need to re-comment.

I don't think adding a base class makes sense becasue the Rails provided adapter takes different arguments than the capybara provided adapters. It might make sense as this project evolves but I'll tackle that when that happens.

actionpack/lib/system_testing/driver_adapter.rb Outdated
@@ -0,0 +1,31 @@
require "system_testing/driver_adapters"

module SystemTesting

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

Should not this class be in one namespacing?

Copy link
Member

@rafaelfranca rafaelfranca left a comment

It feels to me that this should be a new gem. It have its own namespace, its own railtie it can be skipped and I don't think we should add capybara as dependency of actionpack. I'd create a new gem, inside the rails repository and add it as dependency of Rails.

actionpack/lib/system_testing/driver_adapters/capybara_driver.rb Outdated
@port = port
end

def call

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

Maybe call it start?

actionpack/lib/system_testing/driver_adapters/rails_selenium_driver.rb Outdated
# config.system_testing.driver = SystemTesting::DriverAdapters::RailsSeleniumDriver.new(
# server: :webrick,
# port: 28100,
# screen_size: [ 800, 800 ]

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

Our style guide doesn't put spaces inside array literals

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

Actually it is the other way around. I'll make rubocop point that.

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

There is no support for that in rubocop 馃槩 .

actionpack/lib/system_testing/driver_adapters/rails_selenium_driver.rb Outdated

attr_reader :browser, :server, :port, :screen_size

def initialize(browser: :chrome, server: :puma, port: 28100, screen_size: [ 1400,1400 ]) # :nodoc:

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

missing space after the , and remove the space inside the array literal

actionpack/lib/system_testing/railtie.rb Outdated
options.driver ||= Rails::SystemTestCase.default_driver

ActiveSupport.on_load(:system_testing) do
options.each { |k,v| send("#{k}=", v) }

This comment has been minimized.

@rafaelfranca

rafaelfranca Oct 29, 2016
Member

space after the ,

@eileencodes
Copy link
Member Author

@eileencodes eileencodes commented Oct 29, 2016

@rafaelfranca David and I talked about the naming etc this week and decided on creating a new gem inside Rails, so I'll be doing that soon. Thanks for the feedback!

@eileencodes eileencodes force-pushed the eileencodes:rails_system_tests branch Nov 7, 2016
@eileencodes eileencodes force-pushed the eileencodes:rails_system_tests branch Nov 11, 2016
actionsystemtest/actionsystemtest.gemspec Outdated

s.license = "MIT"

s.author = ["Eileen Uchitell", "David Heinemeier Hansson"]

This comment has been minimized.

@kaspth

kaspth Nov 11, 2016
Member

Personally I'd throw an e on there, but I'll let you be the judge of how to spell your name 馃槃

This comment has been minimized.

@eileencodes

eileencodes Nov 11, 2016
Author Member

I noticed that, still working on a bunch of stuff. I'll let you know when I'm ready for more review.

This comment has been minimized.

@kaspth

kaspth Nov 11, 2016
Member

Sounds good 馃憤

actionsystemtest/lib/action_system_test/driver_adapters.rb Outdated
# === Driver Features
#
# | | Default Browser | Supports Screenshots? |
# |-----------------|-----------------------|-----------------------|

This comment has been minimized.

@kaspth

kaspth Nov 11, 2016
Member

Nitpick: saw another table where we put spaces between the rows and pipes, e.g.: ---- | ----. Not sure if we have a preferred style.

actionsystemtest/lib/action_system_test/gem_version.rb Outdated
@@ -0,0 +1,15 @@
module ActionSystemTest
# Returns the version of the currently loaded Action Cable as a <tt>Gem::Version</tt>.

This comment has been minimized.

@kaspth

kaspth Nov 11, 2016
Member

Let's be cable cutters: Action System Test, I presume 馃榿

actionsystemtest/lib/action_system_test/version.rb Outdated
require_relative "gem_version"

module ActionSystemTest
# Returns the version of the currently loaded Action Cable as a <tt>Gem::Version</tt>

This comment has been minimized.

@kaspth

kaspth Nov 11, 2016
Member

Cable :)

railties/test/application/test_test.rb Outdated
puts "before: " + has_user_table.to_s
end

task :after_hook do
has_user_table = ActiveRecord::Base.connection.table_exists?('users')
has_user_table = ActiveRecord::Base.connection.data_source_exists?('users')

This comment has been minimized.

@kaspth

kaspth Nov 11, 2016
Member

How is this change related to system testing?

@eileencodes eileencodes force-pushed the eileencodes:rails_system_tests branch 2 times, most recently Nov 12, 2016
actionsystemtest/lib/action_system_test/test_helpers/form_helper.rb Outdated
# click_checkbox_label 'Admin', checked: true
def click_checkbox_label(name, checked: false)
field = find_checkbox(name, checked)
label = find_label_wrapper(field)

This comment has been minimized.

@twalpole

twalpole Nov 12, 2016
Contributor

Capybara provides a selector - https://github.com/jnicklas/capybara/blob/master/lib/capybara/selector.rb#L432 - for the label that wraps an input or where the id/for attributes match. so this could be find(:label, for: field)

actionsystemtest/lib/action_system_test/test_helpers/form_helper.rb Outdated
end

private
def find_checkbox(name, checked)

This comment has been minimized.

@twalpole

twalpole Nov 12, 2016
Contributor

Capybara provides a :checkbox selector - https://github.com/jnicklas/capybara/blob/master/lib/capybara/selector.rb#L298 - which would limit this to only checkboxes if thats whats desired

eileencodes added 6 commits Feb 18, 2017
I've renamed the server to `rails_puma` so that it doesn't override
Capybara's default puma server. I've also removed the hard port setting.
Users can simply use `Capybara.server_port` writer to set the port if
they require that.
* Move system tests back into Action Pack
* Rename `ActionSystemTest` to `ActionDispatch::SystemTestCase`
* Remove private base module and only make file for public
`SystemTestCase` class, name private module `SystemTesting`
* Rename `ActionSystemTestCase` to `ApplicationSystemTestCase`
* Update corresponding documentation and guides
* Delete old `ActionSystemTest` files
This serves as self documentation so users know how to change the
driver.
We only want the file name to include the word `failures` if it failed,
not any time the user wants to take a screenshot during a test run.
* Override integration test default host

Integration tests automatically set the default host to
'http://example.com'. This works fine for integration tests because they
are not real browser sessions, but doesn't work fine for system tests
because they are real browser sessions.

We can override this by setting the `host!` in `before_setup. The
`Capybara.always_include_port` will allow the test to look at
`127.0.0.1:port capybara picks` and properly redirect the test.

Any application can override this by setting the `host!` in
their system test helper. Generally though, applications are going to be
using localhost.

In this commit I also moved the setup and teardown into their own module
for tidiness.

* Move teardown settings into system test case

These configuration options can be put into the system test case file
instead of the generated system tests helper file. This is an
implementation detail and therefore shouldn't be generated with the
template.
This renames the system test helper file to be application system test
case to match what the rest of Rails does. In the future we should
consider changing the test_helper to match.
@eileencodes eileencodes force-pushed the eileencodes:rails_system_tests branch to 2d61c5d Feb 20, 2017
@dhh
Copy link
Member

@dhh dhh commented Feb 20, 2017

There were some grammar issues and incorrect information in the system
tests documentation.
@eileencodes eileencodes merged commit 2ee146a into rails:master Feb 20, 2017
2 checks passed
2 checks passed
codeclimate no new or fixed issues
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@kaspth
Copy link
Member

@kaspth kaspth commented Feb 20, 2017

@eileencodes yeeees, so cool! 鉂わ笍

@rafaelfranca

This comment has been minimized.

Copy link
Member

@rafaelfranca rafaelfranca commented on actionsystemtest/lib/action_system_test.rb in 893c3b6 Feb 23, 2017

Is this supposed to be nodoc?

@rafaelfranca

This comment has been minimized.

Is this supposed to be documentation of SystemTestCase or just internal documentation for framework contributors?

This comment has been minimized.

Copy link
Member Author

@eileencodes eileencodes replied Feb 23, 2017

It's supposed to be documentation of SystemTestCase but I realized it's in the wrong place. I'm going to move it now.

@vraravam
Copy link

@vraravam vraravam commented Feb 27, 2017

Though I like this, I would like to understand if/whether its easy to separate out the Unit tests from the functional tests as a separate RAILS_ENV? Basically, almost all the code that I am involved in - we try to run these as separate stages in the CI pipeline, thus having a separate RAILS_ENV would make sense so that the test suites can be run independently of each of other?

@pixeltrix
Copy link
Member

@pixeltrix pixeltrix commented Feb 27, 2017

@vraravam having a separate environment isn't something we'd do by default but you should be able to copy config/environments/test.rb to a new file such as acceptance.rb and have that work. If there's some assumption somewhere that we're running as RAILS_ENV=test please do file a bug report in a new ticket.

@vraravam
Copy link

@vraravam vraravam commented Feb 27, 2017

@pixeltrix - that's how I have been proceeding till now on all my rails codebases. But, I was trying to formalize the existence of such an env from the core framework itself. :)

@pixeltrix
Copy link
Member

@pixeltrix pixeltrix commented Feb 27, 2017

@vraravam if that's the case tacking it on the end of a PR with lots of comments is pretty much the worst place to discuss it - you'd be better off discussing it here: https://groups.google.com/forum/#!forum/rubyonrails-core

@ghost
Copy link

@ghost ghost commented Mar 26, 2017

Anyone have an example/instructions how to set this up with rspec-rails?

@eileencodes
Copy link
Member Author

@eileencodes eileencodes commented Mar 27, 2017

@xanview I haven't tried setting it up with Rspec but this basically builds a simple shim between Rails and Capybara to make setup easier. I think you should be able to make the class the same way in Rails and then inherit from that in your test files. If you're having issues with Rspec and suspect it's a Rails bug please open an issue. If you're having issues and suspect it's how you're doing the setup in your app you can ask on the Rails mailing list instead.

@bf4
Copy link
Contributor

@bf4 bf4 commented Apr 30, 2017

FWIW, in case there is anyone confused that the PR description reads that an ActionSystemTest gem was extracted, it was, (as described in #26703 (comment) ) but then, during review, it was subsequently (re)moved in 1a0ca84

(IMHO it's probably all for the good since otherwise people (like me) might try to get it working on earlier versions of Rails, and, after reading the code and related PRs, that would be quite the rabbithole, so I'm glad it wasn't made its own gem and saved me from putting time into that :))

cc @eileencodes 炉\_(銉)_/炉

@eileencodes
Copy link
Member Author

@eileencodes eileencodes commented May 1, 2017

Yes @bf4 creating a gem specifically for system tests didn't make sense in the end and it was moved back inside Action Pack and named ActionDispatch::SystemTestCase. There was some discussion that happened outside of this PR that isn't in here.

Since this PR was open for 3 months I wouldn't really use it to determine how system tests work based on the comments. Please refer to documentation or the code itself as it is in the 5-1-stable /master branch and not this PR. A lot has changed since I merged it as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can鈥檛 perform that action at this time.