Permalink
Browse files

ci setup

  • Loading branch information...
1 parent c02ad64 commit fb3fa8ede3f3b3c5057191ee7ed06cec1fe12b8f @johnbender committed May 22, 2012
Showing with 203 additions and 0 deletions.
  1. +4 −0 tests/ci/Gemfile
  2. +43 −0 tests/ci/Gemfile.lock
  3. +25 −0 tests/ci/README.md
  4. +131 −0 tests/ci/test_qunit.rb
View
@@ -0,0 +1,4 @@
+source 'http://rubygems.org'
+gem "rake"
+gem 'ci_reporter', '>=1.6.3'
+gem "capybara"
View
@@ -0,0 +1,43 @@
+GEM
+ remote: http://rubygems.org/
+ specs:
+ addressable (2.2.8)
+ builder (3.0.0)
+ capybara (1.1.2)
+ mime-types (>= 1.16)
+ nokogiri (>= 1.3.3)
+ rack (>= 1.0.0)
+ rack-test (>= 0.5.4)
+ selenium-webdriver (~> 2.0)
+ xpath (~> 0.1.4)
+ childprocess (0.3.2)
+ ffi (~> 1.0.6)
+ ci_reporter (1.7.0)
+ builder (>= 2.1.2)
+ ffi (1.0.11)
+ libwebsocket (0.1.3)
+ addressable
+ mime-types (1.18)
+ multi_json (1.3.4)
+ nokogiri (1.5.2)
+ rack (1.4.1)
+ rack-test (0.6.1)
+ rack (>= 1.0)
+ rake (0.9.2.2)
+ rubyzip (0.9.8)
+ selenium-webdriver (2.21.2)
+ childprocess (>= 0.2.5)
+ ffi (~> 1.0)
+ libwebsocket (~> 0.1.3)
+ multi_json (~> 1.0)
+ rubyzip
+ xpath (0.1.4)
+ nokogiri (~> 1.3)
+
+PLATFORMS
+ ruby
+
+DEPENDENCIES
+ capybara
+ ci_reporter (>= 1.6.3)
+ rake
View
@@ -0,0 +1,25 @@
+# jQuery Mobile simple CI
+
+After getting the test suite to a place where a full run was taking more time and effort that was realistic to expect from devs a simple CI integration was built using selenium. Generally it relies on Selenium's ability to retrieve information via selectors from the DOM of the page it's operating on which it then reports.
+
+## Dependencies
+
+The test suite depends on Ruby, a host of Ruby Gems listed in the Gemfile, Selenium drivers for both firefox and chrome, the browsers themselves, and an apache server serving the site at localhost:80 (configurable). The output from the tests is exported in JUnit xml format for integration with Jenkins and any other CI server that support said format.
+
+## Setup
+
+The setup is mostly automated (may require tweeking) via [chef](http://wiki.opscode.com/display/chef/FAQ) cookbooks included at the root of the project under the `cookbooks` directory. `chef-solo` is the preferred method of setup. The idea is that building a new CI server with the test suite from scratch should be trivial and documented with code.
+
+## Test lifecycle
+
+When `ruby test_qunit.rb` is executed two `TestUnit` test sets are run, one for each browser. In each test set a seperate test is created for all `tests/unit/` child directories and any files in any sub directory (however nested) that end in `-tests.html`. The two globs are `tests/unit/*` and `tests/unit/**/*-tests.html`. In this way new test pages can be added easily by following the glob conventions.
+
+Each test is identical save for the url it instructs Selenium to visit which is always a QUnit page. Once the page loads, the server side test waits for client side test suite in the page to complete by polling the QUnit banner for failure or success. If it finds a failure at any time it will short circuit the test and report failure immediatley (would be better to record all failures but requires work).
+
+It reports failure or success by asserting on the set of failed tests in the page. If that set is empty, the assertion reports success otherwise failure.
+
+Once the test suite has run all the tests created for each browser it outputs the results into JUnit format.
+
+## Code quality
+
+The Ruby used to run the test suite leaves a lot to be desired in the way of code quality (see, globs and gsubs for test names) but for now it's serviceably small.
View
@@ -0,0 +1,131 @@
+require 'test/unit'
+require 'fileutils'
+require "rubygems"
+require "bundler/setup"
+require "capybara"
+require "capybara/dsl"
+require "rake"
+require "ci/reporter/rake/test_unit"
+
+#must be required here or ci_reporter won't work. it expects rake
+require "ci/reporter/rake/test_unit_loader"
+
+Capybara.configure do |config|
+ config.default_driver = :firefox
+ config.default_selector = :css
+end
+
+Capybara.register_driver :chrome do |app|
+ Capybara::Selenium::Driver.new(app, :browser => :chrome)
+end
+
+Capybara.register_driver :firefox do |app|
+ Capybara::Selenium::Driver.new(app, :browser => :firefox)
+end
+
+Rake::Task['ci:setup:testunit'].invoke
+
+# set the application host using an environment variable
+# allows the ci to set a path for the build command
+# eg http//localhost/1.0-stable
+Capybara.app_host = ENV['CI_APP_HOST'] || Capybara.app_host
+
+# Allow the ci to set the jquery versions for testing, depends on the file
+# js/jquery-#{version} being available for movement to js/jquery.js
+JQUERY_VERSIONS = (ENV['JQUERY_VERSIONS'] || "1.6.4,1.7.1").split(",")
+
+# testing directory relative to this file
+# NOTE not using File.join since this is always going to be nix
+PROJECT_ROOT = File.expand_path(File.join(File.dirname(__FILE__), "../../"))
+TEST_DIR = PROJECT_ROOT + "/tests/unit/"
+JS_ROOT = PROJECT_ROOT + "/js/"
+
+class TestQunit < Test::Unit::TestCase
+ include Capybara::DSL
+
+ def self.driver
+ Capybara.default_driver
+ end
+
+ def driver
+ self.class.driver
+ end
+
+ def swap_jquery(version)
+ FileUtils.cp("#{JS_ROOT}/jquery-#{version}.js", "#{JS_ROOT}/jquery.js")
+ end
+
+ # TODO this globbing and next filtering can be simplified
+ (Dir[TEST_DIR + "*"] + Dir[TEST_DIR + "**/*-tests.html"]).each do |path|
+ next if !File.directory?(path) && !path.include?("-tests.html")
+ # next if !path.include?("disabled")
+
+ # strip the parent directory ... ugh this sucks
+ path = "/tests" + path.split('/tests')[1]
+
+ JQUERY_VERSIONS.each do |jquery_version|
+ # remove the dashes, dots, and slashes from the path (naive), remove
+ # the first dash, and replace tests with test so test unit will pick them up
+ test_name = path.gsub(/\/|-|\./, "_").gsub(/^_/, "")
+
+ # add in the jquery version
+ test_name += "_jquery_" + jquery_version.gsub(".", "_")
+
+ # make sure the test unit picks up the test method
+ test_name += "_test"
+
+ # Define a test method for each unit test path
+ define_method "#{test_name}" do
+ # get some more readable console ouput
+ puts
+ print "method name: #{test_name}, path: #{path}, result: "
+
+ Capybara.current_driver = driver
+
+ # NOTE swapping before every test :(
+ swap_jquery(jquery_version)
+
+ # visit the test suit page
+ visit(path)
+
+ wait_for_tests
+ unless all_failing.empty?
+ # TODO figure out why chained find didn't work here
+ failing_descriptions = all("#qunit-tests .fail .test-name").map { |elem|
+ elem.text()
+ }.join("\n")
+ end
+
+ assert_description = <<-MSG
+ there should be no failures at #{path} for #{driver} with jquery #{jquery_version}. Failing:
+ #{failing_descriptions}"
+ MSG
+
+ assert(all_failing.empty?, assert_description)
+ end
+ end
+ end
+
+ # verify that the tests are finished or there is a failure
+ def wait_for_tests(attempts = 100)
+ attempts.times do
+ banner_class = find("#qunit-banner")[:class]
+ # break if the banner is there or theres a failure already
+ break if banner_class && banner_class.include?("pass")
+ break unless all_failing.empty?
+
+ sleep 1
+ end
+ end
+
+ # get the failing test elements
+ def all_failing
+ all("#qunit-tests .fail")
+ end
+end
+
+class TestQunitChrome < TestQunit
+ def self.driver
+ :chrome
+ end
+end

0 comments on commit fb3fa8e

Please sign in to comment.