Add task for visual regression of review apps

Compare production and a review app hosted on Heroku.

* Pass in a PR number to generate a config and run the test.
* Run against the heroku deployment of master
* Refactors wraith tasks to DRY up

README.md

@@ -156,12 +156,26 @@ for each known `document_type` in the app (see: [Generate a config for known doc
 
 Running the rake task below will retrieve the `document_types` where `rendering_app = government-frontend` from the search api. It will then generate `test/wraith/wip-config-all-document-types.yaml`, this is a wraith config file containing the top 10 (can be overidden with `:sample_size`) example pages for each type.
 
-The yaml file contains a custom key of `:document_types` not used by wraith but can be used to quickly scan and see which types the search api believes `government-frontend` is responsible for.
-
 ```
 bundle exec rake wraith:update_document_types[:sample_size]
 ```
 
+#### Compare a pull request review app with production
+
+Compares production with government-frontend-pr-[pr_number].herokuapp.com
+
+```
+bundle exec rake wraith:pr[:pr_number]
+```
+
+#### Compare master with production
+
+Compares production with government-frontend.herokuapp.com (an automatic deployment of the master branch)
+
+```
+bundle exec rake wraith:master
+```
+
 ### Adding a new format
 
 There’s a rails generator you can use to stub the basic files needed for a new format. It stubs the following:

lib/helpers/wraith_config_helper.rb

@@ -12,7 +12,7 @@ def initialize(name, paths)
   def create_config(extra_config = {})
     config = load_template
     config["paths"] = build_paths
-    config.merge!(extra_config) if extra_config.present?
+    config.deep_merge!(extra_config) if extra_config.present?
 
     write_config(config)
   end

lib/tasks/wraith.rake

@@ -15,19 +15,48 @@ namespace :wraith do
   desc "check top 10 content items for all known document types"
   task :all_document_types, [:sample_size] do |_t, args|
     args.with_defaults(sample_size: 10)
-    # Make sure an up to date document_types file exists
-    Rake::Task["wraith:update_document_types"].invoke args[:sample_size]
-    wraith_config_file = "test/wraith/wip-config-all-document-types.yaml"
-
-    exec("bundle exec wraith capture #{wraith_config_file}")
+    config_file_name = generate_wip_config("all-document-types", document_type_paths(args[:sample_size]))
+    run_wraith_with_config(config_file_name)
   end
 
   desc "creates a wraith config of document type examples from the search api"
   task :update_document_types, [:sample_size] do |_t, args|
     args.with_defaults(sample_size: 10)
-    document_type_paths = DocumentTypesHelper.new(args[:sample_size]).all_type_paths
-    document_types = { "document_types" => document_type_paths.keys }
+    generate_wip_config("all-document-types", document_type_paths(args[:sample_size]))
+  end
+
+  desc "compare a pull request review deployment with production"
+  task :pr, [:pr_number] do |_t, args|
+    options = {}
+    options["domains"] = {
+      "local" => "https://government-frontend-pr-#{args[:pr_number]}.herokuapp.com"
+    }
+
+    config_file_name = generate_wip_config("pr-review", document_type_paths(2), options)
+    run_wraith_with_config(config_file_name)
+  end
+
+  desc "compare master with production"
+  task :master do
+    options = {}
+    options["domains"] = {
+      "local" => "https://government-frontend.herokuapp.com"
+    }
+
+    config_file_name = generate_wip_config("pr-master", document_type_paths(2), options)
+    run_wraith_with_config(config_file_name)
+  end
+
+  def generate_wip_config(name, paths, options = {})
+    WraithConfigHelper.new(name, paths).create_config(options)
+  end
+
+  def document_type_paths(sample_size)
+    DocumentTypesHelper.new(sample_size).all_type_paths
+  end
 
-    WraithConfigHelper.new("all-document-types", document_type_paths).create_config(document_types)
+  def run_wraith_with_config(file_name)
+    puts "Running wraith with config file: #{file_name}"
+    exec("bundle exec wraith capture #{file_name}")
   end
 end

test/wraith/config.yaml

@@ -274,4 +274,4 @@ gallery:
 mode: diffs_first
 
 # (optional) Choose to run Wraith in verbose mode, for easier debugging. Default: false
-verbose: true
+verbose: false