Fix Dangerbot parsing of Travis errors in Rails 4

[jywarren]

Until recently, Dangerbot would repeat out errors and failures in each PR, like in this example:

#1708 (comment) (see how Dangerbot actually reproduced the errors in its comment?)

In this recent one (after the Rails 4.1 upgrade), this no longer happens. This PR, which had the error:

TagNodeAssociationExtension was here
Started with run options --seed 66
 FAIL["test_should_return_error_for_wiki_pages_that_do_not_follow_/slug_format", WikiControllerTest, 60.372967383999935]
 test_should_return_error_for_wiki_pages_that_do_not_follow_/slug_format#WikiControllerTest (60.37s)
        Expected response to be a redirect to <http://test.host/404> but was a redirect to <http://test.host/wiki/invalid>.
        Expected "http://test.host/404" to be === "http://test.host/wiki/invalid".
        test/functional/wiki_controller_test.rb:274:in `block in <class:WikiControllerTest>'
  577/577: [===============================] 100% Time: 00:01:29, Time: 00:01:29
Finished in 89.24451s
577 tests, 1708 assertions, 1 failures, 0 errors, 0 skips

shown here: https://travis-ci.org/publiclab/plots2/builds/314000925?utm_source=github_status&utm_medium=notification

does not show any Danger-reported messages: #1851

(you can open a new PR with failed tests to confirm this)

The Dangerfile is here:

https://github.com/publiclab/plots2/blob/master/Dangerfile

The error it generates is:

There was an error with Danger bot's Junit parsing:
undefined method `value' for nil:NilClass

Let's fix this! It may also involve the .travis.yml file here, which generates the output.xml files which Danger and Junit parse:

plots2/.travis.yml

Lines 22 to 25 in 5ffd7c6

	- mkdir -p ./test/reports
	- touch ./test/reports/TEST_.xml
	- echo -e '<?xml version="1.0" encoding="UTF-8"?>' > output.xml
	- tail -n +2 -q ./test/reports/TEST*.xml >> output.xml

[ryzokuken]

The error message gives an insight that this piece of code is being called here to report the error:

rescue => ex
  puts "There was an error with Danger bot's Junit parsing:"
  puts ex.message
end

Because this is a rescue call, something has thrown an error somewhere, let me single it out.

The error says: undefined method 'value' for nil:NilClass, but the Dangerfile (https://raw.githubusercontent.com/publiclab/plots2/master/Dangerfile) does not even contain the term value, let alone try to call a function. It seems to be an internal issue with the library to me, let's get to the root of the issue.

@jywarren how did you produce this danger error message? Anywhere I could play with these values?

$ mkdir -p ./test/reports

The command "mkdir -p ./test/reports" exited with 0.

$ touch ./test/reports/TEST_.xml

The command "touch ./test/reports/TEST_.xml" exited with 0.

@jywarren are you sure reports were even generated? If so, then why do we need to make the test/reports folder (it should already exist) and make a blank error report file?

[ryzokuken]

I tried running GENERATE_REPORT=true rake test:all on my own computer to check what's wrong, using a modified Rakefile containing

...
task :all => :environment do
    require 'coveralls/rake/task'
    Coveralls::RakeTask.new

    STDOUT.puts 'Reports being generated' # Extra line one
    k = STDIN.gets # Extra line two

    if ENV['GENERATE_REPORT'] == 'true'
      require 'ci/reporter/rake/test_unit'
      Rake::Task["ci:setup:testunit"].execute
    end
...

With two extra lines added. Shouldn't the Rake task halt momentarily for input? It doesn't. Something seems fishy.

[ryzokuken]

Okay, I finally made the last two work. The problem now is, with or without a test failure, no test/reports directory is being made. It should be if CI::Reporter is working, right?

~/Code/plots2 master*
❯ tree | grep report
│       │   ├── jasmine-console-reporter.js

~/Code/plots2 master*
❯ GENERATE_REPORT=true rake test:all
Running Solr-dependent tests
Not running mysql today.
rm -rf test/reports
Running Rails tests
[Coveralls] Using SimpleCov's default settings.
Not running mysql today.
/Users/ryzokuken/.rvm/gems/ruby-2.3.4/gems/activerecord-4.1.16/lib/active_record/associations/builder/collection_association.rb:38: warning: already initialized constant TagNodeAssociationExtension
/Users/ryzokuken/.rvm/gems/ruby-2.3.4/gems/activerecord-4.1.16/lib/active_record/associations/builder/collection_association.rb:38: warning: previous definition of TagNodeAssociationExtension was here
Started with run options --seed 37679

 FAIL["test_notify_node_moderators", AdminMailerTest, 54.385041999979876]
 test_notify_node_moderators#AdminMailerTest (54.39s)
        Expected nil to not be nil.
        test/unit/admin_mailer_test.rb:20:in `block in <class:AdminMailerTest>'

  579/579: [=====] 100% Time: 00:00:54, Time: 00:00:54

Finished in 54.38567s
579 tests, 1705 assertions, 1 failures, 0 errors, 0 skips
rake aborted!
Command failed with status (1): [ruby -I"lib:test" -I"/Users/ryzokuken/.rvm/gems/ruby-2.3.4/gems/rake-10.5.0/lib" "/Users/ryzokuken/.rvm/gems/ruby-2.3.4/gems/rake-10.5.0/lib/rake/rake_test_loader.rb" "test/functional/admin_controller_test.rb" "test/functional/answer_like_controller_test.rb" "test/functional/answers_controller_test.rb" "test/functional/application_controller_test.rb" "test/functional/comment_controller_test.rb" "test/functional/comment_mailer_test.rb" "test/functional/editor_controller_test.rb" "test/functional/features_controller_test.rb" "test/functional/home_controller_test.rb" "test/functional/images_controller_test.rb" "test/functional/like_controller_test.rb" "test/functional/map_controller_test.rb" "test/functional/notes_controller_test.rb" "test/functional/questions_controller_test.rb" "test/functional/relationships_controller_test.rb" "test/functional/restful_search_test.rb" "test/functional/search_api_test.rb" "test/functional/searches_controller_test.rb" "test/functional/stats_controller_test.rb" "test/functional/swagger_doc_test.rb" "test/functional/tag_controller_test.rb" "test/functional/talk_controller_test.rb" "test/functional/typeahead_api_test.rb" "test/functional/user_sessions_controller_test.rb" "test/functional/user_tags_controller_test.rb" "test/functional/users_controller_test.rb" "test/functional/wiki_controller_test.rb" "test/integration/I18n_test.rb" "test/integration/login_flow_test.rb" "test/integration/moderate_and_ban_test.rb" "test/integration/node_insert_extras_test.rb" "test/integration/node_series_tag_test.rb" "test/integration/node_unique_views_test.rb" "test/integration/node_update_test.rb" "test/integration/public_pages_test.rb" "test/integration/search_flow_test.rb" "test/integration/token_comment_test.rb" "test/integration/wiki_creation_test.rb" "test/performance/browsing_test.rb" "test/unit/admin_mailer_test.rb" "test/unit/answer_mailer_test.rb" "test/unit/answer_selection_test.rb" "test/unit/answer_test.rb" "test/unit/comment_mailer_test.rb" "test/unit/comment_test.rb" "test/unit/constants_test.rb" "test/unit/doc_result_test.rb" "test/unit/drupal_user_test.rb" "test/unit/helpers/admin_helper_test.rb" "test/unit/helpers/features_helper_test.rb" "test/unit/helpers/home_helper_test.rb" "test/unit/helpers/location_tags_helper_test.rb" "test/unit/helpers/map_helper_test.rb" "test/unit/helpers/notes_helper_test.rb" "test/unit/helpers/search_helper_test.rb" "test/unit/helpers/searches_helper_test.rb" "test/unit/helpers/tag_helper_test.rb" "test/unit/helpers/user_helper_test.rb" "test/unit/helpers/user_tags_helper_test.rb" "test/unit/helpers/wiki_helper_test.rb" "test/unit/node_shared_test.rb" "test/unit/node_tag_test.rb" "test/unit/node_test.rb" "test/unit/relationship_test.rb" "test/unit/revision_test.rb" "test/unit/subscription_mailer_test.rb" "test/unit/tag_result_test.rb" "test/unit/tag_test.rb" "test/unit/typeahead_service_test.rb" "test/unit/user_tag_test.rb" "test/unit/user_test.rb"  /Users/ryzokuken/.rvm/gems/ruby-2.3.4/gems/ci_reporter_test_unit-1.0.1/lib/ci/reporter/rake/test_unit_loader.rb]
/Users/ryzokuken/Code/plots2/Rakefile:34:in `block (2 levels) in <top (required)>'
/Users/ryzokuken/.rvm/gems/ruby-2.3.4/bin/ruby_executable_hooks:15:in `eval'
/Users/ryzokuken/.rvm/gems/ruby-2.3.4/bin/ruby_executable_hooks:15:in `<main>'
Tasks: TOP => test:all
(See full trace by running task with --trace)

~/Code/plots2 master* 1m 2s
❯ tree | grep report
│       │   ├── jasmine-console-reporter.js

[ryzokuken]

@jywarren Ugh, take a look at this: ci-reporter/ci_reporter#106

[ryzokuken]

@jywarren I submitted a patch, please take a look.