Shippable can't find testresults and testcoverage artifacts

[orendon]

Test artifacts were generated properly, but Shippable can't find them

after_script output:

The files were created correctly, as seen on the screenshot, but Shippable keeps showing this on build details:

• We could not find any test results in your shippable/testresults folder.
• We could not find any code coverage results in your shippable/codecoverage folder.

[rageshkrishna]

@orendon We do not currently support html output for code coverage and unit tests. Code coverage needs to be in Cobertura formatted XML (or SimpleCov CSV for Ruby), and jUnit XML format for unit tests. Would it be possible for you to modify your project to output these formats and try the build again?

[orendon]

@rageshkrishna just changed my formatters and it worked great, thanks 👍

Btw, you should add this to Shippable documentation

[ddarbyson]

@rageshkrishna I've been fighting with Code Coverage for Ruby. Can you please describe how you accomplished this?

[rageshkrishna]

@ddarbyson We have a sample that does this: https://github.com/Shippable/sample_ruby

You need to use simplecov and tell it to output the results as a CSV file in to the ./shippable/codecoverage folder. The sample sets an environment variable (COVERAGE_DIR) which is passed to simplecov from ./spec/spec_helper.rb

[ddarbyson]

@rageshkrishna i've added the sample to my spec_helper.rb, however i'm getting the following:

bundle exec rspec --format RspecJunitFormatter --out shippable/testresults/rspec.xml
Coverage report generated for RSpec to /home/minion/projects/53e19c57b92d70e8032b34d7/code/shippable/codecoverage/results.csv

When I click on Coverage inside my Shippable Lastest build status there are no reports. We could not find any code coverage results in your shippable/codecoverage folder. The Shippable Documentation say's it requires an XML file.

Any ideas?

[rageshkrishna]

Ruby is a special case for us and we do try to pick up .CSV files. I'll update the documentation for this.

The logs are telling me that the minion did indeed find some XML files inside the code coverage folder. It's possible that we're not properly cleaning up the shippable folder between builds. Can you please try adding this to your YML to confirm:

before_script:
  - rm -rf shippable/*

[ddarbyson]

Removing the shippable directory would be good for your docs. Yesterday when I started using Shippable for the first time, I had a couple trial and error runs to get up and running. I ended up hitting some errors that could only be fixed once I destroyed the minion and started re-tracking, and testing from Build #1. Anyway, thought I'd share that with you as rm -rf shippable/* would have quickly resolved my issue yesterday.

Back to my current issue, i'm afraid the before_script: - rm -rf shippable/* did not work. There are still no reports to view.

Here is an output of my results directory:

ls -la shippable/codecoverage
total 20
drwxrwxr-x 2 minion minion 4096 Aug 7 03:29 .
drwxrwxr-x 4 minion minion 4096 Aug 7 03:29 ..
-rw-rw-r-- 1 minion minion 51 Aug 7 03:29 .last_run.json
-rw-rw-r-- 1 minion minion 63 Aug 7 03:29 results.csv
-rw-rw-r-- 1 minion minion 74 Aug 7 03:29 .resultset.json
-rw-rw-r-- 1 minion minion 0 Aug 7 03:29 .resultset.json.lock

[ddarbyson]

Okay, figured it out, my results.csv file did not contain any results due to some internal oversight.

Thanks for helping out, maybe you could add some verbosity warnings like Results Set Empty instead of We could not find any code coverage results in your shippable/codecoverage folder.

[rageshkrishna]

@ddarbyson Glad this worked. We're currently working on a brand new release and all of these issues will be addressed. Thanks for the feedback.