Export a spec in plain text

[sarbogast]

It would be really nice to be able to export specs as a markdown document so that it's easier to communicate to business people what developers are verifying.

[supermarin]

specs like the whole code, or just names?

[sarbogast]

Just the text in the describe, context and it clauses.

[modocache]

Kind of like rspec --format documentation, right? Given the following spec:

describe Car do
  context 'when driving down the highway' do
    it 'drives fast ' do
    end
    context 'but behind a crown vic' do
      it 'drives slow' do
      end
    end
  end
end

RSpec's documentation formatter outputs:

Car
  when driving down the highway
    drives fast
    but behind a crown vic
      drives slow

Finished in 0.00092 seconds
2 examples, 0 failures

We can add this sort of functionality along with custom output formatting, which is part of the roadmap for Kiwi 3.0 (see issue #306).

I have an open RFC for output formatters issued as pull request #449--it would be trivial to add a documentation formatter to that branch. If you'd like to try it out, @sarbogast, I can send you some instructions once I've added a documentation formatter.

[supermarin]

Not sure if this is a testing framework responsibility tho;
XCPretty can generate you a nice HTML, but it should be fairly easy to add Markdown output as well

[modocache]

If it's a question of which formatters to include with a testing framework such as Kiwi, then I agree that we can't include them all. I'd be fine with just including JSON and plain text. Users can create and distribute their own formatters for XML, HTML, Markdown, a progress bar, nyancat format, or whatever.

However, I think a testing framework absolutely is responsible for its own output.

Of course, everyone wants different output. Some people want human-readable output, some want XML. @sarbogast wants markdown. Tools like XCPretty are necessary because xcodebuild doesn't provide a way to format its output. This forces developers to build a parser that relies on the raw output. By not providing an "official" method, Apple can either:

1. Recognize that tools like XCPretty now rely on an implicit contract, and thus never drastically change the output of xcodebuild.
2. Not give a damn, do whatever it wants, and break XCPretty whenever it pleases.

I think it's pretty obvious which of the two Apple will choose.

Luckily, we can do better.

RSpec, for example, provides an "official" hook for custom formatting. RSpec provides users with an explicit contract. This contract guarantees that RSpec will send custom formatters a well-defined set of messages during test execution. Formatters can print whatever they like based on those messages. If the messages ever need to change, RSpec can gradually deprecate them. This makes maintainers and users happy: users know their formatters won't break, and maintainers can change the output of built-in formatters without worrying about breaking tools like XCPretty.

While it's certainly possible to parse RSpec output by piping it to an external tool, RSpec provides a more practical solution for doing so. In Kiwi's case, we can add the capability for custom formatting without changing the output XCPretty relies on. In other words, we can maintain that implicit contract while providing a better, more explicit one.