Improve documentation formatter to show each step on a new line

[aramvisser]

When debugging failed tests, I like to run rspec with --format documentation to see as much info as possible when running tests. While using Turnip I was missing a way to see exactly which step failed.

So I created a new formatter based on the default RSpec documentation formatter to show each step on it's own line.

Before:

After:

The most obvious change is that each step is now on it's own line. This is to see in exactly which step an error occurs.

At the bottom where all failures are listed the actual step where the error occured is highlighted again, as a quick reminder.

Any examples not using Turnip will still use the default documentation style.

Some remarks:

• I chose to extend the existing documentation formatter instead of creating a new class for it. This means that everyone using the default documentation formatter will get the new formatter without any work on their part.

  Is changing the default documentation output okay? I could see some people preferring the old style. If so, we should make this a new formatter that people will have to manually enable instead.

• Highlighting the error step in the failures listing is done by name. This means if 2 steps have the same name and 1 has an error, both steps will be highlighted.

  I tried adding a "<!>" before the step name to indicate the failed step, but that wasn't very clear. And I tried adding a new line below the description that said: "Error in step: And raise error", but I preferred highlight the actual step instead.

[aramvisser]

I see there are some errors with RSpec 3.7 and 3.8. I'll get a fix for that soon.

[ranska]

Very nice improvement 👍

[aramvisser]

@gongo Would you mind taking a look at this pull request?

I know changing the default output could have consequences for others users who rely on the current output staying the same. So I can understand if this pull request is not wanted. Either it's not in line with this project or maintenance might be too much.

In that case, would it be possible to at least include the changes made to /lib/turnip/rspec.rb? Namely sending events to the RSpec reporter and adding the steps to the example metadata.

That doesn't change anything for current turnip users, but it does allow me to improve on the documentation output as a new project.

If you would be okay with that, I'll create a new pull request for just those changes.

Thanks for your time.

[gongo]

Hi @aramvisser . Sorry for the late reply 🙏
Thanks for the Pull Request! Very nice feature!

---

I want the formatter to be a separate gem. (e.g. https://github.com/gongo/turnip_formatter ).
However, --documentation formatter is a standard formatter in RSpec, I don't feel there's anything strange about turnip itself has this format. 😄

Please let me think about it for a bit.

[leoc]

This is super interesting. Is this printing each step as it is performed or after the whole scenario is performed? I would love to get a feel of which step takes which amount of time relative to the scenario.

EDIT: I have a project, where I want to track the git commit/tag of the last change of a feature/scenario/step definitions so I wrote a little patch to hook into steps and nested steps. I have looked briefly into the pull requests code, and I wonder whether these hooks may be added to turnip too, for tracking sub-steps called via step()

Here is my monkey patch: https://gist.github.com/leoc/12caa2cc6abb508954121cd277c9b004

[sobrinho]

@aramvisser would you mind rebasing the branch against master?

There are merge conflicts right now.

cc @jnicklas

[aramvisser]

@sobrinho I released this project as a new extension at https://github.com/aramvisser/turnip_documentation_formatter

It should work on the latest version of RSpec and Turnip, but hasn't been tested beyond that.

I'll add some better tests for it soon to make it work on officially supported versions. Also it monkey patches Turnip in some areas to make it work. I want to try to move those patches in Turnip itself, since I think they could be useful for other projects as well.

[sobrinho]

@jnicklas @aramvisser I would like to see this as default (merged) into turnip, it makes the doc much better.