Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve documentation formatter to show each step on a new line #228

Closed
wants to merge 2 commits into from
Closed

Improve documentation formatter to show each step on a new line #228

wants to merge 2 commits into from

Conversation

aramvisser
Copy link
Contributor

@aramvisser aramvisser commented May 10, 2020

When debugging failed tests, I like to run rspec with --format documentation to see as much info as possible when running tests. While using Turnip I was missing a way to see exactly which step failed.

So I created a new formatter based on the default RSpec documentation formatter to show each step on it's own line.

Before:
turnip_documentation_formatter_before

After:
turnip_documentation_formatter

The most obvious change is that each step is now on it's own line. This is to see in exactly which step an error occurs.

At the bottom where all failures are listed the actual step where the error occured is highlighted again, as a quick reminder.

Any examples not using Turnip will still use the default documentation style.

Some remarks:

  • I chose to extend the existing documentation formatter instead of creating a new class for it. This means that everyone using the default documentation formatter will get the new formatter without any work on their part.

    Is changing the default documentation output okay? I could see some people preferring the old style. If so, we should make this a new formatter that people will have to manually enable instead.

  • Highlighting the error step in the failures listing is done by name. This means if 2 steps have the same name and 1 has an error, both steps will be highlighted.

    I tried adding a "<!>" before the step name to indicate the failed step, but that wasn't very clear. And I tried adding a new line below the description that said: "Error in step: And raise error", but I preferred highlight the actual step instead.

@aramvisser
Copy link
Contributor Author

I see there are some errors with RSpec 3.7 and 3.8. I'll get a fix for that soon.

The full description for a Turnip scenario showes all steps next to each
other in 1 line (e.g. Given step 1 -> When step 2 -> Then step 3).

This commit changes the default documentation formatter to show each
step on it's own line and marking exactly which step failed.
@ranska
Copy link

ranska commented May 10, 2020

Very nice improvement 👍

When Turnip is loaded, but `rspec/core` is not required (outside of
tests usually), then the default documentation formatter doesn't exist.
Accessing it will raise an unitialized constant error.

This commit makes sure the default formatter is loaded before changing
it.
@aramvisser
Copy link
Contributor Author

@gongo Would you mind taking a look at this pull request?

I know changing the default output could have consequences for others users who rely on the current output staying the same. So I can understand if this pull request is not wanted. Either it's not in line with this project or maintenance might be too much.

In that case, would it be possible to at least include the changes made to /lib/turnip/rspec.rb? Namely sending events to the RSpec reporter and adding the steps to the example metadata.

That doesn't change anything for current turnip users, but it does allow me to improve on the documentation output as a new project.

If you would be okay with that, I'll create a new pull request for just those changes.

Thanks for your time.

@gongo
Copy link
Collaborator

gongo commented May 19, 2020

Hi @aramvisser . Sorry for the late reply 🙏
Thanks for the Pull Request! Very nice feature!


I want the formatter to be a separate gem. (e.g. https://github.com/gongo/turnip_formatter ).
However, --documentation formatter is a standard formatter in RSpec, I don't feel there's anything strange about turnip itself has this format. 😄

Please let me think about it for a bit.

@leoc
Copy link
Collaborator

leoc commented May 26, 2020

This is super interesting. Is this printing each step as it is performed or after the whole scenario is performed? I would love to get a feel of which step takes which amount of time relative to the scenario.

EDIT: I have a project, where I want to track the git commit/tag of the last change of a feature/scenario/step definitions so I wrote a little patch to hook into steps and nested steps. I have looked briefly into the pull requests code, and I wonder whether these hooks may be added to turnip too, for tracking sub-steps called via step()

Here is my monkey patch: https://gist.github.com/leoc/12caa2cc6abb508954121cd277c9b004

@sobrinho
Copy link
Contributor

sobrinho commented Nov 5, 2021

@aramvisser would you mind rebasing the branch against master?

There are merge conflicts right now.

cc @jnicklas

@aramvisser
Copy link
Contributor Author

@sobrinho I released this project as a new extension at https://github.com/aramvisser/turnip_documentation_formatter

It should work on the latest version of RSpec and Turnip, but hasn't been tested beyond that.

I'll add some better tests for it soon to make it work on officially supported versions. Also it monkey patches Turnip in some areas to make it work. I want to try to move those patches in Turnip itself, since I think they could be useful for other projects as well.

@aramvisser aramvisser closed this Nov 11, 2021
@sobrinho
Copy link
Contributor

@jnicklas @aramvisser I would like to see this as default (merged) into turnip, it makes the doc much better.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants