Pretty format index out of range with Scenario Outline and Background step

[dasnook]

Pretty format works fine with Scenario Outlines in the feature file unless there is one (or more) Background steps. When I change the godogs example to look like this:

Feature: eat godogs with an outline
In order to be happy
As a hungry gopher
I need to be able to eat different amounts of godogs

Background: Start with a decent amount
Given there are 12 godogs

Scenario Outline: Eat various numbers of godogs
When I eat
Then there should be remaining

Examples: Leaving some behind
  | amount | left |
  | 5      | 7    |
  | 8      | 4    |
  | 11     | 1    |

Examples: Eating them all
  | amount | left |
  | 12     | 0    |

And then run godog against it with the default (pretty) formatter, I get this panic:

C:\Users\David\Documents\development\godog_outline>bin\godog.exe src\godog_outline.feature
Feature: eat godogs with an outline
In order to be happy
As a hungry gopher
I need to be able to eat different amounts of godogs

Background: Start with a decent amount

Scenario Outline: Eat various numbers of godogs # src\godog_outline.feature:9
When I eat
Then there should be remaining

Examples: Leaving some behind
  | amount | left |
  | 5      | 7    |
  | 8      | 4    |
  | 11     | 1    |

panic: runtime error: index out of range

goroutine 17 [running]:
github.com/DATA-DOG/godog.(_pretty).printOutlineExample(0xc08209c000, 0xc0820bc000)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/fmt_pretty.go:190 +0x195f
github.com/DATA-DOG/godog.(_pretty).printStepKind(0xc08209c000, 0xc082084e40)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/fmt_pretty.go:257 +0x132
github.com/DATA-DOG/godog.(_pretty).Undefined(0xc08209c000, 0xc0820884b0)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/fmt_pretty.go:297 +0x207
github.com/DATA-DOG/godog.(_Suite).runStep(0xc08209e0e0, 0xc0820884b0, 0x860028, 0xc08208a100, 0x0, 0x0)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/suite.go:201 +0xb5
github.com/DATA-DOG/godog.(_Suite).runSteps(0xc08209e0e0, 0xc0820ca310, 0x2, 0x2, 0x860028, 0xc08208a100, 0x860028, 0xc08208a100)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/suite.go:245 +0xc1
github.com/DATA-DOG/godog.(_Suite).runOutline(0xc08209e0e0, 0xc0820bc000, 0xc082092240, 0x0, 0x0)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/suite.go:298 +0x74e
github.com/DATA-DOG/godog.(_Suite).runFeature(0xc08209e0e0, 0xc082083700)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/suite.go:324 +0x351
github.com/DATA-DOG/godog.(_Suite).run(0xc08209e0e0)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/suite.go:169 +0xe6
github.com/DATA-DOG/godog.(_runner).run.func1(0xc082088320, 0xc08208b940, 0xc082083700)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/run.go:30 +0x14f
created by github.com/DATA-DOG/godog.(_runner).run
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/run.go:36 +0xd9

goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc08208832c)
c:/go/src/runtime/sema.go:43 +0x2d
sync.(_WaitGroup).Wait(0xc082088320)
c:/go/src/sync/waitgroup.go:126 +0xbb
github.com/DATA-DOG/godog.(_runner).run(0xc082088320, 0xc082088300)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/run.go:38 +0x109
github.com/DATA-DOG/godog.Run(0x6b2238)
c:/Users/David/Documents/development/godog_outline/src/github.com/DATA-DOG/godog/run.go:101 +0x82d
main.main()
C:/Users/David/AppData/Local/Temp/1453177288086389100godog.go:9 +0x2a
exit status 2

---

We are working on a fix; looks like the problem is in the counting of steps.

As an aside, I have a JUnit-style formatter that I wrote so that we can run our tests in CI. It works OK with outlines so this bug isn't holding us up. Any interest in having me add the JUnit formatter in a pull request? It seems to go along with the comments in bug #4.

[l3pp4rd]

that would be great @CodeWriterDavid concerning JUnit formatter and a fix to outlines. I personally do not use outlines as it somehow reads harder and that may be the reason for having it incomplete.
What is your opinion about #13 returning errors in hooks?

[dasnook]

OK, great! I like outlines when there are a lot of similar examples, as it makes missing coverage much more obvious. But I certainly understand how you wouldn't have noticed this issue if you don't use them. :) Regarding the returning of errors in Before/After hooks, I like the idea of having them be like any other step (and also more idiomatic in terms of Golang's error-centric approach), but depending on how those errors are handled by the runner, it might not make much of a practical difference. Before/After are for setting up state, and if there is a failure there isn't much you can do but grind to a halt. It doesn't make sense to try to run any additional tests, as the state of the domain system is now unknown and unreliable. The one area it might make a difference would be reporting, if returning errors instead of panics would allow the runner to better report the current failure and wrap up the reporting of the tests run up to that point. On the other hand, panics are pretty telling with a stack trace. :)

[l3pp4rd]

Yes, concerning errors on return, I agree, it does not make sense to try to continue running because these hooks are meant to manage/reset state. I think I'll keep it as is for now, until the obvious behavior pops in from context or someone proposes something to suit it better ;) thanks for the opinion

[l3pp4rd]

hey @CodeWriterDavid maybe you could open a pull request with jUnit formatter and I'll hunt the bug?

[dasnook]

Hi, @l3pp4rd -- I already have a fix for the bug, I was just hoping to have a set of tests to go with it and it was taking me a bit to figure out how to best do that with the existing test setup. Very cool, by the way, how you use godog to test itself! If you don't need tests, I can have a pull request fairly soon.

[l3pp4rd]

Ah great, thought that jUnit formatter can fade away if you are busy :) good to know you are still around, take your time ;)

[l3pp4rd]

hi @dasnook I've merged your junit implementation and adapted it based on Behat junit output they have spent much time to find nicest way of formatting results. Also I have fixed the issue where background was not being printed together with scenario outline. cheers

just run godog -f junit > junit.xml