TestScenario-less testing

[mirogta]

Is your feature request related to a problem? Please describe.

I love the way steps in PowerShell tests using Pester/Gherkin are matched. You call Given/When/Then funcs and pass the regex matching string as the first parameter. See https://powershellexplained.com/2017-03-17-Powershell-Gherkin-specification-validation/

Compared to go-bdd or godog as well in a similar way, both need code where you match the English text from the feature files to a func. Sure, this code can be generated or suggested (as godog does), but it's just noise.

See

func add(ctx context.Context, var1, var2 int) error {
	res := var1 + var2
	ctx.Set("sumRes", res)
	return nil
}

# can we get rid of this?
func TestScenarios(t *testing.T) {
	suite := NewSuite(t)
	suite.AddStep(`I add (\d+) and (\d+)`, add)
	suite.Run()
}

Describe the solution you'd like

What I would like to achieve in a "better" Go BDD test library is to avoid that extra func.

Would it be possible to have the regex text part of the func that implements it, something like:

func add(given('I add (\d+) and (\d+)'), ctx context.Context, var1, var2 int) error {
...
}

# no func TestScenarios needed

or inside the TestScenarios?

func TestScenarios(t *testing.T) {
  given('I add (\d+) and (\d+)', func(ctx context.Context, var1, var2 int) error {
    ...
  })
  ...
}

Describe alternatives you've considered
N/A

Additional context
This would make this so much awesome-er than godog ;-)

[bkielbasa]

I've been thinking about it for a while. We need more discussion on that.

Firstly, there has to be an instance of GoBDD alive, somewhere. What comes to my mind is "global" instance initialized in the library's init() function.
The first proposition won't compile and I don't think there's a way of making similar things in Go. I have an idea of how to make it differently. But, I'm not sure if it's the right path.

We have to autoload/register those functions somehow. What we can do is parsing test functions looking for functions + comments in a certain format. Something ala annotations.

// GoBDD I add (\d+) and (\d+)
func add(ctx context.Context, var1, var2 int) error {
...
}

As I said previously, it doesn't look a very Go's way of doing stuff.

The second option sounds doable but there are more questions to answer like:

• where to keep registered steps? Global state sounds a good place like in the previous example.
• is there any way of resetting those steps after Text*() execution?

We can talk about it but I don't think that such functionality will land in v1.0.

[sagikazarmark]

You can use so called "markers" as annotation like constructs. I used them in a project of mine for code generation.

Reference: https://book.kubebuilder.io/reference/markers.html

[mirogta]

You can use so called "markers" as annotation like constructs. I used them in a project of mine for code generation.

Reference: https://book.kubebuilder.io/reference/markers.html

That looks neat and certainly go-compatible. I've ran into such annotations/comments on https://magefile.org/ and it works really well there.

@bkielbasa any drawbacks you would think of using this approach? So far I can only think of positives :-)

[bkielbasa]

Thanks for your comments!

To be honest, I don't have any good reason to not do it.
I don't think we can get read of the TestScenarios completely. Imagine a situation where the user has only annotated steps and that's all. Where should be the trigger to scan the source code? There will be no gobdd imports.

func TestScenarios(t *testing.T) {
	NewSuite(t, WithAutoload()).Run()
}

IMO this is the minimal code needed to run such code.
Secondly, I think it should be disabled by default. I'm not a huge fan of doing a lot of magic by default.

[sagikazarmark]

@mirogta note that build tags are not the same as markers.

Build tags go to the beginning of a file and look like //+build sometag.

Markers are just comments and look like // +some:marker (notice the space)

@bkielbasa godog provides an executable for running scenarios without having to write a test method. Personally I prefer having an explicit test method though.

[bkielbasa]

@sagikazarmark

@bkielbasa godog provides an executable for running scenarios without having to write a test method.

can you give an example?

[sagikazarmark]

I'll admit, I don't know how that works internally.

[bkielbasa]

ok, I give the green light to do it but first of all, we have to have a more detailed specification. If you have any idea how to make it work just open an issue with the proposal (example #72).

[bkielbasa]

I'm closing it but I'll create a separate proposal in the future for markers