Cucumber strict

[klausbayrhammer]

Added --strict option
When running via CLI:

• strict == true: return 1 if there are undefined or pending steps
• strict == false: return 0 if there are undefined or pending steps

When running via JUnit-Executor

• strict == true: mark scenario and pending/undefined step as failed
• strict == false: mark pending/undefined step as ignored

[aslakhellesoy]

Thanks!

Your hasPendingSteps doesn't check for pending steps, but for undefined steps. It's undefined steps that cause snippets to be generated.

I'm not too fond of checking the snippets. If we decide to change or remove snippet logic it shouldn't cause the --strict behaviour to change.
And furthermore, it should fail if there are pending steps, not just undefined like now.

Also, the presence of --strict should cause JUnit to report missing and pending steps as failed instead of skipped.

There is more work to be done here.

[aslakhellesoy]

Thanks!

Your hasPendingSteps doesn't check for pending steps, but for undefined steps. It's undefined steps that cause snippets to be generated.

I'm not too fond of checking the snippets. If we decide to change or remove snippet logic it shouldn't cause the --strict behaviour to change.
And furthermore, it should fail if there are pending steps, not just undefined like now.

Also, the presence of --strict should cause JUnit to report missing and pending steps as failed instead of skipped.

There is more work to be done here.

[klausbayrhammer]

In the current implementation pending steps throw a PendingException which is added to Runtime#errors so they already finish with exit code 1. Should a non-strict execution finish with exit code 0 in case that there are pending steps?

[aslakhellesoy]

Yes, it should.