FEATURE REQUEST:Ability to Parse spec files

[brywiser]

Hello! I am looking into switching my team over to gauge for our test runner from Lettuce(lettuce.it). Most of our test frameworks are written in Python, except for 1 in Java. One feature that our current test runner has, that I don't see in Gauge, is the ability to parse the test case file, and spit out some generic step implementations. So my request is for that ability.

###Requirements

• Pass a completed specification file without matching step implementations into gauge via some command(IE. gauge parse specs/spec_file_here)
• Prints out step definitions into the console that you can copy and paste into a step implementation file, saving time and eliminating misspellings

###Stretch Goals/Nice to haves

• Generate empty functions below each step definition with "assert false" for each step and something printed to the console(like ‘you must implement this’).
• If you want to use any data from the spec in your test, those are added as arguments to the corresponding step
• Add the specification name at the top of the result set as a comment
• Put the needed import statements above that(IE. from getgauge.python import step)
• Ask for a file path and save all the above info into a file in the specified directory

###Example(with stretch goals)

• This is from what our current test runner does, but the step implementations are the same, and this applies to Python as thats what I am familiar with. Something along these lines would be nice.
  "from getgauge.python import step

*************************************

GAUGE PARSER TEST

*************************************

@step('Given I am testing the gauge parser')
def given_i_am_testing_the_gauge_parser():
assert False, "You must implement this"

@step('When I pass a into gauge')
def when_i_pass_a_file_into_gauge(file):
assert False, "You must implement this"

@step('Then I see my boilerplate step implementations')
def then_i_see_my_boilerplate_step_implementations():
assert False, "You must implement this"

Gauge version

CHI-BBROUDE-MAC:~ bbrouder$ gauge -v
[DEPRECATED] This usage will be removed soon. Run gauge help --legacy for more info.
Gauge version: 0.9.0

Plugins

csharp (0.10.1)
html-report (4.0.1)
java (0.6.4)
python (0.2.0)
ruby (0.4.1)
spectacle (0.1.2)
xml-report (0.2.0)

Thank you! Please let me know if anything is unclear.

[brywiser]

I see the PR for this has been merged, and this is now closed. Is this implemented with the next release?

[sriv]

@brywiser - this is WIP, it got auto-closed because of this commit message:

Changing the flag name to hide-quick-fix #748, which github interpreted as a fix for #748.

This still needs to be tested by @sguptatw. Watch out for updates. Thanks for pointing us to this!

[sswaroopgupta]

This validation occurs in the execution cycle. In case of parallel execution multiple specs/scenarios/steps are executing together. How will this work in parallel runs?

[brywiser]

@sguptatw Is this question directed at me or the developers? If so, I am unsure how to answer it to be honest, when I suggested the feature, I didn't think about how it would be handled in parallel execution. Perhaps @sriv or @Apoorva-GA may have some insight into that. Thanks!

[Apoorva-GA]

Should be available in nightly >= 14/8/17

[Apoorva-GA]

@brywiser - regarding the feature we implemented.

If you have a unimplemented step in your spec/concept file. When you run gauge validate specs along with the validation error for unimplemented step it gives a suggestion with the step implementation stub for the unimplemented step. If you don't want to get suggestion please use the flag hide-suggestion like gauge validate --hide-suggestion specs.

The feature is available in the latest nightly. Please install the nightly to try out this feature and let us know your feedback.

[sriv]

@brywiser - @sguptatw 's question is addressed to the dev team.

@sguptatw - you are right, validation is a pre step to execution. However parallel run does not parallelize validation. So you should still see all the parse/validation errors first. Parallelization kicks in only while execution, and execution only considers specs that does not have errors.

[sriv]

related issue: #772

[brywiser]

@sriv and @Apoorva-GA So I made a sample spec file, with no matching steps implemented. It generates the validation error that the steps are not found, but thats all I see. I dont see a generic step implementation that I can use. See output here:

`CHI-BBROUDE-MAC:cobalt_gauge bbrouder$ gauge validate
Python: 2.7.12
[ValidationError] /Users/bbrouder/qa-gui-automation/cobalt_gauge/specs/test_spec.spec:8 Step implementation not found => 'Test Step 1'
[ValidationError] /Users/bbrouder/qa-gui-automation/cobalt_gauge/specs/test_spec.spec:9 Step implementation not found => 'Test Step 2'
CHI-BBROUDE-MAC:cobalt_gauge bbrouder$ gauge version
Gauge version: 0.9.2.nightly-2017-08-15

Plugins

csharp (0.10.1)
flash (0.0.1)
html-report (4.0.2)
java (0.6.4)
python (0.2.1)
ruby (0.4.1)
spectacle (0.1.2)
xml-report (0.2.0)`

[brywiser]

I dont see a nightly release for Aug 14, but that could something with timezones perhaps? So i have the 8/15 nightly install.

[brywiser]

Also just tried uninstall the different language plugins to see if that made a difference, same output as before:

`CHI-BBROUDE-MAC:cobalt_gauge bbrouder$ gauge version
Gauge version: 0.9.2.nightly-2017-08-15

Plugins

flash (0.0.1)
html-report (4.0.2)
python (0.2.1)
spectacle (0.1.2)
xml-report (0.2.0)
CHI-BBROUDE-MAC:cobalt_gauge bbrouder$ gauge validate specs/
Python: 2.7.12
[ValidationError] /Users/bbrouder/qa-gui-automation/cobalt_gauge/specs/test_spec.spec:8 Step implementation not found => 'Test Step 1'
[ValidationError] /Users/bbrouder/qa-gui-automation/cobalt_gauge/specs/test_spec.spec:9 Step implementation not found => 'Test Step 2'
CHI-BBROUDE-MAC:cobalt_gauge bbrouder$ `

[sriv]

@brywiser - Please try it with gauge-java nightly installed.

This feature requires changes to gauge as well as language runners. Currently the change is applied only to gauge-java (rest will follow).

Thanks for checking this out.

[brywiser]

Hey @sriv I downloaded and unzipped the file from the link u put above, is there instructions on where to install this to? Or where to save the zip file?

[brywiser]

@sriv I figured it out and confirmed it worked. But id like to do some more testing, but right off the bat, looks good!

[brywiser]

Hey @sriv Looks good! However, I cannot speak for Java, and I am unsure how it might get handled in python, but should the generic function name have somewhat unique title? At least in my experience, we cut and paste these and put whatever necessary test code in the step, but the names have never been the same. See screenshot below:

[Apoorva-GA]

@brywiser Thanks for the feedback. I have made changes for the generic function to have unique name. Also have a look at #772 where validation errors and suggestions are being separated.

Output of validation looks something like this :

[ValidationError] /Users/apoorvag/Desktop/foo/specs/example.spec:17 Step implementation not found => 'The word "gauge" has "3" vowels.'
[ValidationError] /Users/apoorvag/Desktop/foo/xyz.cpt:4 Step implementation not found => 'xyza'
[ValidationError] /Users/apoorvag/Desktop/foo/specs/example.spec:21 Duplicate step implementation => 'qwe'
[ValidationError] /Users/apoorvag/Desktop/foo/specs/example1.spec:5 Step implementation not found => 'abc'
One or more errors were due to : Step implementation not found => 'The word "gauge" has "3" vowels.'
Suggestion :
	@Step("The word <gauge> has <3> vowels.")
	public void implementation1(Object arg0, Object arg1){
		// your code here...
	}
One or more errors were due to : Step implementation not found => 'xyza'
Suggestion :
	@Step("xyza")
	public void implementation2(){
		// your code here...
	}
One or more errors were due to : Step implementation not found => 'abc'
Suggestion :
	@Step("abc")
	public void implementation3(){
		// your code here...
	}

[sswaroopgupta]

What is the expected behavior if the unimplemented step appears in multiple specs.

Actual:
The step validation errors are displayed at different places. The implementation is given only at one location.

[brywiser]

@Apoorva-GA thanks for the change. Is this deployed so I can update and test it out myself?

[Apoorva-GA]

Limiting the method names to 3 words from step text.
If there is no step text (step has only parameters), then method name will be implementation+uniqueNumber.

Should be available in gauge-java nightly > 31/08/17

[sswaroopgupta]

throw a notImplemented Exception - so that the suggested methods do not pass by default

[Apoorva-GA]

As discussed, method names will have all words from step text except parameters.
If there is no step text (step has only parameters), then method name will be implementation+uniqueNumber.

Also throwing UnsupportedOperationException with message Provide custom implementation so that the suggested methods do not pass by default.

Fix should be available in gauge-java nightly > 01-09-17

[sriv]

• method name in suggestion matches the step text

[ValidationError] /tmp/748/specs/example.spec:19 Step implementation not found => 'this is a sample unimplemented step'
Add the following missing implementations to fix `Step implementation not found` errors.


	@Step("this is a sample unimplemented step")
	public void thisIsASampleUnimplementedStep(){
		throw new UnsupportedOperationException("Provide custom implementation");
	}

• method name in suggestion removes param name

[ValidationError] /tmp/748/specs/example.spec:19 Step implementation not found => 'some "foo"'
Add the following missing implementations to fix `Step implementation not found` errors.


	@Step("some <foo>")
	public void some(Object arg0){
		throw new UnsupportedOperationException("Provide custom implementation");
	}

• if step text contains only parameters, method name should be implementation<number>

[ValidationError] /tmp/748/specs/example.spec:19 Step implementation not found => '"foo"'
[ValidationError] /tmp/748/specs/example.spec:20 Step implementation not found => '"foo" "bar"'
Add the following missing implementations to fix `Step implementation not found` errors.


	@Step("<foo>")
	public void implementation1(Object arg0){
		throw new UnsupportedOperationException("Provide custom implementation");
	}

	@Step("<foo> <bar>")
	public void implementation2(Object arg0, Object arg1){
		throw new UnsupportedOperationException("Provide custom implementation");
	}

• default implementation should throw UnsupportedOperationException [JAVA ONLY].

Other languages should throw equivalent errors/exceptions.

[brywiser]

@sriv and @Apoorva-GA I dont know if this is still WIP? or should be closed. So I did some testing with the java nightly release that @Apoorva-GA mentions above. Found an issue, If you have multiple steps that are just an argument, you only get the first one it finds. Not all of them. See below:

CHI-BBROUDE-MAC:test bbrouder$ gauge run [ValidationError] /Users/bbrouder/test/specs/test.spec:4 Step implementation not found => 'This should work' [ValidationError] /Users/bbrouder/test/specs/test.spec:5 Step implementation not found => '"stuff"' [ValidationError] /Users/bbrouder/test/specs/test.spec:6 Step implementation not found => 'If it doesnt' [ValidationError] /Users/bbrouder/test/specs/test.spec:7 Step implementation not found => 'I swear if they take my "stapler" I cant even' [ValidationError] /Users/bbrouder/test/specs/test.spec:8 Step implementation not found => '"freakout"' [ValidationError] /Users/bbrouder/test/specs/test.spec:9 Step implementation not found => 'Did this one appear?' [ValidationError] /Users/bbrouder/test/specs/test.spec:10 Step implementation not found => 'I wonder if this one will too' Add the following missing implementations to fix Step implementation not found` errors.

@Step("This should work")
public void thisShouldWork(){
	throw new UnsupportedOperationException("Provide custom implementation");
}

@Step("<stuff>")
public void implementation1(Object arg0){
	throw new UnsupportedOperationException("Provide custom implementation");
}

@Step("If it doesnt")
public void ifItDoesnt(){
	throw new UnsupportedOperationException("Provide custom implementation");
}

@Step("I swear if they take my <stapler> I cant even")
public void iSwearIfTheyTakeMyICantEven(Object arg0){
	throw new UnsupportedOperationException("Provide custom implementation");
}

@Step("Did this one appear?")
public void didThisOneAppear?(){
	throw new UnsupportedOperationException("Provide custom implementation");
}

@Step("I wonder if this one will too")
public void iWonderIfThisOneWillToo(){
	throw new UnsupportedOperationException("Provide custom implementation");
}`

[Apoorva-GA]

@brywiser If you have multiple steps that are just an argument, then the steps are same. so the suggestion is shown only once.
Where as if steps have only arguments and the number of arguments differ then it will have different suggestions.

[brywiser]

Thank you everyone for working through this! @Apoorva-GA It looks good. I am assuming it is now available for java? Then will be rolled out to other languages soon?

[zabil]

@brywiser ideally we'll try to see if users are finding this useful.

This has to be implemented in the respective language runners.
However, if you want to speed track this for Python we can help you out with implementing this. We always welcome user contributions and provide support in building it.

[sriv]

It looks like @kashishm has added this in gauge-python.

[brywiser]

Thank you @kashishm! And @zabil I am actually very interested in becoming a contributor, I do have another feature that I want to add, and since I am getting my team onto gauge, me being more involved would be good. I am going to start a class in Golang, which I have never used before(that is what gauge is written in?)

[nehashri]

@brywiser It would be good to have you as a contributor. 😄
Gauge core and some of the plugins are written in Golang, whereas the language plugins are written in their respective languages. You can find the technical documentation here. Feel free to reach out to us in case of any queries/issues.

[brywiser]

@nehashri
Sure! Id love to help anyway I can :). Ill take a look through the technical documentation and get familiar with Golang as well. Is there anything else id need to do or fill out regarding this?

[nehashri]

@brywiser That should be a good start. If you have any queries, please do reach out to us.

[kashishm]

@brywiser This feature is available in Gauge Python 0.2.2. To update, run the following commands.

gauge update python
pip install getgauge --upgrade

[brywiser]

Thanks @kashishm! I just upgraded, looking forward to using this, my team will also be excited to as we are switching all our tests over now :)

[zabil]

Ill take a look through the technical documentation and get familiar with Golang as well

@brywiser it's better to go through it and touch only parts required by a feature that you'd like to implement.
We do have issues that need a helping hand, but since you are introducing Gauge to your team you'll have feature ideas Gauge can use.

[brywiser]

@zabil Sure! Im excited to see what more awesome features we can introduce into gauge.