Gherkin Json formatter does not include a result for scenario outlines #165
Comments
|
Confirming issue in 2.10.0 |
|
I'll reopen this issue when someone provides some code that reproduces this claimed (mis)behaviour? |
|
I am able to reproduce this cucumber-jvm 1.0-9 (not sure which gherkin build it picks up as depdendency). With cucumber-jvm-1.0-8, I am observing that json reports has "result" reported for each step. But with cucumber-jvm-1.0-9:, json reports has "result" reported only for last step. This is for both "Scenario" and "Scenario Outline". Feature file is very straight forward: @WIP
Feature: Add two numbers
Scenario Outline: Add two numebrs
Given two numbers <number_1> and <number_2>
When I add them
Then Result is <number_3>
Examples:
|number_1|number_2|number_3|
|2 |3 |5 |
|1 |2 |3 |
Scenario: Add two numebrs
Given two numbers 1 and 1
When I add them
Then Result is 2JSON with cucumber-jvm-1.0-8: {
"id": "add-two-numbers;add-two-numebrs;;2",
"tags": [
{
"name": "@WIP",
"line": 1
}
],
"description": "",
"name": "Add two numebrs",
"keyword": "Scenario Outline",
"line": 11,
"steps": [
{
"result": {
"duration": 130673000,
"status": "passed"
},
"name": "two numbers 2 and 3",
"keyword": "Given ",
"line": 5,
"match": {
"arguments": [
{
"val": "2",
"offset": 12
},
{
"val": "3",
"offset": 18
}
],
"location": "AddNumbersSteps.initializeNumbers(int,int)"
},
"matchedColumns": [
0,
1
]
},
{
"result": {
"duration": 776000,
"status": "passed"
},
"name": "I add them",
"keyword": "When ",
"line": 6,
"match": {
"location": "AddNumbersSteps.add()"
}
},
{
"result": {
"duration": 2300000,
"status": "passed"
},
"name": "Result is 5",
"keyword": "Then ",
"line": 7,
"match": {
"arguments": [
{
"val": "5",
"offset": 10
}
],
"location": "AddNumbersSteps.result(int)"
},
"matchedColumns": [
2
]
}
],
"type": "scenario"
},JSON with cucumber-jvm-1.0-9: {
"id": "add-two-numbers;add-two-numebrs;;2",
"tags": [
{
"name": "@WIP",
"line": 1
}
],
"description": "",
"name": "Add two numebrs",
"keyword": "Scenario Outline",
"line": 11,
"steps": [
{
"name": "two numbers 2 and 3",
"keyword": "Given ",
"line": 5,
"matchedColumns": [
0,
1
]
},
{
"name": "I add them",
"keyword": "When ",
"line": 6
},
{
"result": {
"duration": 2992000,
"status": "passed"
},
"name": "Result is 5",
"keyword": "Then ",
"line": 7,
"match": {
"arguments": [
{
"val": "5",
"offset": 10
}
],
"location": "AddNumbersSteps.result(int)"
},
"matchedColumns": [
2
]
}
],
"type": "scenario"I am just guessing problem might be in this commit in JSONFormatter.java. Let me know if you need anything else. Thanks, |
|
Oops sorry for weird formatting...did not expect to come out this way...hope you can read it... |
|
I fixed the formatting. Reopening since there is now a way to reproduce this. |
|
Very timely. Just ran in to this working on master thought's jenkins/maven json report interpreter. I have a pull request with a full maven project being run as a maven-invoker-test in his plugin (junit->cucumber-jvm->json report-> interpreter plugin) I can add similar tests anywhere you might like them in cucumber-jvm. I'm targeting the this because the JSON formatter will be simple to make thread safe, and the plugin will be simple to extend and configure to break builds on some configurable number of work in progress @tags, number of steps throwing pending exceptions, number of features w/ pending exceptions, etc. |
|
Fixed in which version? I still have this bug in v2.11.1 |
|
I have a fix in my pending change list to gherkin. I also added tests in a pending change to cucumber-jvm. So, not just yet. |
|
Did this fix ever get committed? Still waiting on this. Thanks |
|
I too am experiencing this issue |
|
Hi any update on this - we've just installed jenkis plugin and discovered this. |
|
I started working on it on a branch, but nothing finished yet. Hang in there. |
|
Glad to hear! Really appreciate this awesome tool! |
|
can the duration be captured in the result when making the fix? |
|
Hi, that would be really great to have this fix! |
|
I was almost disappointed thinking that the cucumber-reporting was not working. It really only works with this fix. |
|
Is there any update on the Issue #350 that in the JSON report it gives "Result was missing for this step" for all the steps except the last step of the scenario. |
|
In order to make a clean fix for this I want to make a big refactoring to gherkin. It might take a while. I'll report here when there is progress. |
|
Sounds interesting @aslakhellesoy. Care to elaborate? |
|
I'm pretty excited for your fix @aslakhellesoy ! |
|
Looking forward to your fix @aslakhellesoy ! |
|
When can we expect this fix available? Anxiously waiting for it as this has become a major issue for us. |
|
Agree. This is major issue for me as well. Looking forward to the fix. |
|
@aslakhellesoy ..... I know you are very busy and you have been very helpful when it comes to answering questions online about JVM Cucumber. Thanks for that. Do you know when you 'd make and commit the fix ? I am using Cucumber JVM 1.1.1, are there any drawbacks of going back to JVM 1.0.8 ? |
|
I don't know when I'll have time to fix this. Both work and private life demands a lot of my time, and I tend to do open source work whenever I get a free evening or two. When this happens is so hard to predict that I'd rather not say. You can assess the drawbacks of downgrading yourself by examining History.md in gherkin and cucumber-jvm. |
|
Ok, thank you sir. We look forward to having a fix in the near future |
|
@AbezarBaker If you get the This modified Ant file for the examples/java-helloworld works for me: |
|
@brasmusson Hi,some success i was able to kick of my test using junit runner with the help of your ANT junit snippet and the JSON result was correct! YooHoo!. Problem is that it runs with some hardcoded cucmebr options in my runner class as shown above @CucumberOptions( features = "release/features/Briefcase",//path to the features If i want to override these by passing them from the ANT file to the runner class then do i need to remove the lines above from the runner class or it will just over look it? Right now im stuck trying to pass the cucmber options from ANT junit task.Below is my code Even though i am trying to set the cucumber options above ,the test runs with the options in the runner class.Any ideas if my code to pass the cucumber options is correct ?Appreciate your help! When i remove the cucmber options in the runner class i get the run output below.Seem like the glue parameter is being ignored [junit] Testsuite: com.tr.qa.selenium.bdd.stepdefs.GenericRunner |
|
@AbezarBaker A couple of things. The two first |
|
@brasmusson the typo and the line of code you suggested above did the trick.Thanks a lot for your help!! @RunWith(Cucumber.class) public class GenericRunner { } The reason i ask is because if i leave the cucumber options annotation in the runner class then after the junit runner runs,in the target folder i see two sets of reports (https://f.cloud.github.com/assets/6143899/2241336/36fbfb24-9ccc-11e3-9fb7-d04f9c3fbe8d.png) |
|
Just wanted to say that I also get the "Step is missing Result:..." when using "Scenario Outline:" but in my case I use TestNG in combination with Cucumber. Should I create a new issue for that special case with TestNG or comment here? |
|
@AbezarBaker The values in the |
|
@brasmusson thanks for the information .Cheers! |
|
Using Ruby Cucumber. Following the thread it appears there is a Cucumber-JVM fix. Is there a plan or recipe whereby the fix can be made available to the Ruby community? Within Jenkins this eliminates the ability to use the standard Cucumber reporting plugins. Work around is a custom .feature file processor to turn scenario outline pattern into basic scenarios. |
|
@akellet, good idea. Do you have a custom .feature file processor that you could share? |
Sometimes the json formatter does not include results in step elements of a Scenario Outline. For example, see: cucumber/gherkin#165 This caused an NPE when generating reports: org.apache.velocity.exception.MethodInvocationException: Invocation of method 'getName' in class net.masterthought.cucumber.json.Step threw exception java.lang.NullPointerException at templates/featureReport.vm[line 210, column 25] This looks like a regression introduced in 906b9b9.
|
+1 for fix on Ruby Cucumber. |
|
Also looking for a fix for this in Ruby Cucumber |
|
For anyone who wants a quick-and-dirty way to get cucumber reporting to report the results for outlines, we've been using this with some success: It basically converts each row of a scenario outline into a regular scenario in the JSON output. Not perfect, but works for us! |
|
Thx for this workaround @blt04. I seriously needed something like that, since nobody bothered to fix this for the last two years (srsly wtf), yeah I know it was fixed for java, but not ruby, bleh. |
|
Thank you for this workaround @blt04. This really saved me lot of time. Appreciate it! |
|
@blt04 Appreciate for this support, while one issue during i use the json_expanded_formatter.rb is: I use cucumber version is 1.3.14 anything i do wrong? waiting online. |
|
@DoubleJ2011 your question is unrelated to this bug report. Please ask general questions on the cukes google group. |
|
@aslakhellesoy actually my question is the same |
|
@DoubleJ2011 Oh yes I see that now. You included so much unrelated output (ANSICON, class variable access warning) that I thought that's what you meant. Please only include relevant info, and format your code/logs so it's readable please. |
|
oh, sorry for my mistaken. take care next time and expect to the fix of the issue thank you 来自 MEIZU MX -------- 原始邮件 --------
|
|
@akellett @ShoopIndy @faraazkhan @Chuckv @blt04 @FrostbittenKing @nbollepalli @DoubleJ2011 FYI, cucumber version 2.0.0.beta.1 will produce json files compatible with the cucumber reporting tools when using the |
|
Is there a javascript version of this fix as well? |
and others
Gherkin 2.9.0.
I'll add some concrete examples to this but I suspect that the JSON formatter does not output the result for scenario outlines.
For scenarios we get:
{"steps"=>[{"result"=>{"status"=>"passed"}, ....., "type"=>"scenario", "description"=>"", "keyword"=>"Scenario"}Where result indicates the step passed.
For scenario outlines:
{"examples"=>[{"name"=>"", "line"=>114}], ....}There is nothing about the status of the examples/rows/cells.
The text was updated successfully, but these errors were encountered: