In The Html Report The Duration Of The Scenario Doesn't Include The Before Or After Hooks

[bhreinb]

Hi there,

If you look at the HTML report per the screenshot below:

the scenario has a duration of 2.537 while one of the before hooks has a duration of 3.445. It looks like the duration for a scenario is calculated by summing the durations of the steps. However when you look at the report per the screenshot the hooks are part of the scenario so I would think the duration for the scenario should include the hook steps thus making the duration of the scenario 5.996.

Thoughts?

[damianszczepanik]

Attach JSON file with steps you are talking about

[bhreinb]

Hi

Apologies in the delay getting this to you...I include a sample JSON file which is attached below in zip format.

testReport.json.zip

The steps in that test are

Given current user is set to "superuser" @tg:+BEYWATS_TEST                                                                                              0.002
And current user is set to "developer" @tg:-BEYWATS_TEST                                                                                                0.003
And the user sets variables                                                                                                                             0.039 
And "SOAP" request "Put_Employee_Contract_Reason" within web service catalogue "HCM_Implementation_Service/v33.0"                                       1.033
When the "SOAP" request is transmitted with the response saved to variable "createEmployeeContractReasonResp"                                          33.109
And the "STATUS" equals "OK" with "XPATH" applied on "createEmployeeContractReasonResp" to validate response integrity                                  0.027
And the user sets variables                                                                                                                             0.000
And "SOAP" request "Put_Employee_Contract_Reason" within web service catalogue "HCM_Implementation_Service/v33.0"                                       0.016
And the "SOAP" request is transmitted with the response saved to variable "updateEmployeeContractReasonResp"                                            0.441
And the "FAMILY_STATUS" equals "SUCCESSFUL" with "XPATH" applied on "updateEmployeeContractReasonResp" to validate response integrity                   0.007
And "SOAP" request "Get_Employee_Contract_Reasons" within web service catalogue "HCM_Implementation_Service/v33.0"                                      0.008
And the "SOAP" request is transmitted with the response saved to variable "getEmployeeContractReasonResp"                                               0.440
Then the "FAMILY_STATUS" equals "SUCCESSFUL" with "XPATH" applied on "getEmployeeContractReasonResp" to validate response integrity                     0.008

When this JSON is sent to this module the duration of the test scenario is 35.133 which equals the duration of the above steps listed above. However the test scenario duration doesn't include the time taken to execute the before (4.408) & after (0.006) hooks. Given that the report has the before and after hooks under the test scenario it should include them as part of the overall test scenario duration thus making the total duration for the test scenario to be 39.547 as opposed to 35.133. Hope this makes sense.

[damianszczepanik]

Once I calculate duration from hooks I see that overall duration is changed - what is expected but it will change duration for all projects that use duration for hooks and that makes then surprise :)

[damianszczepanik]

On the other hand, hooks could be treated not as the test part but rather provisioning and then the question is: should we calculate time for preparation as the time for test?

[bhreinb]

Hello @damianszczepanik

I guess this is bit of a subjective opionion. To me it makes sense that hooks are part of the overall test duration/execution becuase ultimately you can't run the test in isolation without the provisioning.

In addition if I run the test via intellij the plugin that provides feeback on the test execution also includes the provisioning as part of the overall test execution. Please see the attachment for more details

So doing so probably aligns with the cucumber eco system per se.