There is only one failed test in report

[filipehb]

Describe the bug
After executing suit with 2 tests or more, the report of all tests isn't generated.

To Reproduce
Steps to reproduce the behavior:

1. Create two test cases.
   
   
   
2. Execute tests.
3. See report.

Expected behavior
All tests should be in report.

Screenshots

Report

Desktop (please complete the following information):

• OS: Win 10 Pro
• Browser Chrome
• Version 72.0.3626.119

Additional context

[filipehb]

@jz-jess, I can analyze this problem and submit a pull request. What do you think?
test.zip

[jessezach]

@filipehb You can try running reportgen path/to/results folder(empty if not within a folder) to fix this issue.
The report autogeneration could be buggy, because it tends to delete report whose last modified at is more than 5 seconds. Feel free to analyse it though.

[jessezach]

@filipehb I think this could be happening because of the slowing down of selenium execution by using set selenium speed.

[jessezach]

@filipehb I have raised a PR with a possible fix since previous time difference calculation was wrong.
Can you please run your tests using this branch?
#24

[filipehb]

@jz-jess, i got this error sometimes, and visual report has the same behavior.

reports.zip

[jessezach]

@filipehb Thank you! Let me take a look at this issue. Keep you posted.

[jessezach]

@filipehb I was able to reproduce the issue. The issue is caused because of keyword set selenium speed 1. The slow execution results in time difference of more than 10 seconds between tests. Auto generation of report works in the following way.

• Since a new RobotEyes instance is created for each test, there is no way of knowing if the visualReport.html file belongs to the current test run.
• Code checks for the last modified time of visualReport.html and gets the difference between last modified and current time. If the difference is more than 5 seconds, it assumes that visualReport.html belongs to an older test run. It deletes the file and creates a new file. The current test result gets written to it.

In your case the difference crosses 10+ seconds between each test, which is why you only see the last test in the execution in the report. I would suggest you to run an alternative command to generate the report manually. reportgen path/to/result folder or reportgen if there is no result folder.

You can also view an interactive report by running eyes /path/to/results or eyes if there is no result folder.

[filipehb]

@jz-jess, i remove the keyword set selenium speed 1 and timeout=5 parameter of Wait Until Element Is Visible, but the issue continues to me.

[jessezach]

@filipehb I could not reproduce the issue after removing set selenium speed.
You don't have to remove timeout of wait until element is visible.

Did you try using the fix in the pull request?

[filipehb]

@jz-jess, i got this error if use PR:

[jessezach]

@filipehb I have fixed that issue. Please pull the code and try once again.

[filipehb]

@jz-jess, sorry i didn't see the new commit. The TypeError problem is solved, but in the report i got the same issue.

reports.zip

[jessezach]

@filipehb okay. After you run your tests, please run command reportgen <results folder> or reportgen if no results folder. This should generate a fresh report. Does this solve your issue?

[jessezach]

@filipehb I have pushed another commit which doubles the threshold. Please give it one last shot and see if it fixes your problem. I dont see any issues in my system. I dont have a windows system

[filipehb]

@filipehb okay. After you run your tests, please run command reportgen <results folder> or reportgen if no results folder. This should generate a fresh report. Does this solve your issue?

Yes, solved :D. Thx!

@filipehb I have pushed another commit which doubles the threshold. Please give it one last shot and see if it fixes your problem. I dont see any issues in my system. I dont have a windows system

I will test with new PR.

[filipehb]

@jz-jess, nice PR! Issue resolved for me 🥇.

[jessezach]

@filipehb Great. Deploy this soon in the next release

[jessezach]

Fix released in version 0.64