Incompatible with [Template]

[kumy]

Describe the bug
RobotEyes save the reference pictures as the same name when Test Cases use the template functionality.

To Reproduce
Steps to reproduce the behavior:

1. Example test case:

Library         SeleniumLibrary  timeout=10  implicit_wait=0
Library         RobotEyes

*** Test Cases ***
Select user banner
    [Template]    Select banner ${templateId}
    1
    2
    3

*** Keywords ***
Select banner ${templateId}
    Sign In ${USER_1.name} Fast
    Go To Url                               ${PAGE_USER_1_BANER_TEMPLATE_URL}
    Select Radio Button                     ${USER_BANER_TEMPLATE_CHOOSER_RADIO_GROUP}    ${templateId}
    Click Button                            ${USER_BANER_TEMPLATE_CHOOSER_SUBMIT_BUTTON}
    Flash message shown                     Your user banner template preference has been successfully saved.

    Open Eyes                               SeleniumLibrary  5
    Wait Until Element Is Visible           ${USER_PROFILE_STATPIC_IMAGE}
    Capture Element                         ${USER_PROFILE_STATPIC_IMAGE}
    Compare Images

2. Start Robot Framework
3. There no distinction in pictures saved in visual_images/ during test loop.

$ tree visual_images/
visual_images/
└── Select_user_banner
    └── img1.png

Expected behavior
Reference images should have been saved independently for each template arguments loop.

Screenshots
N/A

Desktop (please complete the following information):

$ python --version
Python 3.8.2

$ pip freeze|grep eye
robotframework-eyes==1.4.1

Additional context
Here is output.xml from this test.
output.xml.txt

[jessezach]

@kumy There is an optional name argument for the capture keywords. However that will not address the issue.
RobotEyes works on distinction between test case names. It's not really designed for data driven tests.
The issue here is that all test cases probably have the same name. I will take a look at how to address this and push this in the next release

[jessezach]

I was able to fix this issue here #69
I tested it a bit with a mix of regular and template type tests and also with pabot. The fix seems to work. Here's the screenshot

The duplicate test name rows are Template type tests.

[jessezach]

@kumy The above fix adds an optional argument to open eyes keyword called template_id. This is supposed to be a unique string which gets appended to the testcase folder to make it unique. For template type tests passing a unique value to template_id will RobotEyes compatible.
Does this seem like something you would be okay with? Here's how a template test could look like

Library  SeleniumLibrary
Library  RobotEyes

*** Test Cases ***
Sample test6
    [Template]   Sample keyword
    https://www.google.com/   0
    https://www.google.com/   1
    https://www.google.com/   2


*** Keywords ***
Sample keyword
    [Arguments]  ${url}  ${uid}
    open browser  ${url}  chrome
    open eyes  SeleniumLibrary   template_id=${uid}
    sleep  3
    capture element  id=hplogo
    capture element  id=body  50
    compare images
    close browser

[kumy]

Ho great! Thanks a lot @jz-jess.
Once merged into branch v1.5.0 I'll try those new fixes.

[jessezach]

@kumy Just to add to my previous comment. I could actually remove the need for passing in a template_id completely and append something random to the test case folder if the test is of Template type. However that might result in creation of new random folders instead of overwriting existing ones on each test run atleast in you local machine.

Unless ofcourse you delete visual_images folder regularly to keep it clean. I'm fine either of the two approaches. I just felt maybe having the user enter a unique id would avoid having to create random new folders each time. What do you think?

[kumy]

I don't think writing files indefinitely with random names is a good option. If there is no automatic way of finding the template index id, then it's good to specify it ourselves.

Wondering how open eyes SeleniumLibrary template_id=${uid} and Capture Element ${element} name=img${uid} are different? Could the already existing name parameter be "fixed" to handle this?

When using name= I've got pretty nice looking results Acceptance Reports but only the last captured image is taken into account in the VisualReport

[jessezach]

@kumy the ${uid} value is just reserved for template_id argument in open eyes. We don't need to pass ${uid} to capture element keywords.

The name argument in capture keywords is only to give it a custom name. You could pass ${uid} to that. However it makes no difference when it comes to fixing template tests.

The fix is to keep the test case folders unique and not the image files within them. This way the images don't get overwritten and the report can show them as different tests which I think is a lot cleaner. Please refer my sample test above for usage of template_id.

The report only showed the last image as compared maybe because the unique image name was not found in the base folder?

[kumy]

It work fine with branch data-drive-report-fix. @jz-jess Thanks!

*** Test Cases ***
Select banner
    [Template]    Select banner
    1
    2
    3
*** Keywords ***
Select banner
    [Arguments]    ${templateId}
    Open Eyes                               SeleniumLibrary  5       template_id=${templateId}
    Scroll To Element                       ${USER_PROFILE_STATPIC_IMAGE}
    Wait Until Element Is Visible           ${USER_PROFILE_STATPIC_IMAGE}
    Capture Element                         ${USER_PROFILE_STATPIC_IMAGE}
    Compare Images

[jessezach]

Fix released in 1.5.0