Excel input files: unable to execute tests by tag

[joergschultzelutter]

I have this very simple Robot Test which uses an Excel sheet (datadriver.xlsx) as data source:

*** Settings ***
Library         DataDriver   file=datadriver.xlsx
Test Template	DataDriver Demo Test Main

*** Variables ***
${JSON_BODY}	${EMPTY}

*** Test Cases ***
DataDriver Demo Test Call '${API_CALL}'
	[Documentation]	Run the test cases for method ${API_CALL}

*** Keywords ***
DataDriver Demo Test Main
	[arguments]    ${API_CALL}  ${VAR1}  ${VAR2}
	Run Keyword  DataDriver_Demo_Test_${API_CALL}  ${API_CALL}  ${VAR1}  ${VAR2}

DataDriver_Demo_Test_Call_One
	[arguments]    ${API_CALL}   ${VAR1}  ${VAR2}
	Log To Console		Call One

DataDriver_Demo_Test_Call_Two
	[arguments]    ${API_CALL}   ${VAR1}  ${VAR2}
	Log To Console		Call Two

The Excel text file contains multiple tags per test case.

RF recognises these tags and lists them correctly in the test's associated report file.

However, if I try to run a single test based on its tags value, the test cannot be found.

C:\Test_Cases\interneTestsAPI\DataDriver_Tag_Test>robot -i world_domination datadriver.robot
[ ERROR ] Suite 'Datadriver' contains no tests matching tag 'world domination'.

Try --help for usage information.

C:\Test_Cases\interneTestsAPI\DataDriver_Tag_Test>

My assumption is that this is a limitation with the Excel input file digestion as the -i option is triggered before you can actually examine the Excel input file and get the tags from there. I have consulted this article but can't really see an error with my code. Am I missing something here?

[ddufrasn]

I have myself always encounter the same issue as @joergschultzelutter . If you want a quick solution you can used the exclude option which work but it's pretty annoying as you need to exclude all except the one tag you need to use :-/
I.e.: -e checkORmodifyORupdate

[joergschultzelutter]

Some additional thoughts - I think that the root cause might be twofold:

• RF cannot consider the Excel file's tags as input parameters as they are located in the Excel file and not in the Robot Test. This would mean that rather deciding on whether a RF test suite contains valid tests (based on the respective tags value), RF would first need to run and execute the test file - and then make a conclusion on the RF Excel file's input data. Tags which reside in the RF test suite should work - but there is another obstacle which brings me to the second issue:
• Datadriver tests are keywords and not test cases. AFAIR, keywords associated with tags should not work with the --include parameter (and I mean this on their selection - the keywords do work but the tags parameter is simply ignored). Why: because RF usually executes tests and not keywords. In this case however, we have one single datadriver point-of-entry test which then results in 1..n keywords. So even if you can assign the tags to the datadriver tests somehow, these tests would still be keywords from a mere technical perspective.

I don't know if this issue can be solved by using RF"s listeners but for now, I don't see any workarounds which would enable me to run Excel-based tests based on their tags value.

[Snooz82]

https://github.com/Snooz82/robotframework-datadriver#filtering-with-tags

Have you read this in the docs?

To select tests by --include you have to add that test to the "template Test" in the robot file.
Otherwise the suite is cleaned by the parser before DataDriver will be started.

There is also an option when using DataDriver to configure how tags from the template are added to the generated TestCases.

*** Settings ***
Library    DataDriver    handle_template_tags=NoTags

Allowed Values

• ForceTags
• UnsetTags
• DefaultTags
• NoTags

You can configure how to handle tags from the template in generated tests:

ForceTags : Will add all tags of the template test to all data-driven test cases.
UnsetTags : Will add only these tags from template test to the tests, that are not assigned to any of the test cases.
DefaultTags : Will only add tags to data-driven test if no tag is set on that test.
NoTags : Will not add any tags from the template to date-driven tests.

[joergschultzelutter]

Yes, I did read the documentation :-) However, based my initial question, @ddufrasn 's comment and other inquiries related to the same topic I suggest enhancing the documentation on this issue as it was not really clear to all of us how this option really works.

For completeness sake, here is an updated version of my original example RF test suite. The additional Default Tags line now enables to filter on a per-test basis via the --include option (the Excel file has not been modified)

*** Settings ***
Library         DataDriver   file=datadriver.xlsx    handle_template_tags=DefaultTags
Test Template	DataDriver Demo Test Main
Default Tags    check    update    modify    world_domination

*** Variables ***
${JSON_BODY}	${EMPTY}

*** Test Cases ***
DataDriver Demo Test Call '${API_CALL}'
	[Documentation]	Run the test cases for method ${API_CALL}

*** Keywords ***
DataDriver Demo Test Main
	[arguments]    ${API_CALL}  ${VAR1}  ${VAR2}
	Run Keyword  DataDriver_Demo_Test_${API_CALL}             ${API_CALL}  ${VAR1}  ${VAR2}

DataDriver_Demo_Test_Call_One
	[arguments]    ${API_CALL}   ${VAR1}  ${VAR2}
	Log To Console		Call One

DataDriver_Demo_Test_Call_Two
	[arguments]    ${API_CALL}   ${VAR1}  ${VAR2}
	Log To Console		Call Two

(.venv) C:\Test_Cases\DataDriver_Tag_Test>robot -i modify datadriver.robot
==============================================================================
Datadriver
==============================================================================
Test Case 2 :: Do even more                                           Call Two
Test Case 2 :: Do even more                                           | PASS |
------------------------------------------------------------------------------
Datadriver                                                            | PASS |
1 test, 1 passed, 0 failed
==============================================================================
Output:  C:\Test_Cases\DataDriver_Tag_Test\output.xml
Log:     C:\Test_Cases\DataDriver_Tag_Test\log.html
Report:  C:\Test_Cases\DataDriver_Tag_Test\report.html