Data Driven Spec Runner #668

Open
yeugenius opened this Issue Oct 25, 2016 · 0 comments

Projects

None yet

1 participant

@yeugenius
yeugenius commented Oct 25, 2016 edited

It may be very useful to move Data Driven Testing from method level to Spec level and initialize Spec with sort of iterable collection containing data or references you need to test.
So you will run Spec one time for each entry in this collection and create corresponding report.

This also should change the order test executions - run all tests in Spec from top to bottom.
Example of desired behaviour:

@Share assetsIDs = []
def setupSpec(){
    assetsIDs = getAssetsIDsFromExternalStorage() 
}
@Unroll
def "ID=#id test_1"(){
    where:
    id << assetsIDs
}

@Unroll
def "ID=#id test_2"(){
    where:
    id << assetsIDs
}

current order of execution and report (imagine a mess when you have 100 of IDs with different test failures):

ID=1 test_1
ID=2 test_1
ID=3 test_1
ID=1 test_2
ID=2 test_2
ID=3 test_2

what do I need from Spec Data Driven (group by ID, not by test):

ID=1 test_1
ID=1 test_2 
ID=2 test_1
ID=2 test_2
ID=3 test_1
ID=3 test=2

OR (short form)

ID=1 OK
ID=2 Error
|--test_2 failed:
|--details:
ID=3 OK
  1. May combine with @Stepwise, when stepwise works separately for each run (for each ID in the testIDs).
  2. It may be even better to store each spec run report into separate file (parametrize report name by ID variable.)
  3. Short form. May report only spec result, no need to report each test run if it was OK, report only errors (not sure how to do it correctly, just a general idea. It may be a feature for report builder, not for Spock)
@yeugenius yeugenius changed the title from Data Driven Spec Runner to Data Driven Spec Runner [#proposed label enhancement or new feature] Oct 25, 2016
@yeugenius yeugenius changed the title from Data Driven Spec Runner [#proposed label enhancement or new feature] to Data Driven Spec Runner Oct 26, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment