Wrong view in case of suite that contains another suites

[ulmanA]

Hi,
We are using TestNG and we have suite.xml file that contains suite-file tags
So I expect to see in report portal something like:

   "My_Launch":[  
      "MultiSuite":[  
         "Suite1":[  
            "test1",
            "test2"
         ],
         "Suite2":[  
            "test3",
            "test4",
            "test5",
            "test6",
            "test7",
            "test8"
         ]
      ]
   ]
}

And My_Launch will have total of 8 tests

But what actual happens:

{  
   "My_Launch":[  
      "MultiSuite":[  
      ],
      "Suite1":[  
         "test1",
         "test2"
      ],
      "Suite2":[  
         "test3",
         "test4",
         "test5",
         "test6",
         "test7",
         "test8"
      ]
   ]
}

And the total tests is 9 (it counting the multi-suite as 1 more test)
(see attached screenshoot

)

so, there is 2 issues here:

1. The visual representation (the child suite is beside the parent suite)
2. The total tests and passed tests calculations are wrong

[evjlobanova]

@ulmanA Use different methods for start your suites:

• for the MultiSuite use POST /{projectName}/item - start root test item;
• for Suite1 and Suite2 - POST /{projectName}/item/{parentItem} - start a child test item

Please, try to do this, and share your results with us

[ulmanA]

I am not call directly to the API.
I am using the TestNG Listener: com.epam.reportportal.testng.ReportPortalTestNGListener
From the artifact: com.epam.reportportal : agent-java-testng

[avarabyeu]

@ulmanA could you please give us some example of your testng.xml file?

[ulmanA]

Here:
Suites.zip

[ulmanA]

Sorry closed by mistake

[pbortnik]

@ulmanA Hello, do you want to see your root suite as the parent of other suites? How important is to see the root suite?
The problem is that testNG runs inner suites at first and after then the root one to summarize results. ReportPortal creates items on runtime, so it is difficult to change the tests' structure after inner suites have finished.
The possible solution is to ignore root suite if it has only inner suites. If the root suite has test cases as well, so it could be placed with inner suites on the same structure's level. Does it work for you?

[ulmanA]

1. we have project here that run whole tree of suites (suite that have 4 suites that each have few inner suites and so on...).
   so with your solution we will lose all that tree (that represent separation by features for example )

2. Maybe you can add ability to update finished test item with new parent?

[DzmitryHumianiuk]

@avarabyeu @pbortnik
please check possibility to updated parents on Finish.
or consider it in statistics, based on PostgreSQL

[avarabyeu]

Depends on testng-team/testng#1656

[DzmitryHumianiuk]

@avarabyeu can we start automatically master object (startRQ), if child object of it started?

[avarabyeu]

@DzmitryHumianiuk doesn't understand. Could you please re-phrase?

[evjlobanova]

@DzmitryHumianiuk could you please add more details?

[DzmitryHumianiuk]

@evjlobanova @avarabyeu
as i see, related issue in testNG closed without any code change.
They proposed the solution, without code change.

My question was, can we start parent suite for child suite automatically? (in case if context can give us such details)

@ulmanA can you try testNG 6.13+
and this
testng-team/testng#1656 (comment)

[ulmanA]

@DzmitryHumianiuk Still the same behavior in TestNG 6.14.3

[DzmitryHumianiuk]

@ulmanA could you plz share details on your suites structure at testng-team/testng#1656 (comment)

it's testNG related issue, and we can't fix it, until testNG will provide a fix

[ulmanA]

Done.

[DzmitryHumianiuk]

@ulmanA your are awesome, thank you!