-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
org.testng.reporters.XMLReporter doesnt support multiple <test> with same name in same suite #47
Comments
The problem is actually not in org.testng.reporters.XMLReporter but in SuiteRunner.runTest(): private void runTest(TestRunner tr) {
tr.run();
ISuiteResult sr = new SuiteResult(m_suite, tr);
m_suiteResults.put(tr.getName(), sr);
} The result is of TestRunner execution is added to m_suiteResults using test name as key, which is incorrect because TestNG doesn't enforce against having multiple tests with same name within a single suite. Solution:
What do you suggest Cedric? |
I also thought about all these options and neither is really straightforward. The best place to do this is as far up stream as possible, i.e. 1), but the XML parsing needs to be updated in several places (because of all the m_current*** fields I am maintaining because of SAX). |
unless you see a real need for having multiple test tags with same name, we should maybe break the backward compatibility in this case by going with option 2. It's the cleanest and simplest solution. I don't see a lot of people running into this issue, otherwise it would have been reported earlier. Otherwise, If you do go with option 1, won't you run into the issue with different set of parameters defined for different tests. How do you think that would be handled? |
Well, it's already undefined behavior so we wouldn't make things much worse, but yes, maybe we should just catch this in ContentTestNGHandler and abort if we detect two tags with the same name. |
ok. I'll make the change. |
Actually, a correction: we should catch this slightly higher than at XML parsing so that the fix will catch this both in XML and YAML documents. |
Should be fixed with nullin@7f570c9. Opening a pull request now. |
I think this issue should be closed too. |
If I run the following suite:
<suite name="QS AtomFeed tests">
<test name="QS:AtomFeed:Tests" preserve-order="true">
<classes>
<class name="testng.Test1"/>
<class name="testng.Test2"/>
<class name="testng.Test3"/>
</classes>
</test>
<test name="QS:AtomFeed:Tests" preserve-order="true">
<classes>
<class name="testng.Test3"/>
</classes>
</test>
</suite>
I see the following in the xml report:
<suite name="QS AtomFeed tests" duration-ms="2" started-at="2011-05-25T12:39:33Z" finished-at="2011-05-25T12:39:33Z">
<groups>
</groups>
<test name="QS:AtomFeed:Tests" duration-ms="2" started-at="2011-05-25T12:39:33Z" finished-at="2011-05-25T12:39:33Z">
<class name="testng.Test3">
<test-method status="PASS" signature="test1()" name="test1" duration-ms="1" started-at="2011-05-25T12:39:33Z" finished-at="2011-05-25T12:39:33Z">
</test-method>
</class>
</test>
</suite>
As you can see results for only one of the is reported. The TestNG plugin for eclipse correctly handles this and merges the results for <test>s with same name.
The text was updated successfully, but these errors were encountered: