DataProvider not running tests In Parallel when using Factory #1951
Comments
|
@VinceBarresi - I am not quite sure how you are saying that TestNG is running tests sequentially. I altered your test class to print out the thread ids and here's how it looks like import org.testng.annotations.DataProvider;
import org.testng.annotations.Factory;
import org.testng.annotations.Test;
public class MSGWebExampleTest {
private String browser, browserVersion, platform;
@Factory(dataProvider = "dp")
public MSGWebExampleTest(String browser, String browserVersion, String platform) {
this.browser = browser;
this.browserVersion = browserVersion;
this.platform = platform;
}
@Test
public void openSiteTest() {
System.err.println("Running " + this + " on thread [" + Thread.currentThread().getId() + "]");
}
@Override
public String toString() {
return String.format("[%s, %s, %s]", browser, browserVersion, platform);
}
@DataProvider(parallel = true, name = "dp")
public static Object[][] defaultWebDataProvider() {
return new Object[][]{
new Object[]{"chrome", "70", "Windows 10"},
new Object[]{"internet explorer", "11", "Windows 10"},
new Object[]{"safari", "12.0", "Mac 10.13"},
new Object[]{"chrome", "70", "Mac 10.13"},
new Object[]{"firefox", "63", "Mac 10.13"}
};
}
}<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd">
<suite name="Test Engineering Core Suite"
parallel="methods"
data-provider-thread-count="4"
thread-count="2"
verbose="2">
<test name="Test Engineering Core_test">
<classes>
<class name="com.rationaleemotions.github.issue1951.MSGWebExampleTest"/>
</classes>
</test>
</suite>The output As you can see from the above output, the thread ids are different (two threads to be specific since our thread-count is 2) Your data provider in your sample is tied to a factory method and NOT to a Only when the following happens -
Parallelism will be enabled by TestNG and the number of Note: Please let me know if that explanation makes sense and if this issue can be closed. |
|
@krmahadevan Thank you for your detailed response. I'd like to run myself to test this out when I get a chance. I do have a couple of questions, 1: Does the data provider (in this case - "dp) have to be in the same class as the @test we are passing the data provider to in order to still achieve parallelism by data-provider-thread-count and parallel = true? 2: Did you execute tests at the suite level? If so, does this even matter in terms of parallelism? For example, if I kick of my tests of group DEBUG via surefire using mvn test -Dgroups=DEBUG, do I need to pass the dataprovider threadcount via a parameter in this command? |
|
>> 1: Does the data provider (in this case - "dp) have to be in the same class as the ***@***.***` we are passing the data provider to in order to still achieve parallelism by data-provider-thread-count and parallel = true?
Not necessary. It doesnt matter where the data provider resides.
>> 2: Did you execute tests at the suite level? If so, does this even matter in terms of parallelism? For example, if I kick of my tests of group DEBUG via surefire using mvn test -Dgroups=DEBUG, do I need to pass the dataprovider threadcount via a parameter in this command?
Yes I did execute the tests at the suite level. It shouldn't matter. As
long as the data provider thread count is made available in the suite file
(default value if its not provided is 10 I believe), it should work
seamlessly.
|
|
@krmahadevan So I had a chance to test this morning using your suggested changes and have a couple observations.
Any thoughts on this? |
|
Can you please share the test code you used to trigger the error ? |
|
Im using the exact testing.xml file you pasted above (obviously with the class pointing to my test class in my package). Here is my DataProviderManager.java here is my test class If you want to test scenario one then uncomment whats in the test. For scenario 2 you can run as is |
|
@VinceBarresi - The problem in both the cases is due to your test code. I don't think TestNG is at fault here. Scenario 1 The error Scenario 2 The error Since the original problem stands resolved, I am closing this issue with resolution as Question answered |
|
@krmahadevan This is a misunderstanding of DataProvider on my part - Since I don't need the parameters passed from it in the test itself ( I need them in my threadlocal getters and setters to pass to selenium capabilities ) I didn't realize I needed to pass them to the actual test method. I really appreciate you clearing this up! I do have have one more question on parallelism if you have a moment - How do I set data provider thread count and/or thread count so that all 5 data sets from run at the exact same time for each @test ? Currently I can run in parallel but its executing the same set (e.g "chrome", "70", "Windows 10") n number of thread times instead of running each set at the same time. |
|
You need to be using a factory ( The factory uses the data provider to basically create test class instances (a class that houses one or more |
|
@krmahadevan I'm using @factory which is sets up capabilities including browser, version, platform from my DataProvider and initializes thread safe instances of WebDriver. If I set (parallel=instances) the test does run in parallel but is running multiple instances with the same browser, version, platform from my DataProvider as you can see below:
Instead of running multiple threads each of them on Safari, 12, Mac 10.13, I'm trying to run the test with on multiple threads in parallel with - new Object[]{"chrome", "70", "Windows 10"}, THIS SHOULD BE THREAD 1 |
|
@VinceBarresi - I have added a detailed answer to your question on stack overflow. To wrap up in a nutshell, when you use a |
|
@krmahadevan your reply on StackOverflow was very enlightening, thank you! One question: Let's say you use the same data provider for every test. Is there a way we can avoid duplicating the factory code in every class? Thanks! |
|
@borbamartin - You can have all your
Glad to know it helped you 👍 |
|
@krmahadevan yes, let me rephrase. Let's say I have 50 feature classes which contain multiple tests each. This means I have to write the following 50 times (in each class) Question is, does TestNG provide something I can use to avoid writing this in every test class? Thanks once again! |
|
@borbamartin - To the best of my knowledge, there's nothing in TestNG that will automatically do this for you. You would need to do it yourself. |
TestNG Version
7.0.0-beta1
Expected behavior
I have a class that contains a DataProvider with 5 data sets including browser name, version, and platform as you can see here:
In my test class I am using Factory to pass data as such:
testing.xml:
If I set thread-count =2 and data-provider-thread-count =4 and parallel=true my test should run 8 times in parallel.
Actual behavior
Tests run sequentially regardless of how many threads I try to set as if parallel=true is ignored. I tried numerous attempts to get this to run in parallel using the testing.xml above, by passing -Dthreadcount=2 -Ddataproviderthreacount=4 from commandline and even setting these values in the config tag for the surefire plugin and nothing is working.
Is the issue reproducible on runner?
Test case sample
The text was updated successfully, but these errors were encountered: