Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Inherited test methods do not get expected group behavior #182

Open
erikgb opened this Issue · 7 comments

5 participants

@erikgb

Consider the following test classes:

@Test
public abstract class ParentTest {
    public void parentTestMethod() {
    }
}

@Test(groups = "myGroup")
public class ChildTest extends ParentTest {
    public void childTestMethod() {
    }
}

When running this test (ChildTest), I would expect that both test methods (childTestMethod and parentTestMethod) belong to group 'myGroup'. Apparently this is not the actual behavior (tested with version 6.4).

I put together some code to investigate this issue:

TestNGMethodFinder testNGMethodFinder = new TestNGMethodFinder(new RunInfo(), new JDK15AnnotationFinder(new DefaultAnnotationTransformer()));
ITestNGMethod[] testMethods = testNGMethodFinder.getTestMethods(ChildTest.class, new XmlTest());
for (ITestNGMethod testMethod : testMethods) {
    String[] groups = testMethod.getGroups();
    System.out.println(testMethod + " groups:" + Arrays.toString(groups));
}

Output from running this code:

ChildTest.childTestMethod()[pri:0, instance:null] groups:[myGroup]
ParentTest.parentTestMethod()[pri:0, instance:null] groups:[]

I looked into the TestNG source code, and it appears that this behavior originates from the method org.testng.internal.BaseTestMethod#initGroups(Class<?>). To be exact, I think the following statement is the cause of this rather odd behavior:

ITestOrConfiguration classAnnotation = (ITestOrConfiguration) getAnnotationFinder().findAnnotation(getMethod().getDeclaringClass(), annotationClass);

In order to fix this issue, this statement could be modified to:

ITestOrConfiguration classAnnotation = (ITestOrConfiguration) getAnnotationFinder().findAnnotation(getInstance().getClass(), annotationClass);

but I am not sure if this would be the correct way to resolve the issue. Nor eventual implications....

@cbeust
Owner

I see the problem, and your suggestion doesn't quite work, unfortunately. It's a bit tricky to solve because annotations are resolved in the init phase, which means that when the parent's @Test annotation is parsed, it obviously knows nothing about the fact that the child annotation will add more groups to it...

@erikgb

Cedric, thank you for your prompt feedback on this issue. You say "it is a bit tricky to solve", does that mean that you do not think that this issue will have a chance of being resolved in the near future? Can you think about any possible workarounds?

@cbeust
Owner

What I meant was: when the group resolution is performed, no test class has been instantiated yet, so when I look at parentTestMethod(), I don't have an instance of ChildTest that would allow me to put that method in this group.

Group resolution is purely static, while the example above can only be resolved dynamically.

Does this make sense?

@mumrah

I have seen this behavior as well. When I run my tests without a groups filter, everything passes, but when running with groups, things fail. In some cases it is due to my test class inheriting from some other test class, but in other cases I have POJO test classes and it seems like my @BeforeMethod and @AfterMethod methods are not invoked properly.

@mumrah

BTW, my solution is to use excludedgroups which of course won't work in all situations. @cbeust, has there been any work on a proper solution for this?

@Controlix

More than a year later and this still does not work.
@cbeust IMHO this does not make any sense: since the parent class is abstract, its group resolution should be based in a concrete instance of a test class.

@ken-p

FWIW I just ran into this as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.