Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some tests are silently skipped since version 6.11 #1565

Closed
gdemecki opened this issue Oct 4, 2017 · 7 comments
Closed

Some tests are silently skipped since version 6.11 #1565

gdemecki opened this issue Oct 4, 2017 · 7 comments
Labels
Feature: priority

Comments

@gdemecki
Copy link

gdemecki commented Oct 4, 2017 •
edited
Loading

TestNG Version

6.11

Expected behavior

All tests in the project are executed.

Actual behavior

Some tests are silently skipped.
The problem seems to be related to the priority attribute, because without it, all tests in all groups are executed correctly.

Is the issue reproductible on runner?

  • [x ] Maven

Test case sample

https://github.com/gdemecki/TestNG-1565
@krmahadevan
Copy link
Member

@gdemecki - Please help share a sample that can be used to reproduce the issue.

@gdemecki gdemecki mentioned this issue Oct 4, 2017
Closed
@gdemecki
Copy link
Author

gdemecki commented Oct 4, 2017

Hi @krmahadevan, sure, I've already added a sample.

@krmahadevan
Copy link
Member

@gdemecki - Thank you!

@krmahadevan
Copy link
Member

@gdemecki - I am not able to reproduce this with the latest released version of TestNG viz., 6.12 (But able to reproduce this with TestNG 6.11). I would suggest that you please upgrade to using TestNG 6.12

TestNG 6.12

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
...
... TestNG 6.12 by Cédric Beust (cedric@beust.com)
...

===== Invoked methods
    SampleClassATest.should_create_admin_return_ok()[pri:0, instance:testng.bug1565.showcase.SampleClassATest@1b0375b3] 453211571
    SampleClassBTest.should_create_user_return_ok_when_user_is_valid()[pri:0, instance:testng.bug1565.showcase.SampleClassBTest@6b71769e] 1802598046
    SampleClassATest.should_create_admin_return_ok_when_request_is_valid()[pri:1, instance:testng.bug1565.showcase.SampleClassATest@1b0375b3] 453211571
    SampleClassATest.should_get_admin_return_ok_when_given_existing_admin_id()[pri:0, instance:testng.bug1565.showcase.SampleClassATest@1b0375b3] 453211571
    SampleClassATest.should_create_admin_return_ok_when_user_and_buildings_are_correct()[pri:2, instance:testng.bug1565.showcase.SampleClassATest@1b0375b3] 453211571
=====
PASSED: should_create_admin_return_ok
PASSED: should_create_user_return_ok_when_user_is_valid
PASSED: should_create_admin_return_ok_when_request_is_valid
PASSED: should_get_admin_return_ok_when_given_existing_admin_id
PASSED: should_create_admin_return_ok_when_user_and_buildings_are_correct

===============================================
    My_Test
    Tests run: 5, Failures: 0, Skips: 0
===============================================

Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.317 sec - in TestSuite

Results :

Tests run: 5, Failures: 0, Errors: 0, Skipped: 0

TestNG 6.9.10

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running TestSuite
...
... TestNG 6.9.10 by Cédric Beust (cedric@beust.com)
...

[PackageUtils] Looking for test classes in the directory: /Users/krmahadevan/temp/TestNG-1565/target/test-classes/testng/bug1565/showcase
[PackageUtils] Found class SampleClassATest, seeing it if it's included or excluded
[PackageUtils] ... Including class SampleClassATest
[PackageUtils] Found class SampleClassBTest, seeing it if it's included or excluded
[PackageUtils] ... Including class SampleClassBTest
[TestRunner] Running the tests in 'My_Test' with parallel mode:false
[RunInfo] Adding method selector: org.testng.internal.XmlMethodSelector@d70c109 priority: 10
[TestClass] Creating TestClass for [ClassImpl class=testng.bug1565.showcase.SampleClassATest]
[TestClass] Adding method SampleClassATest.should_create_admin_return_ok_when_user_and_buildings_are_correct()[pri:2, instance:null] on TestClass class testng.bug1565.showcase.SampleClassATest
[TestClass] Adding method SampleClassATest.should_create_admin_return_ok()[pri:0, instance:null] on TestClass class testng.bug1565.showcase.SampleClassATest
[TestClass] Adding method SampleClassATest.should_create_admin_return_ok_when_request_is_valid()[pri:1, instance:null] on TestClass class testng.bug1565.showcase.SampleClassATest
[TestClass] Adding method SampleClassATest.should_get_admin_return_ok_when_given_existing_admin_id()[pri:0, instance:null] on TestClass class testng.bug1565.showcase.SampleClassATest
[TestClass] Creating TestClass for [ClassImpl class=testng.bug1565.showcase.SampleClassBTest]
[TestClass] Adding method SampleClassBTest.should_create_user_return_ok_when_user_is_valid()[pri:0, instance:null] on TestClass class testng.bug1565.showcase.SampleClassBTest
[XmlMethodSelector] Including method testng.bug1565.showcase.should_create_admin_return_ok_when_user_and_buildings_are_correct()
[XmlMethodSelector] Including method testng.bug1565.showcase.should_create_admin_return_ok()
[XmlMethodSelector] Including method testng.bug1565.showcase.should_create_admin_return_ok_when_request_is_valid()
[XmlMethodSelector] Including method testng.bug1565.showcase.should_get_admin_return_ok_when_given_existing_admin_id()
[XmlMethodSelector] Including method testng.bug1565.showcase.should_create_user_return_ok_when_user_is_valid()
[SuiteRunner] Created 1 TestRunners
[TestRunner] Running test My_Test on 2  classes,  included groups:[] excluded groups:[]
===== Test class
testng.bug1565.showcase.SampleClassATest
    @Test SampleClassATest.should_create_admin_return_ok_when_user_and_buildings_are_correct()[pri:2, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb]
    @Test SampleClassATest.should_create_admin_return_ok()[pri:0, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb]
    @Test SampleClassATest.should_create_admin_return_ok_when_request_is_valid()[pri:1, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb]
    @Test SampleClassATest.should_get_admin_return_ok_when_given_existing_admin_id()[pri:0, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb]
======
===== Test class
testng.bug1565.showcase.SampleClassBTest
    @Test SampleClassBTest.should_create_user_return_ok_when_user_is_valid()[pri:0, instance:testng.bug1565.showcase.SampleClassBTest@35bbe5e8]
======
[Invoker 1627960023] Invoking testng.bug1565.showcase.SampleClassATest.should_create_admin_return_ok
[Invoker 1627960023] Invoking testng.bug1565.showcase.SampleClassBTest.should_create_user_return_ok_when_user_is_valid
[Invoker 1627960023] Invoking testng.bug1565.showcase.SampleClassATest.should_create_admin_return_ok_when_request_is_valid
[Invoker 1627960023] Invoking testng.bug1565.showcase.SampleClassATest.should_create_admin_return_ok_when_user_and_buildings_are_correct
[Invoker 1627960023] Invoking testng.bug1565.showcase.SampleClassATest.should_get_admin_return_ok_when_given_existing_admin_id
===== Invoked methods
    SampleClassATest.should_create_admin_return_ok()[pri:0, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb] 1057941451
    SampleClassBTest.should_create_user_return_ok_when_user_is_valid()[pri:0, instance:testng.bug1565.showcase.SampleClassBTest@35bbe5e8] 901506536
    SampleClassATest.should_create_admin_return_ok_when_request_is_valid()[pri:1, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb] 1057941451
    SampleClassATest.should_create_admin_return_ok_when_user_and_buildings_are_correct()[pri:2, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb] 1057941451
    SampleClassATest.should_get_admin_return_ok_when_given_existing_admin_id()[pri:0, instance:testng.bug1565.showcase.SampleClassATest@3f0ee7cb] 1057941451
=====
Creating /Users/krmahadevan/temp/TestNG-1565/target/surefire-reports/My_Suite/My_Test.html
Creating /Users/krmahadevan/temp/TestNG-1565/target/surefire-reports/My_Suite/My_Test.xml
PASSED: should_create_admin_return_ok
PASSED: should_create_user_return_ok_when_user_is_valid
PASSED: should_create_admin_return_ok_when_request_is_valid
PASSED: should_create_admin_return_ok_when_user_and_buildings_are_correct
PASSED: should_get_admin_return_ok_when_given_existing_admin_id

===============================================
    My_Test
    Tests run: 5, Failures: 0, Skips: 0
===============================================


Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.315 sec - in TestSuite

Results :

Tests run: 5, Failures: 0, Errors: 0, Skipped: 0

@jeetendrae
Copy link

Hi All,
I am still observing this issue in 6.14.3 TestNG version. But this is not observed in 6.13.1 version. Someone kindly look into this.

@krmahadevan
Copy link
Member

@jeetendrae - The latest released version of TestNG is 7.0.0 If its still a problem, create a new issue and add a simple standalone test that can be used to reproduce the problem.

@gdemecki
Copy link
Author

gdemecki commented Dec 8, 2019

Although I no longer use TestNG, I've checked this specific case, and on my provided sample it looks like this:

  1. testng 6.9.10 <- works
  2. testng 6.11 <- does not work
  3. testng 6.14.3 <- works
  4. testng 7.0.0 <- works

@jeetendrae So it seems the issue you observed is something different that this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature: priority
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@juherr @gdemecki @krmahadevan @jeetendrae