Add tests with enabled=false to report as skipped

[ehborisov]

No description provided.

[braimanm]

Disabled test should be marked as pending or "unknown" (Allure 2 version). Skipped tests in testNG lexicon are tests that was not executed due to failure of depend-on tests or configuration methods. TestNG is not executing disabled tests at all and not keeping any record of them in the test results. In Allure 1 AllureTestListener I added functionality to mark disabled tests as "pending" ("work in progress") for test coverage tracking. It is preferable to label those tests as "Disabled" but there is no such status currently in Allure report.

[slipwalker]

@ehborisov If I got it right, currently Allure2 report doesn't track disabled tests (tests with enabled=false)? Am I right?

[ehborisov]

@slipwalker yes, there isn't any easy way to keep track of those tests. They aren't being executed so test listener doesn't know anything about them.

[tarasmytlovych]

@ehborisov
Hi.
Is there any way to implement this?
It will help a lot.

'Cos use case is the following: we have 1 acceptance test (that connection is established). If this check fails - no sense to continue because it is an entry point for each and every test so we do not proceed with the rest of tests.
And we may see the following trend - e.g. 50 test -> 51 tests -> 1 failed test -> 51 -> 1 failed test and it is very confusing.

allure-cucumber-jvm-adaptor, version:1.6.4
allure-cucumber-plugin, version: 1.0.2

[AlexanderPoleschuk]

According to documentation: http://testng.org/doc/documentation-main.html#ignore-tests the most appropriate way to handle enabled=false case is to add another status called Ignored.

This will store only disabled tests and distinguish it from really skipped tests.