v3 Report logs pass as skip in retry

[saikrishna321]

@anshooarora

i have retry on failure, so when my test fails the first time(Skip) and passed the next run ( Pass), report should consider it as Pass and not as Skipped

Refer the screenshot below

[saikrishna321]

@anshooarora ping, this is giving a wrong test results report.. can you pls help

[anshooarora]

This is because SKIP has higher priority than PASS. There was an old discussion due to which I had set the priority like so, otherwise PASS always had higher priority.

[saikrishna321]

@anshooarora this is giving a wrong feel, the test case skipped the first time and passed the second time on re-run .

Looking at skip, it gives a wrong data on the dashboard and we panic seeing the reports on pipeline and when your on CI/CD which is very aggressive

Do you think we need to consider FATAL, FAIL, ERROR, WARNING, PASS, SKIP, INFO, WARNING ?

[anshooarora]

I will let this hierarchy remain but provide a way to setup your own hierarchy. Would that work?

[saikrishna321]

cheers , that works :-)

[SrinivasanTarget]

@anshooarora Can we have this in 3.0.1?

[anshooarora]

@SrinivasanTarget yes

[saikrishna321]

@anshooarora any tests for this ? which would help on how to use it ..

[anshooarora]

@saikrishna321 - docs & tests coming soon..

quick usage:

List statusHierarchy = Arrays.asList(
                    Status.FATAL,
                    Status.FAIL,
                    Status.ERROR,
                    Status.WARNING,
                    Status.SKIP,
                    Status.PASS,
                    Status.DEBUG,
                    Status.INFO
);

extent.config().statusConfigurator().setStatusHierarchy(statusHierarchy);

[saikrishna321]

can this be used in current version or next release ?

[anshooarora]

3.0.1 onwards.

[fragonib]

Customizable status hierarchy is great. Thank you.

What do you think about letting the user to express status agregation algorithm for any group level (ie.: test groups and suites). It would be more flexible and customizable than simply using status hierarchy. Is this possible currently? Otherwise I don't know the impact of such feature on current code but please, consider it.