When an xfail marked test passes(xpass), the report.outcome value shows as 'failed'

[gprasad84]

When an xfail marked test passes(xpass), the report.outcome value is 'failed'. Since the xfail test passed shouldnt the outcome of the test be 'passed' ?

Running pytest (2.9.1), py (1.4.31) on Mac OSX

# conftest.py
def pytest_runtest_logreport(report):
    print "When: {} Outcome: {}".format(report.when, report.outcome)

# test_sub.py
@pytest.mark.xfail
def test_xpass():
    pass

Snippet of the console output

When: setup Outcome: passed
XPASSWhen: call Outcome: failed
When: teardown Outcome: passed

[RonnyPfannschmidt]

@hpk42

this one is an artifact of xpass being a failure before, xpass got changed to be more forgiving by default
we should probably fix that in pytest 3.0?

[nicoddemus]

I think so... could you take a look @RonnyPfannschmidt?

[The-Compiler]

Should we do this for 3.0 as it's backwards-incompatible but probably relatively straightforward?

[hackebrot]

That would be on master branch, right?

[The-Compiler]

@hackebrot I don't understand the question - are you asking if @gprasad84 is seeing this output on master?

[hackebrot]

@The-Compiler Sorry about that. Looking into the issue right now and I was wondering if that change would go to master or features.

[The-Compiler]

While it is strictly speaking a bugfix, it might actually cause some trouble as it's backwards-incompatible. So I'd say to features. (Then again, in the end it probably doesn't matter as the next release should be v3.0 anyways)