Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support pytest xfail #12

Closed
Mulugruntz opened this issue Jun 4, 2021 · 2 comments · Fixed by #19
Closed

Support pytest xfail #12

Mulugruntz opened this issue Jun 4, 2021 · 2 comments · Fixed by #19
Assignees
Labels
bug Something isn't working

Comments

@Mulugruntz
Copy link

I have a parametrized test with marks=[pytest.mark.xfail] where I expect it to fail.

However, I get an exception when running shut pkg test --no-capture:

Traceback (most recent call last):
  File "/.../bin/shut", line 8, in <module>
    sys.exit(shut())
  File "/.../lib/python3.9/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/.../lib/python3.9/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/.../lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../lib/python3.9/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/.../lib/python3.9/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/.../lib/python3.9/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/.../lib/python3.9/site-packages/shut/commands/pkg/test.py", line 153, in test
    test_run = test_package(package, isolate, keep_test_env, capture)
  File "/.../lib/python3.9/site-packages/shut/commands/pkg/test.py", line 70, in test_package
    return package.test_driver.test_package(package, runtime, capture)
  File "/.../lib/python3.9/site-packages/shut/test/pytest.py", line 126, in test_package
    test_run = load_report_file(self.report_file)
  File "/.../lib/python3.9/site-packages/shut/test/pytest.py", line 81, in load_report_file
    test_status = {'passed': TestStatus.PASSED, 'failed': TestStatus.FAILED, 'skipped': TestStatus.SKIPPED}[test['outcome']]
KeyError: 'xfailed'

Indeed, only passed, failed and skipped are acknowledged. Whereas xfailed and xpassed are not.

Documentation: https://docs.pytest.org/en/6.2.x/example/parametrize.html

@NiklasRosenstein
Copy link
Owner

Thanks for reporting @Mulugruntz , I'll take a look at it soon!

@NiklasRosenstein NiklasRosenstein self-assigned this Jun 4, 2021
@NiklasRosenstein NiklasRosenstein added the bug Something isn't working label Jun 4, 2021
@NiklasRosenstein
Copy link
Owner

Hey @Mulugruntz, the planned fix is here. Do you think it makes sense or should Shut understand xpassed/xfailed as concepts separate from passed/failed?

#19

@NiklasRosenstein NiklasRosenstein linked a pull request Aug 9, 2021 that will close this issue
NiklasRosenstein added a commit that referenced this issue Aug 9, 2021
fix #12 by stripping "x" from the test outcome
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants