Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

--failfast: broken tests do not show up properly in xunit output #559

Open
mhaas opened this issue Mar 23, 2018 · 0 comments
Open

--failfast: broken tests do not show up properly in xunit output #559

mhaas opened this issue Mar 23, 2018 · 0 comments

Comments

@mhaas
Copy link

mhaas commented Mar 23, 2018

When a step fails when lettuce is running with --failfast, the generated xunit output is incorrect. It will contain the testcase with a <failure> child element, but the failures or errors attributes on the <testsuite> root element is not update.

Shortened example output:

<?xml version="1.0" encoding="UTF-8"?>
<testsuite errors="0" failures="0" hostname="localhost" name="lettuce" tests="268" time="0" timestamp="2018-03-23T14:31:54">
  <testcase classname="My cool feature" name="My cool step" time="2.592967"/>
  <testcase classname="My broken feature" name="My failing step" time="30.65113">
    <failure message="My failing step..."/>...</failure>
</testsuite>

Further output of the lettuce command also indicates that the failure is not propagated correctly:

2018-03-23T15:07:02.826107825Z  Lettuce aborted running any more tests because was called with the `--failfast` option
2018-03-23T15:07:02.826110123Z  
2018-03-23T15:07:02.826112230Z  9 features (9 passed)
2018-03-23T15:07:02.826114560Z  9 scenarios (9 passed)
2018-03-23T15:07:02.826116786Z  268 steps (268 passed)
2018-03-23T15:07:02.888016366Z  Traceback (most recent call last):
2018-03-23T15:07:02.888054476Z    File "/var/jenkins/curing_backend/virtualenv/env_1/bin/lettuce", line 11, in <module>
2018-03-23T15:07:02.888058167Z      sys.exit(main())
2018-03-23T15:07:02.888060666Z    File "/var/jenkins/curing_backend/virtualenv/env_1/lib/python2.7/site-packages/lettuce/bin.py", line 134, in main
2018-03-23T15:07:02.901923524Z      result = runner.run()
2018-03-23T15:07:02.901940441Z    File "/var/jenkins/curing_backend/virtualenv/env_1/lib/python2.7/site-packages/lettuce/__init__.py", line 218, in run
2018-03-23T15:07:02.907028817Z      raise LettuceRunnerError("Test failed.")
2018-03-23T15:07:02.907043373Z  lettuce.exceptions.LettuceRunnerError: Test failed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant