Do not close browser on test failures (optionally)

[blueyed]

I am just experimenting with pytest-splinter, and have already used the following method before:

@pytest.yield_fixture(scope='function', …)
def browser(request, live_server):
    # …

    failed_before = request.session._testsfailed
    yield browser

    # Quit the browser only if the test has not failed.
    # ...or request.config.option.maxfail > request.session._testsfailed:
    if request.session._testsfailed == failed_before:
        b.quit()

This uses a rather new feature of py.test (yield.fixture), instead of adding a finalizer to the request.

Do you think it makes sense to add such a feature to pytest-splinter?

I am currently wrapping pytest-splinter's browser fixture with this method and have defined the fixture splinter_close_browser to return False, which appears to work.

[bubenkoff]

not sure how it helps
normally, you just use --pdb and you stop directly where it fails (and
browser is NOT closed then)
what is the use-case of NOT closing the browser when the test session is
done?

On 26 June 2014 23:30, Daniel Hahler notifications@github.com wrote:

I am just experimenting with pytest-splinter, and have already used the
following method before:

@pytest.yield_fixture(scope='function', …)
def browser(request, live_server):
# …

failed_before = request.session._testsfailed
yield browser

# Quit the browser only if the test has not failed.
# ...or request.config.option.maxfail > request.session._testsfailed:
if request.session._testsfailed == failed_before:
    b.quit()

This uses a rather new feature of py.test (yield.fixture), instead of
adding a finalizer to the request.

Do you think it makes sense to add such a feature to pytest-splinter?

I am currently wrapping pytest-splinter's browser fixture with this
method and have defined the fixture splinter_close_browser to return
False, which appears to work.

—
Reply to this email directly or view it on GitHub
#6.

Anatoly Bubenkov

[blueyed]

You are right that --pdb / --ipdb is often more useful in this case.

My use case has been when I was not using --ipdb (by default), where it was useful to have the browser still open to inspect the elements.

I was under the impression that using --ipdb gets a bit in the way, if you do not want to debug the test failure, but are in the process of writing tests: with something like assert 0, "finish the test" at the end, you can dispatch the test, and inspect the window for the next step.
The same applies to using --ipdb though, and it lets you easily try/inspect the code before adding it to the test.

Thanks for your feedback (and this pytest plugin!).
I am closing the issue for now.

[bubenkoff]

I see where it came to you now...
Well, still think that --pdb/ipdb is better for debugging
also, what we do for debugging, we make snapshots when test fails
we use pytest-splinter with combination of pytest-bdd
https://github.com/olegpidsadnyi/pytest-bdd, there we've made a
hook, pytest_bdd_step_error
, which allows you
to catch the moment when it fails and do some action, snapshot, in
particular

it looks like:

def pytest_bdd_step_error(request, feature, scenario, step, step_func,
step_func_args, exception):
"""Make screenshot on test failure."""
try:
browser = request.getfuncargvalue('browser')
except Exception:
return
slaveoutput = getattr(request.config, 'slaveoutput', None)
project_dir = request.getfuncargvalue('project_path')
names = junitxml.mangle_testnames(request.node.nodeid.split("::"))
classname = '.'.join(names[:-1])
screenshot_dir = os.path.join(project_dir, classname)
if not os.path.exists(screenshot_dir):
os.makedirs(screenshot_dir)
name = hashlib.md5(u'{0}-{1}'.format(names[-1], step.name).encode('utf-8',
'ignore')).hexdigest()
screenshot_file_name = '{0}.png'.format(name)
screenshot_path = os.path.join(screenshot_dir, screenshot_file_name)
browser.driver.save_screenshot(screenshot_path)
sys.stderr.write('[[ATTACHMENT|{0}]]'.format(os.path.join(
request.config.option.screenshot_path_prefix, classname,
screenshot_file_name)))
with open(screenshot_path) as fd:
if slaveoutput is not None:
slaveoutput.setdefault('screenshots', []).append({
'class_name': classname,
'file_name': screenshot_file_name,
'content': fd.read()
})

ignore slaveoutput related stuff if you don't need pytest-xdist things so
test parallelization

On 27 June 2014 21:55, Daniel Hahler notifications@github.com wrote:

You are right that --pdb / --ipdb is often more useful in this case.

My use case has been when I was not using --ipdb (by default), where it
was useful to have the browser still open to inspect the elements.

I was under the impression that using --ipdb gets a bit in the way, if
you do not want to debug the test failure, but are in the process of
writing tests: with something like assert 0, "finish the test" at the
end, you can dispatch the test, and inspect the window for the next step.
The same applies to using --ipdb though, and it lets you easily
try/inspect the code before adding it to the test.

Thanks for your feedback (and this pytest plugin!).
I am closing the issue for now.

—
Reply to this email directly or view it on GitHub
#6 (comment)
.

Anatoly Bubenkov

[blueyed]

Thanks, that looks very useful.

I have tried applying it to my non-bdd-based conftest, but browser = request.getfuncargvalue('browser') fails with AttributeError: FixtureDef instance has no attribute 'cached_result'.

self._get_active_fixturedef(argname) is: <FixtureDef name='browser' scope='function' baseid='tmm' > in request.getfuncargvalue.

I have tried to base it on the recipe from http://pytest.org/dev/example/simple.html#making-test-result-information-available-in-fixtures, both with a autouse=True fixture, and by injecting it as screenshot_on_failiure fixture into the browser fixture itself.

[blueyed]

After all, it might be very useful to have a helper fixture in pytest-splinter, which would take a screenshot on failures with browser tests.

[bubenkoff]

show me some code
hard to say something

On 10 July 2014 00:45, Daniel Hahler notifications@github.com wrote:

Thanks, that looks very useful.

I have tried applying it to my non-bdd-based conftest, but browser =
request.getfuncargvalue('browser') fails with AttributeError: FixtureDef
instance has no attribute 'cached_result'.

self._get_active_fixturedef(argname) is: in request.getfuncargvalue.

I have tried to base it on the recipe from
http://pytest.org/dev/example/simple.html#making-test-result-information-available-in-fixtures,
both with a autouse=True fixture, and by injecting it as
screenshot_on_failiure fixture into the browser fixture itself.

—
Reply to this email directly or view it on GitHub
#6 (comment)
.

Anatoly Bubenkov

[blueyed]

https://gist.github.com/88481f40138020bab62d

[bubenkoff]

with this approach you have to directly depend on the browser instead of
calculating on the fly from the request

On 10 July 2014 00:51, Daniel Hahler notifications@github.com wrote:

https://gist.github.com/88481f40138020bab62d

—
Reply to this email directly or view it on GitHub
#6 (comment)
.

Anatoly Bubenkov

[blueyed]

Thanks, that works.

But I had to move it into the/my browser fixture itself, using the test failure detection via request.session._testsfailed, because otherwise the browser is already closed.

It might work with autouse=True and then not using the fixture in browser itself.. I would have thought that a circular dependency between browser and screenshot_on_failure would not have been possible at all.

Thanks again.

Do you think pytest-splinter should ship a simple fixture to provide this functionality?

[bubenkoff]

yes i agree we should provide that feature
please create an issue

On 10 July 2014 01:24, Daniel Hahler notifications@github.com wrote:

Thanks, that works.

But I had to move it into the/my browser fixture itself, using the test
failure detection via request.session._testsfailed, because otherwise the
browser is already closed.

It might work with autouse=True and then not using the fixture in browser
itself.. I would have thought that a circular dependency between browser
and screenshot_on_failure would not have been possible at all.

Thanks again.

Do you think pytest-splinter should ship a simple fixture to provide this
functionality?

—
Reply to this email directly or view it on GitHub
#6 (comment)
.

Anatoly Bubenkov