Automatic screenshots on error? #269
Comments
|
Seeing that it has been 6 days since you posted this yielding no suggestions, I'm just going to drop my thoughts. As far as I can remember from my experiences, the following rules hold:
Frankly, if there is a better solution than this, I would like to know as well. Maybe yours is, and you are just looking at one of the cases where webdriverio unexpectedly throws an error. Also, this is one of the reasons I went for the synchronize solution. Pass all webdriverio commands (except event handling methods) through, start up a fiber, try catch over the whole block to catch all its errors at once (and have the ease of an synchronous API). It still does not catch the unexpected errors through. To the maintainers, I will be more specific about any errors I find as soon as I know for sure they are there and they are caused by webdriverio. All I have right now are hunches, and they generally coincide with mistakes I've made as well (webdriverio unexpected behavior --> try to debug --> find own error --> fix error --> webdriverio works). |
|
Thanks @nickyout. I was trying to catch all errors automatically with one event/callback. |
|
Yeah I get that. I just wonder if webdriverio is supposed to shut down when you do not add a callback to catch the error, or that the fact that nodejs kills the process on an uncaught error does that. If nodejs does that, you might have found a bug in webdriverio that causes it to die before it can report an error. That would explain the symptoms - process.on('uncaughtError') catches everything afaik. |
|
I'm a bit new here, so forgive my ignorance, what you are saying is that errors passed to command callbacks, such as require('webdriverio').remote().init().url('www.example.com').click('#selector', function (err) {
if (err) throw err; // will this stop `.getTitle` from being executed?
}).getTitle(function (err, title) {
if (err) throw err;
if (title !== 'title') throw new Error('Title should match `title`');
}).end(); |
|
I don't know, but I certainly expect an uncaught error to kill the node process, and webdriverio with it. Afaik, all callback passed errors must be received in a user defined callback function (like your example) or else webdriverio will stop itself. So I would expect that the throw err would indeed stop |
|
@nickyout you're right, thanks for clarification. So basically if you have an custom callback like: client.command(parameter1, function callback(err, val, res) {
// ...
});You need to check if an error was thrown. This happens when the selenium server wasn't able to execute the command properly. But things like: client.command(parameter1, function callback(err, val, res) {
if (err) throw err;
});look pretty verbose and not nice. Since WebdriverIO (since v2.0.0) all commands without an custom callback will throw the error automatically, you don't have to check it manually. You can get rid of the callback: client.command(parameter1);In case you want to work with the response values (e.g. when calling getTitle and you want to check if title equals something), you still have to check manually if an error was passed to the callback function. But usually when an error occurs the callback value is null or undefined, so your test would break anyway (in most cases). Hope that clarifies it a bit more. Let me know if you still have questions! @suprMax what test framework do you use? If you want to make a screenshot if a command fails I would recommend to do that within a teardown function (like |
|
@christian-bromann I'm using mocha. I think |
|
Yeah but was I meant is something like this: describe('my app', function(
describe('has a certain feature which', function(
it('should do this', function(done) {
//... do some selenium stuff here
})
// capture screenshot when it fails
after(captureScreenshot)
});
describe('has a certain other feature which', function(
it('should do this', function(done) {
//... do some selenium stuff here
})
// capture screenshot when it fails
after(captureScreenshot)
})
})Like I said, I will dig into it later. So it should do a screenshot first and then throw the error. |
|
I would like to add here that in almost all cases I can catch an error and create a screenshot, as long as webdriverio returns the error as it is supposed to. I'm using the synchronize trick, but essentially it would mean that you should still be able to make a screenshot in the error receiving callback. As long as you catch any potential error after every step... tedious, perhaps, but if you make a dedicated function that makes a screenshot on error and pass that as callback to every command you can at least get your screenshots with webdriverio's current functionality. |
|
@suprMax I looked into it and discovered that a screenshot only gets taken when a Selenium command wasn't executed properly. In this case the server returns a base64 encoded image which gets saved at your desired Have you found a desirable solution for that? Your goal is to take a screenshot if an assertion fails, right? The think is that it is hard to detect when this happens. We could introduce own assertion functions but I this is not part of the job of WebdriverIO. What do you thing? How we can improve WebdriverIO to do something like this? I guess many people want to know how the application looks like when something fails to get an idea what's broken. |
|
@christian-bromann I think it's best to rely on mocha and chai for assertions. It's quite easy to catch assertion errors and it seems to work perfectly now. client.addCommand 'capture', (name)->
done = _.last(arguments)
date = (new Date).toString().replace(/\s/g, '-').replace(/\-\(\w+\)/, '')
location = "#{screenshotsLocation}/#{screenshotsCount++}-#{name}_#{date}.png"
@saveScreenshot(location, false, done)
process.on('uncaughtException', (err)-> client?.capture('error'))You can probably capture these automatically as well, having an option to enable them. Failing assertions always throw. |
|
@suprMax , you can do it by using /**
* Function to be executed after a test (in Mocha/Jasmine) or a step (in Cucumber) starts.
* @param {Object} test test details
*/
afterTest: function (test) {
// if test passed, ignore, else take and save screenshot.
if (test.passed) {
return;
}
// get current test title and clean it, to use it as file name
var filename = encodeURIComponent(test.title.replace(/\s+/g, '-'));
// build file path
var filePath = this.screenshotPath + filename + '.png';
// save screenshot
browser.saveScreenshot(filePath);
console.log('\n\tScreenshot location:', filePath, '\n');
}, |
|
For anybody landing here after a search, @christian-bromann said today at https://gitter.im/webdriverio/webdriverio "the last comment actually nailed it" cc @shadiabuhilal |
|
@shadiabuhilal - Ahh nice work! This worked for me (albeit, if I had 2 browsers within capabilities, it would error out and screenshot the failure from only 1 browser). This code was on webdriver.io however it didn't work for me, no screenshot found: |
|
@shadiabuhilal Thanks for that code suggestion. I've got a revised version, after I ran into a problem with saving screenshots like that( if a file already exists with that name, it fails to save the screenshot). The important change is to use a timestamp in the filename: Note: I based this code on the webdriverio code (function saveScreenshotSync in webdriverio.js) which takes screenshots when a command errors. |
|
I can't get this to work. I'm using Mocha my code looks like this: But the second |
|
Why do you think it should work? Please see an example in docs https://webdriver.io/docs/allure-reporter.html#add-screenshots You can always put breakpoint and inspect available arg properties |
Hi I am using this code to take screenshot of the failed test. But I see there are few more screenshots taken for the passes tests as well. It starts with the name ERRORCHROME. |
Hi!
I'm trying to set up automatic screenshots creation on error. I have following folder structure:
I am creating a webdriverio client passing
screenshotPath: "./test/screenshots"as an option, but that doesn't seem to affect anything.I was trying to add
but this doesn't get called at all.
The only thing that actually works is
but sometimes it fails too.
This is an example if a test that fails:
What would be the best way to handle this?
The text was updated successfully, but these errors were encountered: