Init framework before browser #4531
Conversation
mgrybyk
added
work-in-progress
PR: Breaking Change 💥
Codecov Report
@@ Coverage Diff @@
## master #4531 +/- ##
=========================================
+ Coverage 99.39% 99.4% +0.01%
=========================================
Files 207 209 +2
Lines 5306 5406 +100
Branches 1146 1177 +31
=========================================
+ Hits 5274 5374 +100
Misses 29 29
Partials 3 3
Continue to review full report at Codecov.
|
|
@christian-bromann we can't do this because There is a tricky situation with Mocha and Jasmine. Some users (like me) may use It won't work anymore because So, to make filtering work in Mocha/Jasmine we need users to pass helper files required by tests that are not dependant on What else can be done?
|
|
@christian-bromann here is something that will no longer work for Mocha / Jasmine users who want to use filtering feature: example: Another example when helper file should be moved to |
|
|
This will be a breaking change and we should release it as part of v6 including dropping support for Node v8 (LTS ends at the end of this year). I will create a label for it. I wonder if we should drop support earlier to release this. |
|
Wow, this is amazing and a big changer! Awesome work already! |
|
@christian-bromann we can make it as an optional feature for mocha and jasmine users, and enabled by default for Cucumber users. |
|
This fact that this is not working anymore: I would be fine to only introduce it in Cucumber for now and create an issue to introduce it in Mocha and Jasmine when we move to v6 |
|
@christian-bromann it's configurable. |
|
@christian-bromann and @mgrybyk This is very useful feature. Is it possible to release this soon rather than waiting until v6? Thank you! |
|
@christian-bromann changed a bit how it works.
tested changes in my local project (with mocha and jasmine), it was very easy for me to adopt it, an example for mocha: after It is now possible to use It's still a POC and some functionality is missing |
|
also had to add jasmine's require/helpers support to avoid adding beforeFramework hook: |
|
I think the feature is more or less ready now. It won't cause any issues to existing users, it's pretty much easy to start using it for existing Mocha/Jasmine users (from my point of view). The feature works for Cucumber users out of the box without any breaking changes. Going to update the PR description to provide with more visibility |
|
The PR is ready for initial review, will proceed with adding docs/etc if there are no objections |
|
I have to agree with @wswebcreation, this is really great work! I was wondering, since you mentioned:
Does having spec files in typescript and especially config files in typescript still work? |
|
@martinfrancois there are no changes in this area at all |
|
@mgrybyk thanks for the confirmation! Was just unsure since you need to add |
|
It still work as is if the feature is not enabled for mocha and jasmine users. If the feature is enabled ts node have to be moved to require array in mochaOpts / jasmineNodeOpts |
|
That's actually what I was worried about, when using typed configurations I was wondering if we enable this feature with mocha / jasmine via the feature toggles if it would work, since there you can't move the |
|
No, typed configuration is not affected. If ts node is in js file that requires typed config there is no need to require it again anywhere else |
|
Docs available https://webdriver.io/blog/2019/11/01/spec-filtering.html |
|
Thanks for the link. I assume you count the typescript compilation as transpiler here? If so, there is no difference between my config and your example. |
|
Hi Team I am using cucumber and webdriverio framework, how can avoid deleting the session for each spec run? Please refer my package version below |
There is no way to do it with cucumber framework |
What is the best way to run multiple feature file with one session in cucumber framework with wdio ? |
|
@karthimathan it's not possible with current runner and cucumber. If you like you can run wdio programmatically and use cucumber js. Why do you need to keep session? |
@mgrybyk Thanks for your replay We are using cucumber js only to run our test scripts Sample test case: Given App is loaded Step Definition: We have wdio integrated with the framework and have define two config files 1. Commonconfig.js (wdio capabilities) 2. Device specific config file (Appium capabilities) We run the below command to execute: Is this expected or should we modify anything here Why do you need to keep session? |
|
@mgrybyk @christian-bromann I've been looking at this PR to see if it solves our use case. We have cucumberjs and it's said here that specFiltering is enabled in cucumber and it has it's own tag mechanism.... The problem I'm seeing is that we have scenarios that should be skipped in some browsers and devices (i.e. firefox and ie, only in firefox or only in edge, only in chrome in android, .... lot's of combinations, you get the point ;-) ). With cucumber pickle filtering we can exclude upfront (prior to execution) the tests we want, but we cannot make this filtering dynamic depending on the browser/device AFAIK. We have implemented a cucumber hook that skips the scenario/feature depending on the tags, but session is still started. Is there any way we can apply the same dynamic filtering that has been enabled in mocha or jasmine to cucumber tests? We don't want to have to execute several wdio commands for each capability combination to be able to inject the specific exclude tag for each browser/device.... |
|
I've been able to do it by adding my own framework extending cucumber one... But wouldn't it be worth adding some kind of extension point (may be a hook) so that people can filter tests? class CucumberAdapter extends CucumberFramework.CucumberAdapter {
testCases: {
pickle: any,
uri: string
}[] = [];
constructor(...args) {
super(...args);
}
async init () {
const result = await super.init();
this.testCases = this.testCases.filter(scenario => filter(scenario, (global as any).browser, process.env));
return result;
}
hasTests () {
return this.testCases.length > 0;
}
run () {
return super.run();
}
} |
Proposed changes
Init test framework before starting webdriver session.
(initial) purpose
Filter specs before webdriver session is started which is required to speedup tests execution.
Starting/stopping webdriver session takes some time for browsers and takes lots of time with Appium.
See #4326
NOT a breaking change
Feature Flags
Added
featureFlagsto wdio.conf that can havespecFilteringproperty.Example:
Spec filtering for Mocha/Jasmine users
Only if the feature is enabled with feature flag!
browserobject frommochaOpts.requiretobeforehookbeforehook tomochaOpts.require/jasmineNodeOpts.requiressome common cases
browser.addCommand('foo', () => {})have to be moved to helper files and called inbeforehookjasmineNodeOpts
Added support of built-in
requiresandhelpersto be able to load compilers or helper files that are required to run tests.Flow changes
current flow
2.1. init test framework
2.2. call
beforehook that expectsbrowserobject to be available2.3. run test framework
new flow
isMobile)beforehookacknowledgement
browser.capabilitiesif spec filtering is enabled for Mocha/Jasmine before browser session is started. As a result user won't be able to use some runtime capabilities for tests filtering, likebrowser.capabilities.chrome.chromedriverVersion.isW3CandisSeleniumStandaloneflags before browser session is started.Types of changes
Checklist
Further comments
I think having this feature we can think on tags management for Jasmine and Mocha #2859
Cucumber has its own tag management system, so there is nothing to do for us
Reviewers: @webdriverio/technical-committee