Allow to define in which browser types to run tests

[schmittjoh]

It would be nice if you could define on a scenario basis which browser types to run the test in (if it's a test annotated with @javascript). Let's say I want to generally run all scenarios in chrome and firefox, but certain scenarios I only want to run in firefox.

Right now, it seems like I can only configure this globally, and only a single browser type.

[everzet]

The point is - Mink is session oriented. One session is in relation with driver and, as consequense, with browser 1-to-1. So, you really need to just create a new sessions with different browser configurations. To do this, override registerMinkSessions($mink) method in your FeatureContext:
https://github.com/Behat/BehatBundle/blob/master/Context/MinkContext.php#L88
and create as many sessions in Mink as you need (example here).

After that, you'll be able to switch to interesting session with:

@mink:SESSION_NAME
Scenario: ...
  ...

For example:

<?php # .../FeatureContext.php
...
protected function registerSessions(Mink $mink)
{
    parent::registerSessions($mink);

    $mink->registerSession('chrome',   new Session(
        $this->initSahiDriver('chrome', $this->getParameter('sahi'))
    ));
}
...

@mink:chrome
Scenario: JS scenario to be runned inside chrome browser
  ...

By the way, this:

@mink:sahi
Scenario: ...
  ...

will do exactly the same as this:

@javascript
Scenario: ...
  ...

;-)

[schmittjoh]

Does this allow to do things like

@mink:firefox @mink:chrome
Scenario: ...

Sort of what @dataProvider is doing for phpunit, but for sessions?

[everzet]

Nope. Default scenario steps always run against single default Mink session. And with this @javascript or @mink:... you're just switching this system-wide default session to some specific one (by the way, after each scenario, Mink will switch back to system-wide default).

So, your @mink:firefox @mink:chrome will make this scenario to run in chrome (last one).

This @... stuff is just grouping and filtering mechanism. Tags doesn't control testers flow.

[schmittjoh]

What do you think about adding a feature that allows to run the same scenario against more than one session?

For headless browsers that doesn't matter so much, but if I run a scenario in a Sahi session, then I likely want that scenario to pass in all configured browsers (Firefox, Chrome, IE, etc.). I could copy/paste the scenario and apply different sessions, but then I get potentially many scenarios and something like @sessionProvider would be a bit easier.

[everzet]

Yeah, i've got ya. You can do it quite easily with Scenario Outlines for example:

Scenario Outline: Scenario to be runned against different browsers
  Given I use "<browser>" browser
  ...

  Examples:
    | browser |
    | firefox |
    | chrome  |

The only thing left is to implement Given I use "([^"]+)" browser step, which will switch the session.

[everzet]

This problem is solved properly with Behat Profiles. With profiles, you can provide system-wide configuration with filters like this:

firefox:
    filters:
        tags: '@javascript&&~@chrome'
    context:
        parameters: { browser: firefox }
chrome:
    filters:
        tags: '@javascript&&@chrome'
    context:
        parameters: { browser: chrome }

With such configuration:

$ behat -p firefox

will run all scenarios with tag @javascript, but without @chrome tag.

$ behat -p chrome

will run all scenarios with both @javascript and @chrome tags.

So, calling two this commands:

$ behat -p firefox
$ behat -p chrome

Will run scenarios in both browsers, providing useful output.

The problem is, there's no profiles in BehatBundle. Maybe we should add them... What do you think?

[stof]

👍 for profiles in BehatBundle

[vglushonkov]

Hello everyone. This issue is 10 months age, so maybe it is why i can't make my Behat run tests in different from Chrome browser (why not firefox, i don't really know).
My task is to run ALL the tests i have with different browsers. Is it achievable thru the command line of Behat, ot Sahi, anyhow?
I tried to #38 (comment) but it still says "session ff is not registered". As long as i understang, this soliution requires me to create 4 sets of identical tests, but with @mink: ff, chrome, ie and opera. This is not really what i need. Maybe there is a way to set Sahi to run a browser type i want before running behat tests?

[stof]

@iamexe as pointed 10 months ago by @everzet in his latest comment, the answer is to use behat profiles, and launching the behat features once for each browser you want to test.
Profiles were not supported in BehatBundle because of the way the bundle was working, but this flaw disappears for Behat 2.4 as the bundle will now die in favor of the Symfony2Extension for Behat.

[vglushonkov]

You say if I have behat 2.4 (I have 2.3.3) I need just to add firefox profile to behat.yml like this:
parameters: { browser: firefox }
and then just run my CL like this?
C:\projects\oli-port\repo\branch>php app/console --env=test behat src\Ailove\OlimpBundle\Features\Regression\TeamSochiProducts.feature -f html --out ....\test_results.html --profile ff

[stof]

yes, and it works in Behat 2.3 standalone too (but not when using Behat 2.3 through BehatBundle)

[vglushonkov]

Thanks, everything works fine