Multi-browser x Multi-environment tests get overwritten by last environment

[xmj]

Hi,

I'm trying to run a multi-environment (lab, dev) x multi-browser (edge, chrome) setup.

I've been able to do multi-browser successfully using cli command injection (as the serendipity.conf doesn't let me specify multiple drivers yet, discussed in #3077 (comment))

serenity.conf contains driver capabilities for both ms:edgeOptions & goog:chromeOptions, and environment specifications for lab and dev (and some more).

The cli calls look like this:

selenium-edge container:

export GRADLE_ARGS='--no-daemon -Dinfluxdb.token=$INFLUXDB_TOKEN'
export BROWSER_ARGS='-Dwebdriver.remote.url=http://selenium_edge:4444 -Dwebdriver.driver=edge -Dcontext=edge -Dwebdriver.capabilities.browserName=edge -Dinjected.tags=browser:edge'
gradle $GRADLE_ARGS $BROWSER_ARGS -Denvironment=lab  -Dinjected.tags=environment:lab test || true

selenium-chrome container:

export GRADLE_ARGS='--no-daemon -Dinfluxdb.token=$INFLUXDB_TOKEN'
export BROWSER_ARGS='-Dwebdriver.remote.url=http://selenium_chrome:4444 -Dwebdriver.driver=chrome -Dcontext=chrome -Dwebdriver.capabilities.browserName=chrome -Dinjected.tags=browser:chrome'
gradle $GRADLE_ARGS $BROWSER_ARGS -Denvironment=lab  -Dinjected.tags=environment:lab test || true

Artifacts from the edge container get fetched into the chrome container, so I get a nice single-page-report with both browsers inside.

That's already a good start.

As soon as I add a second gradle call to each script with -Denvironment=dev -Dinjected.tags=environment:dev I can verify in the logs that all 4 gradle calls run successfully, but the single-page report will only contain the last environment's results.

That is, the artifact from selenium-edge will contain results only for the dev environment (with lab being overwritten), and the final result will contain both dev results for both browsers (but not the lab results; they vanish).

How do I prevent serenity from overwriting the various permutations already run?

Also, is there a way to test two environments in one cli call?