Problem producing aggregate reports for a microservices project

[nbarrett]

On my client's microservices project, I'm trying to get the most out of Functional Coverage Overview part of Serenity 2.x. The current challenge is finding a configuration setup that will allow a user to drill down from the index page into the areas of interest - and ideally to have the Specifications > Scenarios populated from the requirements. This is proving hard to achieve however, as both the test run and the aggregation stage present their own issues, for the reasons explained in the following sections.

Firstly, the project structure, being a monorepo, is laid out as follows:

universe
  +- microservice 1
              +-src/test/resources/some/arbitrary/path/down/to/cucumber
  +- microservice 2
              +-src/test/resources/some/other/arbitrary/path/down/to/cucumber
...
  +- microservice 99
              +-src/test/resources/yet/another/arbitrary/path/down/to/cucumber

(1) Problem at the microservice test phase
Each microservice has its own acceptance test suite with a serenity.conf as follows:

serenity {
  outputDirectory = buck-out/serenity-unit-integration
  requirements.dir = ${user.dir}/my/client/project/universe/microservice_1/src/test/resources/cucumber
}

The only way we can get Scenario details and other document specific content to populate correctly during the test phase is if serenity.outputDirectory is set to an absolute path by means of ${user.dir}. Using src/test/resources/cucumber doesn't work as the test suite is run from the monorepo universe root directory.

(2) Problem at the aggregation phase
Whilst the aggregation does produce test results (as microservices do produce output to a common location due to serenity.outputDirectory being the same), no Functional Coverage Overview is rendered because there isn't a single value that can be supplied for serenity.requirements.dir that will allow requirements for all microservices to be resolved into a single tree structure. If nothing, or the root directory universe is supplied, then this causes the aggregation to hang indefinitely due to #1252. But even if this didn't hang, the structure would still not look right or map to the test results as the individual microservices have their serenity.requirements.dir at arbitrary points in the monorepo directory structure.

Can you think of something we can do on our project to overcome this problem as we are having quite a problem selling the new look reports to the users as they feel they are not quite as easy to navigate any more (hence our release of 2.x is blocked at the moment).

[nbarrett]

Afterthought: If the requirements directory structure was 'materialised' into the test output during the test phaes, then the aggregation phase would not have to care about the location of serenity.requirements.dir as it would automatically be placed into a consistent location in the results?

[wakaleo]

That would only work if you don't care about feature definitions or other higher level documentation, which cannot be stored with the tests.

[nbarrett]

Oh, I was visualising test requirements files being copied into a fixed requirements subdirectory under serenity.outputDirectory, not being embedded in the json.

[wakaleo]

That’s not a bad idea, for Cucumber at least.