Conflicts in measurement filenames when running on parallel CI jobs #291
Description
Context:
In my company, we have the following development pipelines:
The idea is to run the test and the integration tests in parallel (respectively with sbt coverage test and sbt coverage it:test) and after to aggregate both results to get the global coverage of our project
Issue:
We noticed a difference between the coverage obtained locally running sbt coverage compile && sbt coverage test && sbt coverage it:test && sbt coverageReport coverageAggregate and the one obtained on the CI
We identified that the problem was coming from the measurement filename:
def measurementFile(dataDir: String): File = new File(dataDir, MeasurementsPrefix + Thread.currentThread.getId)
On the different machines/cloud instances on which the test and the integration tests are run, some code will be run on threads with the same id (again: on different machines). Having the same name, these files will then collide in the coverage stage when gitlab retrieve the artifacts/cache. Thus losing a some of scoverage-data/scoverage.measurements.* files. Thus losing some coverage %
Evolution:
The idea would be to add a "runtime random id" to the measurement filepath, then the files produced by different machines/cloud instances (ie runtimes) won't collide
I don't see any problems with this solution, so if you see one, please share 😄
I am submitting a PR. I will link it here