Support for parallel test configuration

[marchof]

I've maven-surefire plugin configured with this options :
Concurrency config is parallel='classes', perCoreThreadCount=true, threadCount=4, useUnlimitedThreads=false

This means, that maven runs 4 separate JVM, each JVM has own jacoco-agent and all agents needs to write to the same target/jacoco.exec file.

After all tests are finished, I've malformed target/jacoco.exec file.

Caused by: java.io.UTFDataFormatException: malformed input around byte 0
    at java.io.DataInputStream.readUTF(DataInputStream.java:639)
    at java.io.DataInputStream.readUTF(DataInputStream.java:547)
    at org.jacoco.core.data.ExecutionDataReader.readExecutionData(ExecutionDataReader.java:145)

Is there a synchronisation/write locking for target/jacoco.exec ?

I think that this is continuation of http://sourceforge.net/apps/trac/eclemma/ticket/191

[flozano]

Here we reproduced this problem with a different error message, I will add it as a reference for others who find similar problem:

java.io.IOException: Unknown block type b.
        at org.jacoco.core.data.ExecutionDataReader.readBlock(ExecutionDataReader.java:114)
        at org.jacoco.core.data.ExecutionDataReader.read(ExecutionDataReader.java:85)
        at org.sonar.plugins.jacoco.AbstractAnalyzer.readExecutionData(AbstractAnalyzer.java:76)
        at org.sonar.plugins.jacoco.AbstractAnalyzer.analyse(AbstractAnalyzer.java:59)
        ... 38 more

[Godin]

@marchof Indeed we can lock file, when we dump coverage (i.e. for a normal scenario - during JVM shutdown) - http://docs.oracle.com/javase/1.4.2/docs/api/java/nio/channels/FileChannel.html#lock() Do you see a drawbacks or side effects of such solution? For example - I don't know what happens with locks if JVM was killed or crashed. Or we'd like to avoid locks (i.e. concurrent access to dump file) and should find another solution to this problem?

[marchof]

Let's give it a try. Writing exec data is quite fast, so waiting for the exclusive lock shoudn't be an issue. Hopefully the OS frees the lock once the process goes away -- but we can test this.

[marchof]

@Godin Hi Evgeny, the file lock was an excellent idea and quite simple to implement. Do you have a parallel test example at hand?

[Godin]

@marchof Yes, I have. So I will do the tests.

[Godin]

But even without tests - this method returns FileLock which should be released.

[marchof]

JavaDoc says: "It remains valid until the lock is released by invoking the release method, by closing the channel that was used to acquire it, or by the termination of the Java virtual machine, whichever comes first."

I verified this in OS X: As soon as I close the FileOutputStream another process can acquire the lock.

[Godin]

Indeed - FileOutputStream.close closes the channel. But now what about your second statement about concurrent dump operations?

[marchof]

Sorry, I didn't forget to mention this: Probably we're already safe here: The method

org.jacoco.core.runtime.RuntimeData.collect(IExecutionDataVisitor, ISessionInfoVisitor, boolean)

already is synchronized, so I don't think parallel writes from the same agent are possible.

[Godin]

Same feelings. But anyway I'll do more tests (including real-life) tomorrow morning and will come back to you about merge.

[Godin]

@marchof Works fine for me, so lets merge!

[Raynevun]

Still reproducible for me, when using TeamCity 9.1.1 (build 37059 which should contain JaCoCo 0.7.5) integration and running unit and integration tests in parallel with two agents writing in one exec file.

[Godin]

@Raynevun The best place to get support is our users mailing list - https://groups.google.com/forum/#!forum/jacoco , so could you please switch to it? And describe problem more detailed, at least provide stacktrace of exception, also maybe you know a way to reproduce this issue without need to install TeamCity?

Thanks for your understanding.