Skip to content

Possible race condition with TempDir and parallel mode #1282

Closed
#1283
Closed
Possible race condition with TempDir and parallel mode#1282
#1283
@smancill

Description

@smancill
smancill
opened on Feb 28, 2021

Issue description

Using @TempDir with parallel mode enabled throws an exception.

java.nio.file.NoSuchFileException: /var/folders/kk/59b8hjx139qb_kk7fbhw9b_w0000gn/T/spock_Try_creating_multip_1_tmpDir7390812081439575953
        ...
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.tryMakeWritable(TempDirInterceptor.java:104)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.deleteTempDir(TempDirInterceptor.java:99)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.destroy(TempDirInterceptor.java:76)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.intercept(TempDirInterceptor.java:86)
        ...

It seems a race condition because sometimes the tests pass without issues. I see no errors with parallel mode disabled.

Full report for one of the failed runs (the failed tests vary on different runs) Captura de Pantalla 2021-02-28 a la(s) 19 56 00

Do nothing just test the temp dir existance

java.nio.file.NoSuchFileException: /var/folders/kk/59b8hjx139qb_kk7fbhw9b_w0000gn/T/spock_Do_nothing_just_tes_1_tmpDir7972582930599896650
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
	at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1763)
	at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
	at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2716)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2796)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.tryMakeWritable(TempDirInterceptor.java:104)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.deleteTempDir(TempDirInterceptor.java:99)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.destroy(TempDirInterceptor.java:76)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.intercept(TempDirInterceptor.java:86)
	at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:101)
	at org.spockframework.runtime.model.MethodInfo.invoke(MethodInfo.java:136)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
	at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

Do nothing just test the temp dir existance this time with unroll

java.nio.file.NoSuchFileException: /var/folders/kk/59b8hjx139qb_kk7fbhw9b_w0000gn/T/spock_Do_nothing_just_tes_1_tmpDir7972582930599896650
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
	at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1763)
	at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
	at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2716)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2796)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.tryMakeWritable(TempDirInterceptor.java:104)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.deleteTempDir(TempDirInterceptor.java:99)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.destroy(TempDirInterceptor.java:76)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.intercept(TempDirInterceptor.java:86)
	at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:101)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
	at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool.awaitJoin(ForkJoinPool.java:1708)
	at java.base/java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:397)
	at java.base/java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:721)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.joinConcurrentTasksInReverseOrderToEnableWorkStealing(ForkJoinPoolHierarchicalTestExecutorService.java:162)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService.invokeAll(ForkJoinPoolHierarchicalTestExecutorService.java:136)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:143)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:129)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
	at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

Try creating multiple dirs

java.nio.file.NoSuchFileException: /var/folders/kk/59b8hjx139qb_kk7fbhw9b_w0000gn/T/spock_Try_creating_multip_2_tmpDir5062398322594910498
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
	at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:149)
	at java.base/java.nio.file.Files.readAttributes(Files.java:1763)
	at java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
	at java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
	at java.base/java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2716)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2796)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.tryMakeWritable(TempDirInterceptor.java:104)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.deleteTempDir(TempDirInterceptor.java:99)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.destroy(TempDirInterceptor.java:76)
	at org.spockframework.runtime.extension.builtin.TempDirInterceptor.intercept(TempDirInterceptor.java:86)
	at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:101)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:127)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:126)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:84)
	at org.junit.platform.engine.support.hierarchical.ForkJoinPoolHierarchicalTestExecutorService$ExclusiveTask.compute(ForkJoinPoolHierarchicalTestExecutorService.java:185)
	at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

Try creating multiple dirs [name: bbb, 1]:

java.nio.file.NoSuchFileException: /var/folders/kk/59b8hjx139qb_kk7fbhw9b_w0000gn/T/spock_Try_creating_multip_1_tmpDir1175750083630960256/bbb
	at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
	at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116)
	at java.base/sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
	at java.base/java.nio.file.Files.createDirectory(Files.java:689)
	at TempDirSpec.Try creating multiple dirs(TempDirSpec.groovy:16)

How to reproduce

I have created this test case:

@TempDir Path tmpDir

def "Try creating multiple dirs"() {
    when:
    var dir = Files.createDirectory(tmpDir.resolve(name))

    then:
    Files.exists(dir)

    where:
    name << ["aaa", "bbb", "ccc"]
}

def "Try creating single dir"() {
    when:
    var dir = Files.createDirectory(tmpDir.resolve("foo"))

    then:
    Files.exists(dir)
}

def "Do nothing just test the temp dir existance"() {
    expect:
    Files.exists(tmpDir)
}

def "Do nothing just test the temp dir existance this time with unroll"() {
    expect:
    Files.exists(tmpDir)

    where:
    n << [1, 2, 3]
}

and in SpockConfig.groovy:

runner {
    parallel {
        enabled true
        defaultSpecificationExecutionMode = ExecutionMode.CONCURRENT
        defaultExecutionMode = ExecutionMode.SAME_THREAD
    }
}

It seems to be fixed when @Execution(ExecutionMode.SAME_THREAD) is added to the spec. But I do have that setting in the config file, so I am not sure if the problem is with @TempDir, or SAME_THREAD declared in the config is actually not being used?

Link to a branch with the failing spec

I've used the spock-example repository to reproduce the problem, and I've pushed a branch here:

https://github.com/smancill/spock-example/tree/parallel-mode-tempdir

Running ./gradlew should fail the tests, but it cannot be reproduced consistently and it may require a few ./gradlew cleanTest test runs.

Additional Environment information

Java/JDK

Using the spock-example project, I can reproduce it with Java 11, 14 and 15.

Groovy version

The dependency declared by the spock-example project: org.codehaus.groovy:groovy:3.0.7

Build tool version

Gradle

------------------------------------------------------------
Gradle 6.7.1
------------------------------------------------------------

Build time:   2020-11-16 17:09:24 UTC
Revision:     2972ff02f3210d2ceed2f1ea880f026acfbab5c0

Kotlin:       1.3.72
Groovy:       2.5.12
Ant:          Apache Ant(TM) version 1.10.8 compiled on May 10 2020
JVM:          11.0.9 (Oracle Corporation 11.0.9+11)
OS:           Mac OS X 10.16 x86_64

Operating System

macOs

Build-tool dependencies used

Gradle

testCompile platform("org.spockframework:spock-bom:2.0-M4-groovy-3.0")
testCompile "org.spockframework:spock-core"

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions