Pipeline hangs when all processes execution are skipped

[pditommaso]

See the log file in the above link and the example at #41

https://gist.github.com/yesimon/cf59053d90ee3ea6c700

[pditommaso]

Unfortunately like the previous one I'm not able to reproduce the problem however I'm almost sure that I've patched it.

Please give a try to the patch that I've just uploaded following the usual steps:

export NXF_VER=0.13.4-SNAPSHOT
CAPSULE_RESET=1 nextflow info

It should print this output:

Version: 0.13.4-SNAPSHOT build 2974
Modified: 02-05-2015 17:52 UTC (19:52 CEST)
System: Mac OS X 10.9.5
Runtime: Groovy 2.3.11 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_40-b25
Encoding: UTF-8 (UTF-8)

[pditommaso]

@yesimon I've forgot to mention you. Please have a look to the above comment.

[yesimon]

Ok that specific bug looks like it's fixed, but now I get a different nondeterministic bug. This one crops up much rarer, about 1/10 runs when poolSize is 1. Doesn't seem to occur with larger poolSize but might have to run it more.

N E X T F L O W  ~  version 0.13.4-SNAPSHOT
Launching race.nf
[warm up] executor > local
[skipping] Stored process > proc3 (1)
[skipping] Stored process > proc4 (1)
[skipping] Stored process > proc1 (1)
[skipping] Stored process > proc5 (1)
[skipping] Stored process > proc2 (1)
[skipping] Stored process > proc4 (2)
[skipping] Stored process > proc1 (2)
[skipping] Stored process > proc4 (3)
[skipping] Stored process > proc2 (2)
[skipping] Stored process > proc3 (2)
[skipping] Stored process > proc3 (3)
[skipping] Stored process > proc5 (2)
[skipping] Stored process > proc1 (3)
[skipping] Stored process > proc2 (3)
[skipping] Stored process > proc6 (1)
The actor processing thread has been interrupted Actor Thread 7
java.lang.InterruptedException
        at groovyx.gpars.util.AsyncMessagingCore.run(AsyncMessagingCore.java:133)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:744)
[skipping] Stored process > proc5 (3)
[skipping] Stored process > proc7 (1)
[skipping] Stored process > proc7 (2)
[skipping] Stored process > proc7 (3)

.nextflow.log:
https://gist.github.com/yesimon/cf59053d90ee3ea6c700

[pditommaso]

I think this new error trace is related to the fact that you are using a thread pool with one thread. That does not make a lot of sense in a parallelisation framework. If it arise even when using more than one thread, please open a new issue. I will try to investigate it with the Gpars guys.

[yesimon]

Makes sense. If unfixable throw an error when trying to run with one thread?

[pditommaso]

Actually there are two thread pools. One that executes the user tasks and another used by the Gpars parallelisation library. A better strategy could be to set the poolSize value only for the tasks executor and use the default, that is num_cores +1, for the other.