Skip to content

Fire events when a Default Task is started, finished, or cancelled#431

Closed
NicoKiaru wants to merge 7 commits intoscijava:masterfrom
NicoKiaru:autotaskupdate
Closed

Fire events when a Default Task is started, finished, or cancelled#431
NicoKiaru wants to merge 7 commits intoscijava:masterfrom
NicoKiaru:autotaskupdate

Conversation

@NicoKiaru
Copy link
Contributor

One question : should isDone be set to true when the task is cancelled ?

@NicoKiaru NicoKiaru marked this pull request as draft March 19, 2022 22:21
@NicoKiaru
Copy link
Contributor Author

NicoKiaru commented Mar 19, 2022

I've added a test because it seems there could be some issues when many tasks are executed in parallel.

Maybe it's not that critical, but maybe it is. The test TaskEventTest is a parallel stress test which starts thousands of tasks (not doing much) almost at the same time. An event listener listens to task events, and adds a task each time a new one is detected, and removes it each time a task is done (basically it's a counter of active tasks). After a delay long enough, normally, no task is left. The counter should give back 0.

However, it seems that from time to time, either a few tasks are not done, or many are not done. Try the test a few times, and you will see that it fails (around 1 time every 3 or 4 times, with java jdk 1.8.0_322 aws corretto, Win 10).

Here's a few issues I had by looking at visual vm Thread dumps, showing why some tasks/threads are stuck:

I think the first situation is when there are way too many threads (I've tested with 5000), but here the thread pool reaches a limit (around 3700 in my configuration):

"SciJava-5a8e6209-Thread-3078" #8144 prio=5 os_prio=0 tid=0x000001ca2616b800 nid=0x1bc28 waiting on condition [0x00000058192fe000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at sun.misc.Unsafe.park(Native Method)
        - parking to wait for  <0x00000006c307f780> (a java.util.concurrent.SynchronousQueue$TransferStack)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

   Locked ownable synchronizers:
        - None

Sometimes, System.out.println creates this issue - especially if you put a println within the task:

"Thread-2042" #2060 prio=5 os_prio=0 tid=0x0000021fee14f800 nid=0x32b3c waiting for monitor entry [0x0000003f02cfe000]
   java.lang.Thread.State: BLOCKED (on object monitor)
        at java.io.PrintStream.println(PrintStream.java:805)
        - waiting to lock <0x00000006c3046688> (a org.scijava.console.MultiPrintStream)
        at org.scijava.ui.swing.task.DemoTask.lambda$createTask$1(DemoTask.java:52)
        at org.scijava.ui.swing.task.DemoTask$$Lambda$64/521960438.run(Unknown Source)
        at java.lang.Thread.run(Thread.java:750)

   Locked ownable synchronizers:
        - None

Another potential Thread issue:

"SciJava-5a8e6209-Thread-3471" #3492 prio=5 os_prio=0 tid=0x000002d292ff3800 nid=0x3483c runnable [0x000000f6dcffe000]
   java.lang.Thread.State: RUNNABLE
        at java.util.WeakHashMap.remove(WeakHashMap.java:610)
        at org.scijava.thread.DefaultThreadService.lambda$wrap$1(DefaultThreadService.java:214)
        at org.scijava.thread.DefaultThreadService$$Lambda$6/1843368112.run(Unknown Source)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:750)

   Locked ownable synchronizers:
        - <0x00000006c313d460> (a java.util.concurrent.ThreadPoolExecutor$Worker)

That's hard to debug. Do we care ? I think yes. It's important to get consistently no error with a thousand parallel tasks.

@NicoKiaru NicoKiaru closed this Mar 29, 2022
@ctrueden
Copy link
Member

@NicoKiaru Could you remind me why this PR ended up getting closed? Did we discuss and decide not to go this direction? I don't remember.

@NicoKiaru
Copy link
Contributor Author

I rewrote the PR a bit more cleanly in #432 (merged). I should have mentioned it when closing this PR.

Regarding the many threads issue, I think it's ok, we're not supposed to start this many threads directly with new Thread. Maybe when / if loom is available, we will be able to launch millions of virtual threads!

@NicoKiaru
Copy link
Contributor Author

I've summarised the status of the 'task feature' in https://forum.image.sc/t/demo-and-proposal-new-progress-bars-for-fiji/64956/5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants