Grails 3.1.x Async Reactor based Tasks Fail sporadically

[davydotcom]

If running 1000 reactor grails.async.Promises.task in succession , they will not all complete and hang forever..

Steps to Reproduce

1. Create fresh grails 3.1.12 app
2. Run the following script somewhere in the app (Bootstrap was a good place to test or a controller)

            def total = 0
            def tasks = []
            for(int i=0; i < 1000 ; i++) {
            tasks << Promises.task {
               def b = 1
               return b
             }.onComplete { res ->
                println "On Complete ${i}"
               total++
             }
            }

            Promises.waitAll(tasks)
            println "Result ${total}"

This will hang forever... total will never outright = 1000 because of concurrent write on same memory buffer so thats expected.

1. Try again by changing promiseFactory back to legacy gpars:

Promises.promiseFactory = new GparsPromiseFactory()

Promises will complete successfully

Expected Behaviour

All instructed promises should complete successfully and return a promise result appropriately

Actual Behaviour

Promise tasks are lost and sometimes not even executed

Environment Information

• Operating System: Mac OS X
• Grails Version: 3.1.12 (all the way back to at least 3.1.6)
• JDK Version: 8

[jeffscottbrown]

Related to but not a duplicate of #10152.

[graemerocher]

This is probably just down to implementation differences. Reactor by default uses a thread pool that contains the same number of threads as processors. If you overwhelm that threadpool then you could get weird behavior like you are seeing. It may be you have to tweak the reactor configuration to your needs.

See the config options http://docs.grails.org/latest/guide/async.html#events

[davydotcom]

shouldn't it queue up the tasks to be run within the context of a thread pool?

[davydotcom]

I should mention i can get this to fail with just setting the counter to 3 or 4 threads and I know the default thread pool as bigger than that

[graemerocher]

The issue appears to be down to Reactor internals. Given that Reactor 2 is no longer maintained and it is unlikely we will be upgrading to Reactor 3 it makes sense to deprecate Reactor-based promises and restore Gpars as the default.

The Event bus part of Grails will continue with Reactor until we implement a bridge layer to GPars that can facilitate the complete removal of Reactor from core.

[aaloise]

@graemerocher , can I suggest Vert.x EventBus as an alternative?

http://vertx.io/docs/vertx-rx/java/
http://vertx.io/docs/vertx-core/java/#event_bus

Do you think it's a valid option?

[graemerocher]

it may be, like I said the event bus notion needs to be extracted out of Grails and then there could be Reactor/Vert.x/RxJava/Spring Event implementations that back that

[goeh]

(sorry for commenting on a closed issue, is there an issue created for new event bus in Grails 3.x?)

A Grails event abstraction sounds like a good idea. I have good experience with event bus implementations like Reactor 1.x, Vert.x and plain Spring application events. Not so good experience with Reactor 2.x.

One thing I really like is TransactionalEventListener in Spring 4.2+ (http://docs.spring.io/spring/docs/current/javadoc-api/org/springframework/transaction/event/TransactionalEventListener.html).

When designing a new event abstraction in Grails I want us to have (GORM) transactions in mind. The event should not be fired until the transaction it was created in is committed. (optional of course).

[burtbeckwith]

That would be great, and would support workflows that are only practical now with XA and two-phase commit, e.g. sending a JMS message with data based on a change from the current database update(s) where the message isn't actually sent if there's a transaction rollback.

[aaloise]

@goeh here is the issue #10273