Concurrency in Enso

[radeusgd]

I was writing a heuristic for dropping unused temporary tables in our Database driver and had to learn a bit about Enso's concurrency and finalizers. Sharing my insights below.

Enso supports multithreaded execution

That's what I suspected, but I have never really tried it in practice.

from Standard.Base import all

polyglot java import java.lang.Thread
polyglot java import java.util.concurrent.ConcurrentLinkedQueue

type My_Runnable
    Instance v x

    run self =
        0.up_to 100 . each _->
            self.v.add self.x

main =
    vb = ConcurrentLinkedQueue.new

    i0 = My_Runnable.Instance vb "0"
    t1 = Thread.new (My_Runnable.Instance vb "a")
    t2 = Thread.new (My_Runnable.Instance vb "b")

    t1.start
    t2.start
    i0.run

    v = Vector.from_polyglot_array vb.toArray
    IO.println (v.join "")

will print something like:

ab0ababababaababababaaa0b0b0b00bb0bbb0b0bb0ba0b0b0b0ab0a000aa0aabaabaaba0aba0baab0aabaab0ababa0aba0abaab0aaba0aaab0ab0a0ba0baab0aa0aa0ab0aba0ba0baabababa0aabaa0baba0ba0aba0bab0abba0bab0aba0ab0abab0abab0aaaaaaaa0a0baabb0bab0aba0a0b0b0b0b0b00bbb0b0b0b00bb0b0b0b0b0bbb0bb0b0bb0b0000000000000000000000000

showing that the 2 new threads are running in parallel to the main thread.

We can use locks to synchronize

We can modify the snippet from above to add locks for synchronization:

from Standard.Base import all

polyglot java import java.lang.Thread
polyglot java import java.util.concurrent.ConcurrentLinkedQueue
polyglot java import java.util.concurrent.locks.ReentrantLock

type My_Runnable
    Instance l v x

    run self =
        self.l.lock
        0.up_to 100 . each _->
            self.v.add self.x
        self.l.unlock

main =
    vb = ConcurrentLinkedQueue.new
    l = ReentrantLock.new

    i0 = My_Runnable.Instance l vb "0"
    t1 = Thread.new (My_Runnable.Instance l vb "a")
    t2 = Thread.new (My_Runnable.Instance l vb "b")

    t1.start
    t2.start
    i0.run
    t1.join
    t2.join

    v = Vector.from_polyglot_array vb.toArray
    IO.println (v.join "")

yielding something like:

0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb

Finalizers run on the main thread so we CANNOT use re-entry locks to synchronize them!

The first part of that sentence is probably well known to ones more familiar with the ResourceManager. It schedules to run the finalizers at a safepoint, thus they are being run on the main thread. The safepoints may happen in more or less arbitrary places during execution - there are some states during which a safepoint will not 'occur', but in general it can 'happen' between Enso "instructions" / method calls.

So the safepoints (that's their whole idea) can happen more or else anywhere during execution of Enso code. And the finalizers are run during these safepoints.

That tells us that extra care has to be taken if the finalizer contains logic that may influence other 'parts' of the system. Probably it's best to just call .close on a resource and not do much more. If doing more - the logic has to be implemented very carefully.

In particular, we cannot just use the most common [citation needed 🙂] type of lock used in Java code - ReentrantLock. That is because, since a finalizer is run on the main thread - it will usually be run on the same thread as regular code. So if the regular code and finalizer logic were to be synchronized using a such a re-entrant lock, when the regular code is executing some action holding the lock, the finalizer may 'interrupt' it to run its logic - it will then attempt to take the lock and it will succeed! because it is running on the same Thread that is already holding that lock - so due to the 're-entrant' property, it will be let in.

In conclusion, if finalizer contains logic that needs to be synchronized with regular code, it needs to rely on some other synchronization primitives than a ReentrantLock.

Example proving this in practice:

from Standard.Base import all
import Standard.Base.Runtime.Managed_Resource.Managed_Resource

polyglot java import java.util.concurrent.ConcurrentLinkedQueue
polyglot java import java.util.concurrent.locks.ReentrantLock

type My_Runnable
    Instance l v x

    run self =
        self.l.lock
        # This needs to be sufficiently large to ensure that GC will run in the meantime.
        0.up_to 1000 . each _->
            self.v.add self.x
        self.l.unlock

type My_Resource
    Instance l v x

make_resource l v x =
    r = My_Resource.Instance l v x
    Managed_Resource.register r cleanup_resource

cleanup_resource r =
    r.l.lock
    0.up_to 10 . each _->
        r.v.add r.x
    r.l.unlock

allocate_and_dispose_resource l v x =
    r = make_resource l v x
    r.with res->
        v.add "["+res.x+"]"
    Nothing

main =
    vb = ConcurrentLinkedQueue.new
    l = ReentrantLock.new

    i0 = My_Runnable.Instance l vb "o"
    allocate_and_dispose_resource l vb "1"
    allocate_and_dispose_resource l vb "2"
    allocate_and_dispose_resource l vb "3"

    i0.run

    v = Vector.from_polyglot_array vb.toArray
    IO.println (v.join "")

may result in e.g.:

[1][2][3]ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo111111111133333333332222222222ooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo

which shows us that while the main thread is holding the lock for writing the os, the 3 finalizers happily interrupt it, even though they too acquire the lock.

[JaroslavTulach]

The change to run "finalizers" at a safepoint was implemented by #6335

[JaroslavTulach]

An interesting sub-topic of the concurrency discussion:

Blocking REST Queries in Visualizations

We are using visualizations infrastructure more and more. Not only to compute visualizations, but also to compute type and content of drop-down widgets. Possibly we could do even more - for example issue a query to some REST API and only when it returns back produce the visualization result. However such additional usages may negatively affect the execution of the original user program. For example, an HTTP request is currently synchronous and blocking in Enso. While the REST endpoint is being queried no other computation can happen - e.g. the execution of user program is blocked.

Callbacks or (Virtual) Threads

There are many ways to solve this problem, but all of them involve some form of concurrency. Either a node.js callback+event queue like solution or support to run in multiple threads. Enso has always been designed as a multi-threaded language (demonstrated by the finalizers requiring its own thread prior to #6335). Moreover the Java 21 (to be released during summer) will officially support virtual threads and support for them will be an integral part of Truffle soonish. Support for virtual threads renders callback & event queue programming style obsolete - we should probably focus on running Enso in multiple threads safely.

Run (selected) visualizations in parallel

The simplest way to execute an Enso visualization querying an HTTP service then is: make sure the visualization is computed in its own (virtual) thread since the begging till the end. That way we can use existing (blocking) Enso APIs while avoiding blocking the computation of user program by these non-primary visualizations. All that we need is to support Enso execution in multiple threads properly and an extension to Enso language server protocol to opt-in and execute a visualization in parallel.