Calling a Julia function from a non-Julia thread

[barche]

As discussed on julia-users, I would like for it to be possible to call a Julia function from a non-Julia thread:
https://groups.google.com/forum/#!topic/julia-users/TqC79eYTYx8

My use case is calling a Julia OpenGL rendering function from within the QML rendering thread. The current workaround is to force Qt to use the non-threaded rendering engine.

[stevengj]

One way to do this is to have a Julia task waiting to receive requests from your thread. You can send requests from another thread by calling uv_async_send, which is thread-safe.

There was some discussion of this in the julia-users mailing list, and this mechanism is used in ZMQ.jl to some extent.

[stevengj]

See also #5657, which added this discussion of thread-safe callbacks to the manual.

[StefanKarpinski]

Note that as @yuyichao mentioned, this is a very long-term feature request – even when we have fully non-experimental Julia threading, calling Julia code from a non-Julia thread may still not work. Given that reality, the uv_async_send approach is likely to be the most workable in a useful timeframe for you.

[barche]

@stevengj Thanks, I had actually seen that in the docs, but I wonder if it would lead to a deadlock here? The QML application is started from Julia, blocking the main Julia process (i.e. the REPL is unresponsive). Would the request to render then not block until the QML app is closed?

@StefanKarpinski I realise that now, thanks. I'm somewhat reassured by the Qt docs that the option for single-threaded rendering is also used in other circumstances and not likely to go away soon.

[StefanKarpinski]

If the Julia thread is "blocked" in the uv event loop, then it's not really blocked in the sense that C code is blocked – i.e. in the middle a call to the kernel and unable to do anything. So sending work to it via uv_async_send should cause it to do that work. If that doesn't work for some reason, please file issues and we'll make sure that it does work since this use case is quite common and important.

[stevengj]

@barche, if the main thread is blocked in the Qt event loop, not the Julia event loop, then you indeed would have a deadlock problem. Combining two event loops is always problematic. Three options:

• Run the Qt main event loop (i.e. start the QML app) in another thread.
• Run the Julia event loop, not the Qt event loop, but call a Qt event handler every 50ms or so in Julia. (This is what PyCall does to stay responsive while a Qt-based GUI is running.) Since the Qt event loop is mainly responding to human inputs, a 50ms latency is not too noticeable.
• Run the Qt event loop, but call jl_yield every once in a while to handle Julia tasks.

You can also find some other discussion online of merging libuv and Qt event loops, but I haven't seen anything very elegant.

[vchuravy]

For me the uv_async_send approach works, but it has drawbacks that I would like to get around.

1. Until the message is delivered all subsequent messages are ignored (at least that is what I understood from the libuv documentation)
2. The call into Julia doesn't block the C++ side so it needs extra synchronisation between the two runtimes.
3. Sending data along is fragile: https://github.com/JuliaGPU/OpenCL.jl/blob/716add3c4315727ff611cc3ac1b6b086be909a95/src/event.jl#L97-L140

I am aware that the things I am allowed to do a very limited in the cfunction that is called, but would it be feasible to allow the function to acquire locks? That might solve 1. and 2. for me.

[vtjnash]

It can acquire pthread locks, the only restriction is that it can't interact with the julia runtime.

[barche]

OK, I've experimented with uv_async_send, and unfortunately there is a deadlock in this case. I can't find a way to make the main Julia thread -which is executing the Qt event loop- receive the uv_async_send while at the same time have the rendering thread wait for the callback to complete.

@stevengj Running the Qt event loop in another thread (using @threadcall for example) would be interesting, but it would not work on OS X, since there the GUI thread must be the main thread. Also, every call into Julia would have to go through uv_async_send. I already have an option to run the GUI in the background using a uv_timer, but it is indeed not very elegant.

[yuyichao]

If it's waiting in the Qt event loop, use the qt equivalent of uv_async_send

[barche]

@yuyichao Just tried this, the deadlock remains, so it seems that is purely Qt related and I'll not clutter the discussion further with this.

[barche]

It seems that in practice a crash happens in jl_gc_pool_alloc when trying to call a function from an outside thread. This function contains the comment

// FIXME - need JL_ATOMIC_FETCH_AND_ADD here

Would fixing that be a major part of the solution, or is that just the first of a long series of required steps?

[yuyichao]

No. That change is undesired right now and is completely unrelated to this issue.

[stevengj]

@barche, if the GUI thread needs to be the main thread, why not start the Julia eventloop in another thread?

[barche]

@stevengj I haven't tried that because I couldn't figure out how to do that. Note that the entry point to the code is the julia executable, not a custom program that initialises the embedding interface. I think this fixes the Julia event loop on the main thread.

[stevengj]

@barche, you would have to make the entry point a custom executable that links libjulia. Fortunately, this is pretty easy to do.

[barche]

@stevengj Yes, but it would disturb the workflow for QML.jl, where you can just launch the GUI from a normal Julia program now. Even with a custom main application, I would still need to make an async call into Julia for every callback from QML, and adding that seems like a lot of work. For now I prefer to await the improvements to the basic Julia threading infrastructure.

The separate main application might be interesting when distributing a full-blown GUI application built on Julia, though, where users would expect to just click an icon rather than firing things up from the command line.

[ravismula]

Is there any work around which works currently?

[StefanKarpinski]

Aside from the uv_async_send technique already mentioned in this thread? No. If there was a simple way around this, we would have just implemented the feature already.

[ravismula]

Rephrasing my question:
Is there a sample which demonstrates uv_async_send?

[stevengj]

@ravismula, it is used in ZMQ.jl for thread-safe deallocation.

[aytekinar]

Is there a plan on writing a simple example in the documentation that demonstrates data sending from a multi-threaded C-API while using the thread-safe uv_async_send?

[cdsousa]

Hi, I've tried to implement the async work sending to the main Julia thread from any C++ thread in https://github.com/cdsousa/embedding_julia/blob/master/julia_cpp_ts.cpp . It seems to work well but I have not used it anywhere besides that test. The work dispatch in the Julia thread side is a little hacky though.

Nevertheless, it seems that the real solution to the original issue would be to enable the registration of external (to Julia) threads with Julia --- setting up TLS, registering with the GC and stacks, etc. It would be better that such new threads won't go to the worker thread pool though. Refs #16134.

[cdsousa]

People, core devs, I've been peeking to the code, trying to get a big picture of the multithreading system, but it has been hard to get it. It would be marvelous to have some schematics on this but I understand that with such source code so much in flux (just like with recent partr addition) that is hard to maintain. I'm trying to realize what is the minimum to add threads during runtime (not available as a worker).

Is there a place where one can understand the architecture of multithreading support, other than the code (and comments) itself?

[bvdmitri]

Is there any progress on this issue? On recent Julia Virtual meetup with @JeffBezanson and @StefanKarpinski it was the top-liked question. In our particular use-case we tried to use audio I/O C-library called libportaudio, which creates a separte thread for audio processing. It is impossible to pass julia algorithm there since it immediatly fails with segmentation fault.

Using uv_async_send is not a solution because it does not allow to pass data and requires additional complex synchronization which is fragile and mostly relies on Julia's internal private c functions. It would be perfect to have something like init_external_thread.

[bramtayl]

Is it possible to implement the uv_async_send strategy as a macro? Something like @threadsafe_cfunction? If a better solution than uv_async_send ever comes about, we could just update the macro.

[bramtayl]

Ok, fishing for help here. I've had limited success in the following strategy inspired by @vchuravy 's code in #17573 (comment)

• Making an Exchange struct to pass
  • the condition handle to the "fake" callback
  • arguments from the "fake" callback to the "real" callback
  • results from the "real" callback back to the "fake" callback
• Passing a pointer to this struct through to the "fake" callback via userdata
• Using unsafe_load to retrieve the Exchange struct in the "fake" callback
• Using unsafe_store! to store the arguments from the "fake" callback in the Exchange struct
• Using uv_async_send on the condition handle in the Exchange struct to trigger a waiting Julia @async task
• In this task, passing the arguments from the Exchange struct into the "real" callback
• Once the real callback is finished, storing its result in the Exchange struct
• Returning the result from the Exchange struct at the end of the "fake" callback

And, um, after all this, was unreasonably excited when I was able to get it to kind of work. I feel like I'm close, but I still need a way for the "fake" callback to wait until the "real" callback is finished. Has anyone figured this out? @vtjnash 's comment

It can acquire pthread locks

seems like a potential answer, if I could figure out what a pthread lock is and how to use it...

[vchuravy]

I recently came up against two libraries where I needed a more principled approach and sat down today
to write https://github.com/vchuravy/ForeignCallbacks.jl -- in brief this uses uv_async_send and the atomics added in 1.7 to communicate a payload to Julia for later processing. It overcomes the 1:1 limitation that previously uv_async_send had (due to the coalescing you could lose messages).

I haven't battle-tested the implementation, but in case others are interested.

[StefanKarpinski]

Wooo!

[barche]

Very nice, many thanks!