Feature request: priorities for tasks and threads

[ronisbr]

Hi!

I am having a problem when working with threads and tasks in Julia. In my embedded application, I have three tasks. One is very important, and the other two are not. To meet the real-time requirement of the first, I currently need to write a lot of yield() in the threads that are not that important. Even so, I cannot guarantee that, when yield() is called, the next thread to be executed will be the first one. Hence, I would like to propose a schedule priority feature for Julia.

It can be something very simple, like this:

When the scheduler is called, list all runnable threads and execute the one with the highest priority. If there are multiple threads with the same priority, then execute the one that is waiting longer on the list.

This kind of approach will solve at least my problem. I will still need a lot of yield()s (maybe I can even construct a macro to add a yield() after every newline), but I will be sure what will be the next thread to be run.

[felipenoris]

One requirement to keep in mind is wether you need preemptive tasks or not. Even if you had the priority feature in Threads, you may still need to wait until a task is finished to start a new priority task in case all threads are busy, which is not a good thing for real time applications (and then you start using yield everywhere).

Distributed computing using the Distributed stdlib is preemptive given that you have different OS processes. Each process has its own pool of threads.

What I've been doing to mimic this feature is to mix the two. I have a master and slave processes using the Distributed stdlib. All priority work runs on master, and the rest is scheduled in the slave process.

[ronisbr]

Hi @felipenoris

Very interesting, thanks! I will look into this combination with Distributed. Nevertheless, I still think it would be important to have a priority scheduling of threads to let us select which thread must run when the current one sleeps.

[quinnj]

Just as an FYI, I made a very lightweight package that allows spawning multithreaded tasks on a non-main thread. i.e. it will keep a single main thread free to respond to other tasks/things, and only extra "worker" threads will be eligible to perform @spawn-ed tasks. It works well in the application/microservice setup where it's important to keep a "main" thread open to respond to health checks, heartbeats, etc. ref: https://github.com/JuliaServices/WorkerUtilities.jl

[felipenoris]

Hi @quinnj ! Thanks for the info! I noticed that package uses a strategy similar to the MusicAlbums project featured in your JuliaCon seminar.
I used that strategy for sometime and I ran into a few problems: you need to sacrifice 1 core to be idle all the time waiting for requests, and it does not play well with code that spawn other threads (sometimes I could see that the scheduler hanged in some point, not using all CPUs available). In my experience, any mix of Threads.@threads and Threads.@spawn leads to problems.
Now I use only Threads.@spawn whenever I need to schedule a new task, and mixing with Distributed allows me to use all CPU cores.