Multi Threading

Ruby offers a number of options for multi-threaded code - Threads, Fibers and Continuations. Note however, there is no support for parallelism - MRI Ruby will only do one thing at a time.

Threads

Ruby's Threads are its richest form of multithreading, and work as follows:

• They implement cooperative multi-threading - only one thread runs at a time, and it runs until it voluntarily yields its control and another thread begins to run.
• Thread yielding is controlled primarily by the Ruby VM, not by the application. Threads will yield according to a set of rules which mimic preemptive multitasking (e.g. after their slice of time has expired).
• They can invoke blocking calls, and other threads may run while a thread is blocked

In the underlying implementation, these properties are achieved because each Ruby thread is a native OS thread, but they all compete to acquire a global mutex-like object called the Global VM Lock (GVL) before they do any work (so only one Thread runs at a time), and then release the lock under certain circumstances to voluntarily yield control to another Thread.

Thread lifecycle

A thread has a program which it begins executing when the thread is created, and when the program is completed the thread exits. A Ruby program always has at least 1 thread, which is the main thread, and its program is the main sourcefile. Other threads are created by application code with a designated block or Proc as their program.

For non-main threads, their block or Proc runs in a slightly altered execution context. For the purposes of the new thread it is allocated a new special variables region in its local variable context, but the closure on the other local variables is preserved.

When a thread other than the main thread completes, its completion status is made available to other application code:

• If the thread completes normally, the value of its program's statement block is available (through Thread.value)
• If the thread completes by raising an exception out of its program, any other thread that joins on it will have the exception raised out of the join method
• If the thread completes by raising an exception out of its program, for some types of exceptions the exception will also be thrown on the main thread (to be raised at the next interrupt point):
• If the exception is a SystemExit exception
• If the thread's or vm's abort on exception parameter is set
• If the exception is a fatal exception, which is a special type of exception only raised internally by Ruby
• Control transfer instructions??

When the main thread completes (either normally or by throwing an exception), Ruby will begin exit processing, even if other threads are still running.

The main thread has another special property - signal handlers are always invoked on the main thread.

Thread interrupt points

At frequent points in the execution of a Ruby thread, there will be an interrupt point in which the Thread can perform some out-of-band actions:

• It will yield control to another thread, if its timeslice has expired
• If the thread has been instructed to raise an exception (e.g. by another thread), it will do so
• If the thread has been killed, it will begin its kill processing
• If the thread is the main thread, it will run any signal handlers that are pending
• If the thread has been instructed to run finalizers, it will do so

In general, you probably shouldn't rely on knowledge of when an interrupt point will occur. Nevertheless, interrupt points occur at the following points (these are not exhaustive, but they are the most typical interrupt points):

• After method calls - although the Ruby VM will by default optimize away some fundamental method calls unless they are overridden (such as arithmetic operators on numeric values)
• When jumping to a different point of Ruby code as a result of a conditional, loop or control transfer expression
• During blocking, if there is work to be done in the interrupt point (e.g. raise exception, kill, or run signal handlers)

This slightly coarse-grained approach can mean that multi-threaded Ruby code is resilient against some typical race condition scenarios. For example, it appears that a series of simple assignments will be executed atomically as there can be no interrupt point between them.

Thread yield

Threads will yield control to another thread:

• At interrupt points, if the thread's timeslice has expired. The timeslice is calculated based on the thread's priority setting.
• If instructed to do so by application code (e.g. by the Thread.pass method)
• If it is entering a blocking operation, including:
• Locking on a Ruby Mutex object
• Sleeping
• Joining on another thread (i.e. waiting for it to complete execution)
• File descriptor I/O operations such as file read/write, network read/write, joining on a process
• Other blocking operations that may be implemented in the Ruby core or extension libraries

When a thread enters a blocking operation, it will do so atomically with releasing control to another thread. This means, in particular, that e.g. if the next thread to be scheduled sees that the thread is blocked and decides to wake it up, the thread is guaranteed to see the wakeup instruction.

Inter-thread actions

A thread can influence the behaviour of other threads by:

• Raising an exception on another thread - the target thread will raise the exception at its next interrupt point
• Killing another thread - the target thread will begin kill processing at its next interrupt point
• Waking another thread from a block

A thread can be woken up from a blocking operation by:

• Raising an exception on the thread or killing the thread
• Issuing a wakeup instruction to the thread (e.g. Thread.wakeup) - note that only certain types of blocking operation are eligible for wakeup, specifically sleep blocking. Thread join, File descriptor I/O and Mutex lock blocking will not be woken up by this instruction
• When the main thread is in a blocking operation, it will be woken up to run a signal handler if one is pending, however if the signal handler completes without throwing an exception, it will resume the blocking operation

Thread kill

A thread can be killed by application code. A killed thread will notice it has been killed at its next interrupt point, execute any ensure clauses that are applicable, and exit. Effectively, when at the interrupt point it raises a special type of exception that cannot be caught.

The main thread can't be killed - rather, if application code issues a kill instruction on the main thread it executes the default implementation of Kernel.exit(0).

Thread deadlocks

A deadlock occurs when all threads enter certain eligible blocking calls which wait for other threads to do something, but because all threads are blocked, none of the threads will ever achieve the conditions for unblock (short of an OS signal being sent to the Ruby program). Eligible calls include:

• Thread joins
• Mutex locks
• Certain unlimited-time sleep calls (e.g. Thread.stop but not Kernel.sleep)

If all living threads enter such blocking states, Ruby will detect it and raise a deadlock detected exception on the main thread, whose class is an anonymous subclass of Exception.

Fibers

Ruby's Fibers work as follows:

• Similar to Threads, they implement cooperative multi-threading, however the application code controls if and when the running fiber will yield to another fiber
• Fibers are associated with a specific thread. Each thread has at least one fiber, and fibers can't yield to fibers on different threads.
• If a fiber invokes a blocking call, the whole thread will be blocked i.e. other fibers in the thread won't be yielded to just because one fiber blocks
• There are 2 main styles of Fiber use supported:
• Subroutine style - fibers are used like a subroutine that executes successive portions of its code each time it is invoked. A sub-fiber is invoked (either by being created or being yielded to) with arguments. The sub-fiber resumes execution, does some work (possibly including invoking other sub-fibers) and at some point returns a value, yielding to its calling fiber. Note that reentrant invocations of fibers are not supported.
• Coroutine style - fibers are used for application-controlled cooperative multi-threading. Fibers yield to each other, with arguments, and the invoked fiber resumes execution from where it previously left off. There is no hierarchy of calling and called fibers, there are no restrictions on which fibers can yield to which within the same thread, and there is no need for a "return" instruction to yield to a calling fiber. This mode of fiber use is only enabled if the "fiber" extension library is loaded (i.e. run require 'fiber')

In the underlying implementation, Ruby fibers typically are represented by OS fiber facilities, however if OS facilities are not available, Ruby will implement fibers using a system of setjmp/longjmps and stack saves and restores.

Fiber lifecycle

Similar to threads, a fiber has a program which it begins executing when the fiber is created, and when the program is completed the fiber exits. A thread always has at least 1 fiber, which is the root fiber, and its program is the thread's program. Other fibers are created by application code with a designated block or Proc as their program.

For non-root fibers, their block or Proc runs in a slightly altered execution context, similar to how non-main threads are run. For the purposes of the new fiber it is allocated a new special variables region in its local variable context, but the closure on the other local variables is preserved.

When a fiber other than the root fiber completes normally, it will yield to either its calling fiber (if it exists) or the root fiber, passing 1 argument which is the value of the fiber's program's statement block. If the fiber completes with an exception, it will yield the same way but immediately raise the exception on the yielded-to fiber.

When the root fiber completes (either normally or by throwing an exception), the thread completes.

Fiber-local storage

Ruby provides a thread-local storage facility which is in fact fiber-local storage.

Continuations

Continuations provide functionality similar to an exceptionally powerful goto statement. A continuation is like the label in the goto statement idiom - it is an object containing a saved snapshot of the execution state of a fiber. It can be used as follows:

• The continuation can be restored at any point during the execution of its fiber - the current execution state of the fiber will be discarded, and execution will resume from the point after the continuation was saved
• When a continuation is created, some code can be specified that will be run after the continuation is created, but not after the continuation is restored (by passing a block to the continuation creation function)

This is similar to the setjmp/longjmp facility on many OSes, except that this facility is restricted to restoring to a point that is "up the stack", whereas continuations can be used to restore to any point, including points with a totally different stack. Ruby implements continuations with a combination of setjmp/longjmp calls and stack saves/restores.

Continuation closure

Continuations form a closure on every local and dynamic variable in the fiber at the time that the continuation was created. That means that all code in the fiber will see the same local/dynamic variables before as after a continuation restore, variable values are preserved across continuation restores, and therefore updates to any local/dynamic variables in the fiber are visible across continuation restores.

Mutexes

Ruby implements a simple Mutex class, which operates in the standard mutex fashion. Ruby Mutexes are:

• Non-recursive - a thread can't re-lock a mutex it is already locking
• Robust - when a thread terminates, it releases the locks it is holding