Which scheduler should be used to enable fibers using virtual threads to run preemptively concurrently?

[mattwelke]

I'm trying to get fibers to run concurrently when using JRuby 9.3.8.0 with Java 19 with preview features enabled. I should make it clear that I'm not trying to solve any issue in a production app, I'm just playing around with things, so this isn't something I need urgent help with. It's just for fun. If you have important work to attend to, feel free to ignore me. :P

I was playing around with fibers recently. I ended up posting this Stack Overflow question to get a better understanding of what a scheduler was doing with CRuby 3.0+ code: https://stackoverflow.com/questions/73723382/why-do-ruby-fibers-that-run-sequentially-without-a-scheduler-set-run-concurrentl

As noted in the SO question, I ended up with a Gemfile that looked like this:

source 'https://rubygems.org'

gem 'libev_scheduler', '~> 0.2'

And a Ruby program that looked like this:

require 'libev_scheduler'

set_sched = ARGV[0] == "--set-sched"
if set_sched then
  Fiber.set_scheduler Libev::Scheduler.new
end

N_FIBERS = 5
fibers = []

N_FIBERS.times do |i|
  n = i + 1

  fiber = Fiber.new do
    puts "Beginning calculation ##{n}..."
    sleep 1
  end

  fibers.push({fiber: fiber, n: n})
end

fibers.each do |fiber|
  fiber[:fiber].resume
end

puts "Finished all calculations!"

If I ran that code with CRuby 3.1.2, with the command time bundle exec ruby main.rb --set-sched, it ran the fibers concurrently:

Beginning calculation #1...
Beginning calculation #2...
Beginning calculation #3...
Beginning calculation #4...
Beginning calculation #5...
Finished all calculations!

real    0m1.173s
user    0m0.150s
sys     0m0.021s

At that point, I had determined how to run fibers concurrently (when they involve IO), which is helpful for real world applications, and with the help of the SO answer, I understood better what was going on under the hood to make the fibers run concurrently. I concluded that I was able to achieve a preemptive concurrency programming model with CRuby 3.0+ using that particular scheduler, libev_scheduler.

Then, I tried to get this to work in JRuby too. First I did a small test to confirm that the virtual threads were usable from JRuby as the release notes for 9.3.8.0 said. It looked good to me:

# vthreads.rb
puts java.lang.System.getProperty("java.version")
puts java.lang.Thread.ofVirtual()

> jruby -J--enable-preview vthreads.rb
19
java.lang.ThreadBuilders$VirtualThreadBuilder@2b43f314

Then, I tried putting these concepts together. I commented out the code in my program that referred to libev_scheduler because I wasn't sure yet if it was compatible with JRuby. Also, I wanted to test things out one at a time and first just see what would happen if I used fibers in JRuby with virtual threads without a scheduler first. So the code looked like this:

# require 'libev_scheduler'

# set_sched = ARGV[0] == "--set-sched"
# if set_sched then
#   Fiber.set_scheduler Libev::Scheduler.new
# end

N_FIBERS = 5
fibers = []

N_FIBERS.times do |i|
  n = i + 1

  fiber = Fiber.new do
    puts "Beginning calculation ##{n}..."
    sleep 1
  end

  fibers.push({fiber: fiber, n: n})
end

fibers.each do |fiber|
  fiber[:fiber].resume
end
puts "Finished all calculations!"

And when I ran it, I noticed that the fibers did not run concurrently:

> time jruby -J--enable-preview main.rb 
Beginning calculation #1...
Beginning calculation #2...
Beginning calculation #3...
Beginning calculation #4...
Beginning calculation #5...
Finished all calculations!

real    0m6.391s
user    0m4.796s
sys     0m0.790s

Because I knew that adding a scheduler to CRuby enabled fibers to run concurrently without me having to write yield points in the code, helping me achieve that simple, preemptive concurrent programming model I like, I tried adding a scheduler to the code when running it with JRuby to see if it would help the same way it helped CRuby. I tried using libev_scheduler again. But when I ran bundle install with that Gemfile while having my system set to JRuby, I got this error:

Bundler found conflicting requirements for the Ruby version:
  In Gemfile:
    libev_scheduler (~> 0.2) was resolved to 0.2, which depends on
      Ruby (>= 3.0)

  Current Ruby version:
    Ruby (= 2.6.8)

I understand this error. It's saying that this gem isn't compatible with JRuby. I tried googling for a bit for a scheduler compatible with JRuby, but I didn't find anything.

So at the end of this experiment, there are a few questions I have:

1. Which scheduler should be used to enable fibers using virtual threads to run concurrently preemptively? (could be a few choices I imagine, if this question makes technical sense)
2. If 1 doesn't make technical sense, because schedulers aren't needed in JRuby, what is it about JRuby that allows it to handle its own fiber scheduling?
3. Because 9.3.8.0 is clearly meant to be a quick experiment to get virtual threads into the hands of JRuby users via fibers, is the idea that this would be iterated on in the future to enable fibers to run preemptively concurrently (and not just cooperatively concurrently)? If so, any blockers we expect to do that?
4. Am I even correct in the first place that with CRuby 3.0+ and a scheduler, my code was preemptively concurrent? Or, was it just that the scheduler being added meant there was a hook where my code could yield, so it was still cooperatively concurrent, and it was just "more concurrent", so to speak, without any more effort from me?

[headius]

Which scheduler should be used to enable fibers using virtual threads

I'm not sure about the answer to this question, but JRuby does not yet implement the scheduler interface in either JRuby 9.3 or 9.4. You would want to be targeting 9.4, though, since that is the 3.0/3.1-compatible release. Nightly snapshots are available and hopefully we will have a release out soon (ideally with the scheduler interface finished, but maybe not).

what is it about JRuby that allows it to handle its own fiber scheduling

Well, currently we are just falling back on the default JDK vthread scheduler by creating simple Thread.ofVirtual instances. It appears to schedule them correctly, but of course we are artificially limiting their concurrency because that is traditionally how fibers work in CRuby.

is the idea that this would be iterated on in the future to enable fibers to run preemptively concurrently

If Ruby fibers ever actually run in parallel, or migrate across native threads, of course we would support this. Currently that is not the case...

Am I even correct in the first place that with CRuby 3.0+ and a scheduler, my code was preemptively concurrent?

You figured it out here. The fibers you created are running "concurrently", but not really running in parallel or "preemptively". The scheduler just sees that a fiber goes into a sleep, and so while it is sleeping it is free to schedule a different fiber. That one also sleeps, so it schedules another one. You end up with all fibers sleeping in parallel, but once they return to Ruby they will go back to running one at a time.

This is a good trick, but it is essentially just expanding the number of places where the CRuby global lock can be released to allow another thread or fiber to execute. It makes all blocking calls potential scheduling points so while the thread or fiber is blocking another one can get work done.

If you modify your script to do non-blocking work, like calculating a large fib() value, you'll see they do not run in parallel and the total time is a multiple of the number of fibers.

We will implement scheduling to match CRuby, but even in JRuby we must still obey the limitations on fibers running in parallel.

You may want to experiment with the vthread APIs from Java directly, since there are thread-pooled executors and vthreads can migrate... you would be able to get more parallel execution of vthreads than we can allow with Fiber today.