Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for Ruby 3.0 Fiber Scheduler #77

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

machty
Copy link

@machty machty commented Jan 13, 2022

This implements support for the Ruby 3.0+ Fiber Scheduler interface.

If a Fiber Scheduler (such as the one defined by the Async library)
is present, then we invoke its hooks whenever we're about to perform
an IO wait operation. This makes it possible for the scheduler to
pass control to another Fiber/task while waiting for the socket to
become readable/writable.

Closes #76

Example

Ruby 3.1 + Async 2.0.0:

# using two separate Redis clients REDIS1 and REDIS2 for simple test;
# typically we'd use a pool

Async do |t|
  t.async do
    puts "1: starting blpop loop\n"
    loop do
      value = REDIS1.blpop "test_queue"
      puts "received value #{value}\n"
    end
  end

  t.async do
    puts "2: starting enqueuer loop\n"
    i = 0
    loop do
      sleep 1
      REDIS2.rpush "test_queue", i
      i += 1
    end
  end
end

This outputs:

1: starting blpop loop
2: starting enqueuer loop
received value ["test_queue", "0"]
received value ["test_queue", "1"]
received value ["test_queue", "2"]
received value ["test_queue", "3"]
received value ["test_queue", "4"]
received value ["test_queue", "5"]
...

Without this PR, the output is just the following (it just hangs):

# 1: starting blpop loop

TODO:

  • Add tests

This implements support for the Ruby 3.0+ Fiber Scheduler interface.

If a Fiber Scheduler (such as the one defined by the Async library)
is present, then we invoke its hooks whenever we're about to perform
an IO wait operation. This makes it possible for the scheduler to
pass control to another Fiber/task while waiting for the socket to
become readable/writable.
#include "ruby/fiber/scheduler.h"
#include "ruby/io.h"

// TODO: I copied these from Ruby; are they supposed to be exposed as part of the C extension API?
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can we get rid of this boilerplate? Seems like a lot of C extensions implementing support for Fiber Scheduler are going to want these utility fns.

VALUE result = rb_fiber_scheduler_io_wait(scheduler,
io_from_fd(fd),
RB_UINT2NUM(RUBY_IO_READABLE),
rb_fiber_scheduler_make_timeout((struct timeval *) timeout));
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I be using io_from_fd here? Is it going to allocate a Ruby IO object every time it gets to this fn? What's the alternative?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The simplest most compatible option is to look up IO.for_fd method and call it dynamically. We can introduce a C interface for this, but it won't be supported on older rubies, so you'll need to have that as a fallback.

@machty
Copy link
Author

machty commented Jan 13, 2022

cc @ioquatix

This is the first time I've done work on a C extension, not to mention working with the native Fiber scheduler API, so I welcome any feedback. I've already added a few comments; curious as to your thoughts on whether some of the io_from_fd copypasta can be removed.

@ioquatix
Copy link

I see so we are missing an interface to wrap a native FD into a Ruby IO at the extension level?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

This is probably not compatible with Ruby 3.0's Fiber Scheduler?
2 participants