Action Cable server adapterization #50979

palkan · 2024-02-06T02:13:01Z

Motivation / Background

This PR aims to make Action Cable more flexible and extensible as a library/framework by separating low-level, implementation specific details (e.g., handling WebSockets, dealing with concurrency) from abstractions (Connection, Channel classes). That would make it possible to use user-level Action Cable code with alternative web servers or execution models (e.g., Fibers) without rewriting it or monkey-patching internals.

Tip

You can find a more detailed description of the problem, motivation and potential benefits of such refactoring in this document.

Note

This PR does not introduce any significant new features or public API breaking changes. It doesn't affect the existing functionality of built-in Action Cable server.

Important

This functionality from this PR is already available for Rails 7+ application through the actioncable-next gem. Please, give it a try and share your feedback here, this would help to get this PR merged into Rails.

Detail

The changes are grouped into commits to reflect the following major changes.

Reorganizing threaded executors.

Action Cable relies on two threaded executors right now: one to perform user-space code (Worker) and another one to perform internal async operations around pub/sub. The latter was for some (historical?) reason hidden within the StreamEventLoop class that implements and IO loop to perform read/write operations on sockets. I've extracted the executor and promoted it into a server attribute independent of IO event loop. The main motivation of this extraction is not only in enforcing SoC but to separate WebSocket-specific code from more generic async executor used throughout the codebase.

I've also wrapped the Concurrent Ruby threaded executor into a custom class with a minimal interface (#post, #timer and #shutdown methods), so it will be possible to configure a custom executor for a server in the future (one use-case we have in mind is using a Fiber-based executor instead of a Thread-based one).

Note that this PR doesn't include any configuration-related code (e.g., to support smth like config.action_cable.async_executor = :fiber) but I'm considering adding something like that in the follow-up PRs.

Limiting the scope of async operations.

This step is the follow-up to the async executor extraction. We have a few places in the Channel internals which directly refer to the event_loop but they're all related to pub/sub operations. So, to avoid unnecessary async operations at the abstraction level, I moved all it to the pub/sub layer by and standardized it across adapters by using a SubscriberMap::Async subclass.

Previously, we might have double async calls when performing #stream_from (one from the channel and one from the pub/sub adapter); now it's up to the adapter whether to create subscriptions asynchronously or not. We shouldn't duplicate this logic.

Splitting ActionCable::Connection::Base into ActionCable::Connection::Base and ActionCable::Server::Connection

Today, the Connection class knows about everything from performing user code to setting up WebSocket event listeners to dealing with Rails executor wrapping. That prevents from reusing this class in environments when we don't need to deal with all of these (non-WebSocket servers or unit tests, more about them below).

This change is mostly code reorganization and minor tests updates (e.g., we no longer need to always use run_in_eventmachine, because now we deal with pure Ruby classes).

So, now we have two types of connections: low-level Server::Connection and application-level Connection::Base. The communicate with each other via the following interfaces:

classDiagram
  class _PubSub {
    subscribe: (channel, handler, callback) -> void
    unsubscribe: (channel, handler) -> void
  }
  
  class _Executor {
    post: (&block) -> void
    timer: (interval, &block) -> handle
  }
  
  class _Server {
    pubsub: _PubSub
    executor: _Executor
    config: Configuration 
  }
  
  class _Socket {
    logger: Logger
    env: Hash[String, untyped]
    request: Request
    protocol: String
    initialize: (_Server, env) -> void
    transmit: (untyped) -> void
    close: () -> void
    perform_work: (untyped receiver, method_name Symbol, *untyped args) -> void
  }
  
  class _Connection {
    initialize: (_Server, _Socket) -> void
    handle_open: () -> void
    handle_incoming: (payload) -> void
    handle_close: () -> void
    beat: () -> void
    statistics: () -> Hash[Symbol, untyped]
  }
  
  _Server --> _PubSub
  _Server --> _Executor
  _Connection --> _Socket
  _Connection --> _Server

Examples / follow-ups

Important

This repository contains examples, tests, and benchmarks for this PR: https://github.com/anycable/action-cable-next-playground

Action Cable testing

~~Here is an example commit (one of the future PRs) demonstrating how this refactoring simplifies Action Cable testing and make it more robust (no patches, real connection and channel classes, etc.)~~

Merged into this PR: commit

The corresponding rspec-rails upgrade is tracked here (minor changes): palkan/rspec-rails#1

Action Cable over SSE

Here is a sketch of how SSE transport for Action Cable can be implemented (based on the changes from this PR): https://gist.github.com/palkan/270a192e79b05d5601fe497ad3ec83b5

Iodine server implementation

Iodine comes with native (meaning implemented within the C extension) WebSockets and pub/sub support. We can now leverage these features for Action Cable in an adapteresque way: https://github.com/anycable/action-cable-next-playground/blob/master/lib/servers/iodine.rb

AnyCable integration

An upgraded (and less hacky) AnyCable Rails integration is in development here: anycable/anycable-rails#205

Other changes

Improved error reporting for subscriptions(AlreadySubscribed, ChannelNotFound, etc.)

Previously, we ignored some of this malfunction examples or simply wrote error log messages. With the new design, Subscriptions shouldn't make assumptions of how to deal with this violations, they must raise exceptions. Rescue handlers have been moved up the stack for the same reasons.

Additional information

Here is a Discord thread where we discuss the project.

There is an old PR attempting to achieve the same goal: #27648.

Checklist

Before submitting the PR make sure the following are checked:

This Pull Request is related to one change. Changes that are unrelated should be opened in separate PRs.
Commit message has a detailed description of what changed and why. If this PR fixes a related issue include it in the commit message. Ex: [Fix #issue-number]
Tests are added or updated if you fix a bug or add a feature.
CHANGELOG files are updated for the changed libraries if there is a behavior change or additional feature. Minor bug fixes and documentation changes should not be included.

/cc @ioquatix @matthewd @rafaelfranca

ioquatix · 2024-02-06T02:13:49Z

I am so going to make time today to review this!!!!

actioncable/lib/action_cable/connection/base.rb

ioquatix · 2024-02-06T11:18:50Z

actioncable/lib/action_cable/connection/base.rb

-          respond_to_invalid_request
-        end
+      def handle_open
+        connect if respond_to?(:connect)


What about just having an empty connect method rather than using if respond_to?

Yeah, that's a good question. I ported it from the current version; I will try to dig why it has been implemented this way in the first place.

ioquatix · 2024-02-06T11:28:28Z

actioncable/lib/action_cable/server/configuration.rb


        @connection_class = -> { ActionCable::Connection::Base }
        @worker_pool_size = 4
+        @executor_pool_size = 10


This configuration seems specific to threaded execution, does it make sense in general or should we consider server specific configuration?

actioncable/lib/action_cable/server/connection.rb

ioquatix · 2024-02-06T11:35:59Z

actioncable/lib/action_cable/server/base.rb


+      # Executor is used by various actions within Action Cable (e.g., pub/sub operations) to run code asynchronously.
+      def executor
+        @executor || @mutex.synchronize { @executor ||= ThreadedExecutor.new(max_size: config.executor_pool_size) }


Maybe this - and the pool size, should be provided entirely by the user, e.g. a traits class. Certainly for Falcon, ideally this is very different.

The plan is to adapterize this as well (I mentioned in the description, "support smth like config.action_cable.async_executor = :fiber"), so it will be possible to plug in any executor.

But I decided to leave it for follow-up PRs to focus only on refactoring what we already have right now. I'd better introduce features one by one after we finalize the refactoring part.

ioquatix · 2024-02-06T11:37:41Z

actioncable/lib/action_cable/server/connection.rb

+        @message_buffer = Server::Connection::MessageBuffer.new(self)
+
+        @protocol = nil
+        @app_conn = config.connection_class.call.new(server, self)


I wonder if we should cache the result of config.connection_class.call?

I guess, the primary reason why it's implemented like that is code reloading. We can check the app configuration value and decide if we need to cache it or not; however, I'm not sure it worth the effort.

Also, I see a potential extension here—we can turn it into a connection class selector depending on the request information:

config.connection_class = ->(request) { if request.path == "/admin/cable" Admin:Connection else ApplicationCable::Connection end }

So, keeping it dynamic makes sense.

This is useful, I'm working on a NoStr server, I hack a NostrConnection, and inject to ActionCable in this way

ActionCable.module_eval do module_function def nostr_server @nostr_server ||= begin config = ActionCable::Server::Base.config.dup config.connection_class = -> { "NostrCable::Connection".safe_constantize } ActionCable::Server::Base.new config: config end end end

ioquatix · 2024-02-06T11:38:59Z

actioncable/lib/action_cable/server/connection/client_socket.rb

+      # This class is heavily based on faye-websocket-ruby
+      #
+      # Copyright (c) 2010-2015 James Coglan
+      class ClientSocket # :nodoc:


I will want to use an entirely different implementation of WebSocket - is this going to be a problem?

Nope.

The way I see Falcon integration is as follows: (almost) nothing from the action_cable/server folder is gonna be used. This is the code specific to Rails WebSocket server implementation. Other server are free (and encouraged) to use low-level code which works better for them.

We will probably extract some parts from the Rails server code along the way (e.g., access logs, etc.). But today, you can already start building a Falcon-specific version and use only the action_cable/channel and action_cable/connection code from Rails.

ioquatix · 2024-02-06T11:40:14Z

actioncable/lib/action_cable/server/connection/stream.rb

+      # This class is heavily based on faye-websocket-ruby
+      #
+      # Copyright (c) 2010-2015 James Coglan
+      class Stream # :nodoc:


This class also seems extremely specific to the underlying implementation - again, is this something we can replace/bypass?

ioquatix · 2024-02-06T11:42:00Z

actioncable/lib/action_cable/server/stream_event_loop.rb


 module ActionCable
-  module Connection
+  module Server


I wonder if we can move the default server implementation into it's own directory, so that it's clear where the boundary between "the client/server interface" and the actual "rack hijack server" is.

where the boundary between "the client/server interface" and the actual "rack hijack server" is.

(Already answered above but here is a shorter version 😁)

The line is drawn at the file structure level: action_cable/server is the Rails default rack hijack server, everything else is to be shared between different implementations.

ioquatix · 2024-02-06T11:45:53Z

This PR looks like a good start to me.

I would like to understand a little more how I might implement async-actioncable according to the changes presented in this PR.

I would like to propose that we use some kind of traits class to control more of the internal execution model, and ideally, a clear separation of "interface" vs "rack hijack_io implementation". That would help me understand how things fit together (layered).

palkan · 2024-02-07T03:22:27Z

Added one more example of using new architecture—Action Cable over SSE the Rails way (via ActionController::Live): https://gist.github.com/palkan/270a192e79b05d5601fe497ad3ec83b5

We can see some copy-pasting from Server::Connection (server.config.connection_class.call, new_tagged_logger) that makes me think that we should move this logic to the server object itself 🤔

jasl · 2024-02-11T08:00:28Z

I need to build a WebSocket service, and I'm trying to find a modern WebSocket framework in Ruby,
unfortunately, I don't see such an active Gem for this.
After several hours of research, I found ActionCable has potential.
The AC original protocol can easily be overridden.

ioquatix · 2024-02-11T22:15:43Z

@jasl Are you building a backend service or are you looking for full stack?

For the backend, people have been having great success with https://github.com/socketry/async-websocket + falcon.

For full stack, https://github.com/socketry/live provides both a front end and a backend library. I'm going to fix the linked example so you can try it out.

jasl · 2024-02-12T02:44:12Z

@jasl Are you building a backend service or are you looking for full stack?

For the backend, people have been having great success with https://github.com/socketry/async-websocket + falcon.

For full stack, https://github.com/socketry/live provides both a front end and a backend library. I'm going to fix the linked example so you can try it out.

I already tried async-websocket + falcon. IMO it's much better than Faye stack, but it's low-level.
For a real product, I need to handle connection lifecycle, ping-pong, broadcast, etc.

that's why I seek this PR, if this works, async-websocket + falcon can be the backend (the executor)?

jasl · 2024-02-12T16:56:50Z

@ioquatix
I made a PoC https://github.com/jasl/dephy-relay/blob/main/app/channels/nostr_cable/connection.rb

Most of my codes are under https://github.com/jasl/dephy-relay/blob/main/app/channels/nostr_cable

This sample replaces the ActionCable protocol with the NoStr protocol, including subscription and broadcasting

ioquatix · 2024-04-07T04:36:45Z

@palkan How can we move forward with this proposal?

I'm running into more issues, for example, when running Rails on HTTP/2, ActionCable does not work correctly, because the browser tries to connect using HTTP/2 WebSockets :(

    3m    error: Rails: Incoming request: CONNECT "/cable". [ec=0x8cdc] [pid=178916] [2024-04-07 16:34:32 +1200]
               | Failed to upgrade to WebSocket (REQUEST_METHOD: CONNECT, HTTP_CONNECTION: , HTTP_UPGRADE: )

palkan · 2024-04-16T02:17:03Z

Hey @ioquatix! Just in time: I planned to work on the next iteration this week (already started). Stay tuned!

ioquatix · 2024-04-16T03:04:25Z

Awesome, if there is anything I can do to help, please let me know!

palkan · 2024-04-16T04:29:46Z

Did one more round:

Incorporated main changes (mostly Markdown-related + ActionCable: allow to pass websocket.close args #51043)
Improved naming as discussed above (raw_conn -> socket and Server::Connection -> Server::Socket; app_conn -> Connection)
Updated SSE example to reflect the changes and also made it a single-file Rails/Puma app (so it's easy to play with and see the changes in action): https://gist.github.com/palkan/270a192e79b05d5601fe497ad3ec83b5
Merged Action Cable testing changes into this PR: commit

ioquatix · 2024-04-16T04:51:14Z

Wow, awesome work! Thanks. If I can make time, I'll try it out.

ioquatix · 2024-04-22T07:05:37Z

@matthewd any chance you can review this PR? Thanks!

matthewd · 2024-04-22T17:29:17Z

Yep, not sure how much time I'll have in the next week or two, but failing that, I'm sure we can practice some Conference Driven Development. 👍

ioquatix · 2024-04-24T05:22:00Z

@palkan will you be at RubyKaigi?

palkan · 2024-04-26T19:43:04Z

@palkan will you be at RubyKaigi?

Yep. I'm also coming to RailsConf before that, so we I can try to find someone else to pitch this PR 🙂

matthewd · 2024-05-02T19:33:24Z

actioncable/lib/action_cable/server/base.rb

      end
+
+      # Tags are declared in the server but computed in the connection. This allows us per-connection tailored tags.
+      # You can pass request object either directly or via block to lazily evaluate it.


This doesn't seem lazy at the moment?

The block is only called if any of the config.log_tags is a Proc. Otherwise, we don't need to initialize a request object. So, the "lazy" refers to the difference between:

new_tagged_logger(request) # and new_tagged_logger { request }

ioquatix · 2024-05-08T03:07:09Z

Is there any chance we can merge this before 7.2 is released?

…_conn -> connection

…tead of custom patches

Silence Test is missing assertions warnings

It would make providing a custom identifier->class logic and instantiating channel instances without calling #subscribed

palkan · 2024-09-13T04:25:18Z

Some updates:

Implemented a PoC AnyCable Rails integration on top of this new architecture (it works but I'd love to do another round to eliminate most of the patches that are still there): Rails 8 version anycable/anycable-rails#205
- Conformance tests are being executed as a part of the AnyCable Rails test suite—all good for now (but, as I said above, I plan to add more tests). Here is an example build: https://github.com/anycable/anycable-rails/actions/runs/10842919665/job/30089284091?pr=205
Also started tracking (and fixing) RSpec Rails compatibility: Rails 8 Action Cable support palkan/rspec-rails#1
Introduced some changes to the way we treat misbehaviour in Subscriptions; in other words, we no longer keep silence, we raise exceptions! (And let rescue them via the rescue_from API). This commit f0d0f3f.

ioquatix · 2024-09-13T11:23:53Z

Super excited to see renewed effort on this PR, thanks everyone and especially thanks @palkan for all your effort.

Make it visible that Redis is not accessible

palkan · 2024-09-14T05:15:39Z

@matthewd @ioquatix Okay, I think, I've reached the state I consider good and usable enough to let you do the final reviews 😀

Here is the project with examples, tests and benchmarks: https://github.com/anycable/action-cable-next-playground (helped me to find and fix some bugs). Now I feel pretty confident about the refactoring. Also, updated the PR description.

@ioquatix I would like you to take a look at the falcon integration and.. well, rewrite it, probably. I'm pretty sure my async code is far from perfect. I aimed to demonstrate the technique of using Action Cable with other servers. Feel free to propose a PR :)

Regarding the async compatibility of Action Cable I'd like to note is that most production subscription adapters uses the same Thread.new + #listen pattern. And it doesn't play well with Async, since we perform broadcasts (via the async executor) from a separate thread. In other words, it hits the "No async task available!" situation. Not sure if we can generalize them to work with both threads and fibers (I ended up writing a custom async-redis subscription adapter).

ioquatix · 2024-09-28T00:40:24Z

I tried out your implementation and I think it's okay.

I think the only thing missing is a schedule operation on a public interface for the executor, which is used to replace Thread.new{#listen} -> @executor.schedule{...}.

Thinking of this as a queueing problem, extracting out the logic into queue style operations might not be a bad idea, e.g. redis is providing a queue, but any suitable system can provide a queue, e.g. a database, in-memory, shared memory, unix IPC, etc. In other words, extracting the queue logic into (1) I want to publish an event and (2) I want to consume events would be a nice abstraction (I know some of this already exists).

palkan · 2024-09-30T21:25:24Z

@matthewd @ioquatix I made this PR available as a gem, so we can collect more feedback from the apps running on Rails 7+: https://github.com/anycable/actioncable-next. Please, spread the word 🙂

I think the only thing missing is a schedule operation on a public interface for the executor, which is used to replace Thread.new{#listen} -> @executor.schedule{...}.

Thanks! I'll take a look at this a bit later.

ioquatix · 2025-03-12T01:34:19Z

With the above gem, it is now straight forward to implement https://github.com/socketry/async-cable

If anyone is able to test async-cable in a real application, it would help validate this PR.

palkan · 2025-05-09T17:41:17Z

Hey @matthewd,
Hope you're doing well.

Let's figure out what we need to make it a part of 8.1. With the release of async-cable, the primary (but not the only) target of this refactoring, we see more and more people switching to it or at least considering it. I'm pretty sure it's been in production for months (should we ask folks to show here and testify?).

Even though everything works through the actioncable-next gem, I consider it a temporary band-aid and not the final solution to the problem. There should be one Action Cable, not two.

yasoob · 2025-08-12T21:41:42Z

Hello folks! Any plans on getting this in the next release :)

jasl · 2025-08-13T01:31:29Z

Kindly ping, what's progress now?

ioquatix · 2025-08-13T01:42:11Z

The way for us to move forward here, is for people to test out Falcon + Async::Cable and report back that it is working in production (ideally with a little bit of evidence).

gjtorikian · 2025-08-13T04:05:02Z

I don’t know what evidence is required, but I’ve built a small LLM chat app, deployed to Fly.io, with that setup. It works, and it’s fantastic. I became interested in the async stack after reading these docs from the RubyLLM gem.

Happy to run a stress test or whatever is needed. I’ve been lurking as a subscriber on this thread, so I’m not sure how aware the broader Rails/Ruby+AI community is of this work.

ioquatix · 2025-08-13T04:18:43Z

If we can collect a few data points, then I'll try to escalate up.

andreaslillebo · 2025-09-09T07:38:56Z

I'm really grateful for all the effort being put into this. We've been running Falcon + Async::Cable in production now since August 19th. Have not encountered any issues yet.

We are mainly using it for LLM chat streaming. 6.2k assistant messages, about 1.4M turbo-streams.

That's at least one more data point. Please let me know if there is anything else I can do to help.

rails-bot bot added the actioncable label Feb 6, 2024

ioquatix reviewed Feb 6, 2024

View reviewed changes

actioncable/lib/action_cable/connection/base.rb Outdated Show resolved Hide resolved

ioquatix reviewed Feb 6, 2024

View reviewed changes

actioncable/lib/action_cable/server/connection.rb Outdated Show resolved Hide resolved

ioquatix reviewed Feb 6, 2024

View reviewed changes

palkan mentioned this pull request Feb 16, 2024

ActionCable: allow to pass websocket.close args #51043

Merged

4 tasks

palkan force-pushed the refactor/action-cable-server-adapterization branch from 4104394 to e01afaa Compare April 16, 2024 02:55

matthewd reviewed May 2, 2024

View reviewed changes

palkan added 7 commits September 12, 2024 19:22

Rename: Server::Connection -> Server::Socket, raw_conn -> socket, app…

1635b58

…_conn -> connection

Move #new_tagged_logger to Server::Base

e1b99f0

Refactor Action Cable testing to use new adapterized architecture ins…

d35c6e1

…tead of custom patches

Move #allow_request_origin? to Server::Base

f72eb60

chore: test_server -> testserver

6cca2d2

Silence Test is missing assertions warnings

Extract Subscriptions#subscription_from_identifier method

a5022b5

It would make providing a custom identifier->class logic and instantiating channel instances without calling #subscribed

Move TestServer from Channel to Connection

fcb1487

palkan force-pushed the refactor/action-cable-server-adapterization branch from 1e53f93 to f0d0f3f Compare September 13, 2024 02:34

This was referenced Sep 13, 2024

Rails 8 version anycable/anycable-rails#205

Merged

Rails 8 Action Cable support palkan/rspec-rails#1

Open

palkan force-pushed the refactor/action-cable-server-adapterization branch from f3c418e to 486e24e Compare September 13, 2024 04:13

palkan added 4 commits September 13, 2024 12:50

Improve exceptions in Action Cable subscriptions

554e70c

Fix channel testing API to have fewer conflicts with rspec-rails

46224ca

Add error logging to Redis subscription adapter

f3c3b80

Make it visible that Redis is not accessible

Fix periodic timers

258d20c

palkan force-pushed the refactor/action-cable-server-adapterization branch from e36c71a to 258d20c Compare September 13, 2024 19:50

Allow specifying pubsub adapter class explicitly

c75d47d

Action Cable server adapterization #50979

Are you sure you want to change the base?

Action Cable server adapterization #50979

Uh oh!

Conversation

palkan commented Feb 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation / Background

Detail

Examples / follow-ups

Action Cable testing

Action Cable over SSE

Iodine server implementation

AnyCable integration

Other changes

Additional information

Checklist

Uh oh!

ioquatix commented Feb 6, 2024

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ioquatix Feb 6, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ioquatix commented Feb 6, 2024

Uh oh!

palkan commented Feb 7, 2024

Uh oh!

jasl commented Feb 11, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ioquatix commented Feb 11, 2024

Uh oh!

jasl commented Feb 12, 2024

Uh oh!

jasl commented Feb 12, 2024

Uh oh!

ioquatix commented Apr 7, 2024

Uh oh!

palkan commented Apr 16, 2024

Uh oh!

ioquatix commented Apr 16, 2024

Uh oh!

palkan commented Apr 16, 2024

Uh oh!

ioquatix commented Apr 16, 2024

Uh oh!

ioquatix commented Apr 22, 2024

Uh oh!

matthewd commented Apr 22, 2024

palkan commented Feb 6, 2024 •

edited

Loading

ioquatix Feb 6, 2024 •

edited

Loading

jasl commented Feb 11, 2024 •

edited

Loading

ioquatix commented Sep 28, 2024 •

edited

Loading

palkan commented May 9, 2025 •

edited

Loading