Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for async-websocket. #219

Merged
merged 10 commits into from Sep 7, 2018
Merged

Added support for async-websocket. #219

merged 10 commits into from Sep 7, 2018

Conversation

@dblock
Copy link
Collaborator

@dblock dblock commented Aug 25, 2018

Closes #210.

Seems to work for the example, but integration tests not so much. They are green because we don't run integration tests with a slack token to avoid exposing it in PRs. So WIP.

@dblock dblock force-pushed the async branch 2 times, most recently from 63e15bd to b06244d Aug 25, 2018
@dblock dblock changed the title Added support for socketry/async. Added support for async-websocket. Aug 25, 2018
end
end

class Socket < Slack::RealTime::Socket
attr_reader :client

def start_async(client)
@client = client
client.run_loop
Thread.new do
Copy link
Collaborator Author

@dblock dblock Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally want the caller to be responsible for calling this, otherwise we're spawing a thread per client (or a bot)? Or this this necessary even if you run the code within a Async::Reactor.run?

Copy link
Contributor

@ioquatix ioquatix Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did it to make it the same as how Celluloid is working, which spawns one thread per actor AFAIK. However, it's not necessary. We can push that requirement further up the call chain, but it might mean that the top level needs to embed the code in a reactor. Let me try to figure out the race conditions first then we can figure out how to push that code further up.

@ioquatix
Copy link
Contributor

@ioquatix ioquatix commented Aug 27, 2018

@dblock What's your preference for how it behaves?

If you want to make a server with multiple connections/clients, can we make something to wrap around that? so that we can have a top level reactor, or enforce event machine reactor creation?

Threading makes everything tricky and on Ruby you will have worse performance except in very specific situations. So if we can avoid it, it will make code simpler.

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 27, 2018

@ioquatix This is generally used in a stack of libraries, for example

https://github.com/slack-ruby/slack-shellbot
uses https://github.com/slack-ruby/slack-ruby-bot-server
uses https://github.com/slack-ruby/slack-ruby-bot
uses slack-ruby-client

So we tell users to put reactor type stuff as high as possible. For example slack-ruby-bot-server is the first opinionated library that currently defaults to Celluloid and can/should default to async in the next big release.

So for this library we should be removing any ::Async::Reactor.run except for tests. But maybe that doesn't hurt at all? It looks reentrant, we can also keep it?

I removed one in connect! that didn't seem necessary. But it looks like I need one per thread so I have to keep the one in start_async.

I also fixed the example and I think this PR if green is ready to go. I'll wait for the dust to settle and to hear from you ;)

ioquatix
Copy link
Contributor

ioquatix commented on d3140e9 Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haha, good idea.

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 27, 2018

Thanks so much for your help @ioquatix. If you want to play with a real bot, slack-shellbot is a fun one. You'll have to replace references to Celluloid in https://github.com/slack-ruby/slack-ruby-bot-server.

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 27, 2018

Async branch for slack-ruby-bot in slack-ruby/slack-ruby-bot#198.

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 27, 2018

Async branch for slack-ruby-bot-server in slack-ruby/slack-ruby-bot-server#75

@ioquatix How does one do timers in async correctly? What's the equivalent of this:

class Foobar
  include Celluloid

  def whatever
    every 10 do
     # timers
    end
  end
end

@ioquatix
Copy link
Contributor

@ioquatix ioquatix commented Aug 27, 2018

Async actually uses the same timers and nio4r gems as celluloid, so some parts are the same. That being said, you are better using an explicit loop.

require 'async/reactor'

Async::Reactor.run do |task|
	while true
		task.sleep(2)
		puts "Hello World"
	end
end

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 27, 2018

slack-ruby/slack-shellbot#11, and https://shell.playplay.io/ is running with this, lets see how it behaves over the next few days

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 28, 2018

@ioquatix

  • What's the equivalent of this with async? How do I force-close a connection from the outside?

  • How do I terminate a task from within? The equivalent of Celluloid terminate?

That ping thread is still noticing disconnects, so I want to put the ping thread back to working and then we can debug what's going on.

@ioquatix
Copy link
Contributor

@ioquatix ioquatix commented Aug 28, 2018

  • What's the equivalent of this with async? How do I force-close a connection from the outside?

The following is a thread-safe way to stop a reactor:

thread = Thread.new do
  @reactor = Async::Reactor.new

  @reactor.run(&block)
end

@reactor.stop
thread.join

It's safe to call from anywhere.

With respect to your specific question, you probably don't want to kill the reactor unless you can guarantee you created it at the very top level (otherwise you will end up potentially stopping other unrelated tasks. In this specific case, you are fine to do that since you are creating it at the top of a new thread.

If you weren't doing that, you need to capture the task, and call Task#stop, which will terminate that task and all sub-tasks, e.g.

def start_async(client, task: ::Async::Task.current)
	task.async do
		client.run_loop
	end
end

# Elsewhere:
task = start_async(client)

# To stop:
task.stop

@ioquatix
Copy link
Contributor

@ioquatix ioquatix commented Aug 28, 2018

In theory, this also works:

def start_async(client)
	::Async::Reactor.run do
		client.run_loop
	end
end

# Elsewhere:
task = start_async(client)

# To stop:
task.stop # This might actually be the reactor if none was created, because the operation blocked, calling stop is a no-op, but it's still valid to do it.

@ioquatix
Copy link
Contributor

@ioquatix ioquatix commented Aug 28, 2018

How does the life-cycle of Slack::RealTime::Socket work?

Firstly, it's a bit confusing since it's not a socket, is it?

Then, it has several methods:

      def connect!
        # I've modified this a bit in my local code...
        unless connected?
          connect
          logger.debug("#{self.class}##{__method__}") { driver.class }
        end
        
        yield driver if block_given?
      end

      def disconnect!
        driver.close
      end

      def connected?
        !driver.nil?
      end

      def start_sync(client)
        thread = start_async(client)
        thread.join if thread
      rescue Interrupt
        thread.exit if thread
      end

      def start_async(_client)
        raise NotImplementedError, "Expected #{self.class} to implement #{__method__}."
      end

      def close
        @driver = nil
      end

I can sort of see why we have connect and connect! although the distinction is a bit confusing, and calling connect multiple times seems like a bug. I don't understand why we have close and disconnect!.

Where exactly should we call @reactor.stop?

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 28, 2018

You can see slack-ruby/slack-ruby-bot-server#75 for the changes I made to the ping thread, seems to work OK.

@dblock
Copy link
Collaborator Author

@dblock dblock commented Aug 28, 2018

You're right that Slack::RealTime::Socket is not a socket, it's a connection. It's probably an artifact of the past because it represents a websocket.

It can exist in two ways, synchronously and asynchronously, so there's a bit of confusion between the two "modes" I think. You either start_sync or start_async and you can connect or disconnect it at any time for whatever reason (possibly not even used), then finally cleanup by calling close. Or at least that's the intent. We can probably get rid of close.

I am not sure why or where we want to stop the reactor explicitly, do we?

@dblock dblock mentioned this pull request Aug 28, 2018
@dblock dblock force-pushed the async branch 5 times, most recently from e6f4509 to 6df313c Sep 5, 2018
@dblock dblock force-pushed the async branch 4 times, most recently from 0250cd0 to 694b574 Sep 7, 2018
@dblock
Copy link
Collaborator Author

@dblock dblock commented Sep 7, 2018

I'm merging this as is.

@dblock dblock merged commit adf9672 into slack-ruby:master Sep 7, 2018
1 check passed
@dblock dblock deleted the async branch Sep 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants