ActionCable testing #23211

Open
wants to merge 1 commit into
from

Conversation

Projects
None yet
@palkan
Contributor

palkan commented Jan 23, 2016

I've started to play with ActionCable and realized that we don't have any tools for Cable testing.
Here is a kind of scratch of what (as I think) Cable testing could be.

This test helper only deals with transmissions (just like ActiveJob's one). Example:

test 'message broadcasting block style' do
  assert_transmissions(1, "messages:#{@hello_msg.id}:comments") do
    CommentRelayJob.perform_now(@hello_comment)
  end
end

More examples can be found here.

And there are some questions to discuss:

  • How to unit test channels?
  • Do we need such testing at all?
@rails-bot

This comment has been minimized.

Show comment
Hide comment
@rails-bot

rails-bot Jan 23, 2016

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

Thanks for the pull request, and welcome! The Rails team is excited to review your changes, and you should hear from @eileencodes (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 Jan 23, 2016

Member

Thanks for submitting this @palkan - this looks really great! I think asserting that data is being sent is the main thing that should be provided as a test helper (as you have implemented). Not sure what else we want to allow users to test... cc @dhh

Member

maclover7 commented Jan 23, 2016

Thanks for submitting this @palkan - this looks really great! I think asserting that data is being sent is the main thing that should be provided as a test helper (as you have implemented). Not sure what else we want to allow users to test... cc @dhh

+
+ def channels_data
+ @channels_data ||= {}
+ end

This comment has been minimized.

@eileencodes

eileencodes Jan 23, 2016

Member

Can you remove the space under private and then indent private methods like you did in the other file?

@eileencodes

eileencodes Jan 23, 2016

Member

Can you remove the space under private and then indent private methods like you did in the other file?

This comment has been minimized.

@palkan

palkan Jan 23, 2016

Contributor

Sure

@palkan

palkan Jan 23, 2016

Contributor

Sure

+ assert_includes new_messages, serialized_msg, "No messages sent with #{data} to #{channel}"
+ end
+
+ def cable_adapter

This comment has been minimized.

@maclover7

maclover7 Jan 23, 2016

Member

Can we call this pubsub_adapter like we do in other parts of ActionCable?

@maclover7

maclover7 Jan 23, 2016

Member

Can we call this pubsub_adapter like we do in other parts of ActionCable?

+ end
+
+ def assert_transmissions(number, channel)
+ if block_given?

This comment has been minimized.

@maclover7

maclover7 Jan 23, 2016

Member

This is checking for a block, but it's not passed in as a parameter...

@maclover7

maclover7 Jan 23, 2016

Member

This is checking for a block, but it's not passed in as a parameter...

This comment has been minimized.

@palkan

palkan Jan 23, 2016

Contributor

Why do we need block arg here?

@palkan

palkan Jan 23, 2016

Contributor

Why do we need block arg here?

This comment has been minimized.

@maclover7

maclover7 Jan 23, 2016

Member

Because you are checking for it in this line...? Could be wrong here.

@maclover7

maclover7 Jan 23, 2016

Member

Because you are checking for it in this line...? Could be wrong here.

This comment has been minimized.

@palkan

palkan Jan 23, 2016

Contributor

No, we don't need it here, it's checked implicitly. This is how block_given? works.

@palkan

palkan Jan 23, 2016

Contributor

No, we don't need it here, it's checked implicitly. This is how block_given? works.

+ def assert_transmited_data(data, channel)
+ serialized_msg = ActiveSupport::JSON.encode(data)
+ new_messages = transmissions(channel)
+ if block_given?

This comment has been minimized.

@maclover7

maclover7 Jan 23, 2016

Member

This is checking for a block, but it's not passed in as a parameter...

@maclover7

maclover7 Jan 23, 2016

Member

This is checking for a block, but it's not passed in as a parameter...

+module ActionCable
+ module SubscriptionAdapter
+ # == Test adapter for Action Cable
+ class Test < Base

This comment has been minimized.

@maclover7

maclover7 Jan 23, 2016

Member

Can we add a :nodoc: and also a note saying to never use this in any environment except through tests?

@maclover7

maclover7 Jan 23, 2016

Member

Can we add a :nodoc: and also a note saying to never use this in any environment except through tests?

This comment has been minimized.

@palkan

palkan Jan 23, 2016

Contributor

Yes, of course. I'll add some docs to this class and to the helper after we decide that everything is ok.

@palkan

palkan Jan 23, 2016

Contributor

Yes, of course. I'll add some docs to this class and to the helper after we decide that everything is ok.

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Jan 23, 2016

Contributor

TestHelper tests added.

Contributor

palkan commented Jan 23, 2016

TestHelper tests added.

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 Jan 23, 2016

Member

👍 Thank you for you work on this!

Member

maclover7 commented Jan 23, 2016

👍 Thank you for you work on this!

+ # ActionCable.server.broadcast 'messages', { text: 'hello' }
+ # end
+ # end
+ def assert_transmited_data(data, channel)

This comment has been minimized.

@dhh

dhh Jan 24, 2016

Member

Let's flip these parameters and go with assert_transmission_on channel, data so we don't have to wrap the hash.

@dhh

dhh Jan 24, 2016

Member

Let's flip these parameters and go with assert_transmission_on channel, data so we don't have to wrap the hash.

+ # end
+ def assert_transmited_data(data, channel)
+ serialized_msg = ActiveSupport::JSON.encode(data)
+ new_messages = transmissions(channel)

This comment has been minimized.

@dhh

dhh Jan 24, 2016

Member

Add CR.

@dhh

dhh Jan 24, 2016

Member

Add CR.

+
+ # Restore all sent messages
+ (old_messages + new_messages).each { |m| pubsub_adapter.broadcast(channel, m) }
+ end

This comment has been minimized.

@dhh

dhh Jan 24, 2016

Member

Add CR.

@dhh

dhh Jan 24, 2016

Member

Add CR.

+ ActionCable.server.pubsub
+ end
+
+ delegate :transmissions,

This comment has been minimized.

@dhh

dhh Jan 24, 2016

Member

Single line these.

@dhh

dhh Jan 24, 2016

Member

Single line these.

+ def after_teardown # :nodoc:
+ super
+ ActionCable.server.instance_variable_set(:@pubsub, @old_pubsub_adapter)
+ end

This comment has been minimized.

@dhh

dhh Jan 24, 2016

Member

Why is this setup/teardown needed when we're setting the test adapter in cable.yml?

@dhh

dhh Jan 24, 2016

Member

Why is this setup/teardown needed when we're setting the test adapter in cable.yml?

This comment has been minimized.

@palkan

palkan Jan 25, 2016

Contributor

First of all, to use fresh server for every example. Of course, we can just use kind of ActionCable.server.reset! in before_setup.

Secondly, to make sure that we use test adapter and to allow users to use their preferred adapter in other test cases (e.g. acceptance testing).
Maybe it's better to provide some convenient helpers to switch between adapters and use "test" by default (from cable.yml).

@palkan

palkan Jan 25, 2016

Contributor

First of all, to use fresh server for every example. Of course, we can just use kind of ActionCable.server.reset! in before_setup.

Secondly, to make sure that we use test adapter and to allow users to use their preferred adapter in other test cases (e.g. acceptance testing).
Maybe it's better to provide some convenient helpers to switch between adapters and use "test" by default (from cable.yml).

+ # ActionCable.server.broadcast 'messages', { text: 'how are you?' }
+ # end
+ # end
+ def assert_transmissions(number, channel)

This comment has been minimized.

@dhh

dhh Jan 24, 2016

Member

I like how tranmission is continuing to play with our metaphor, but I actually think it's probably simpler to just stick with broadcasts. Then there's parity between what you call and what you test. The broadcast->transmission part is too subtle imo.

So how about assert_broadcasts_on channel, number

@dhh

dhh Jan 24, 2016

Member

I like how tranmission is continuing to play with our metaphor, but I actually think it's probably simpler to just stick with broadcasts. Then there's parity between what you call and what you test. The broadcast->transmission part is too subtle imo.

So how about assert_broadcasts_on channel, number

This comment has been minimized.

@dhh

dhh Jan 24, 2016

Member

(obviously this would be a rename throughout this, transmission->broadcast everywhere).

@dhh

dhh Jan 24, 2016

Member

(obviously this would be a rename throughout this, transmission->broadcast everywhere).

This comment has been minimized.

@palkan

palkan Jan 25, 2016

Contributor

My first idea was to use word "messages" (because message is what we really send through the cable). But I took "transmissions" from existing tests.

So, the other candidate is "message". "Broadcast" or "message"?

@palkan

palkan Jan 25, 2016

Contributor

My first idea was to use word "messages" (because message is what we really send through the cable). But I took "transmissions" from existing tests.

So, the other candidate is "message". "Broadcast" or "message"?

This comment has been minimized.

@kaspth

kaspth Jan 25, 2016

Member

Broadcasts because that's the method you're calling on the server :)

@kaspth

kaspth Jan 25, 2016

Member

Broadcasts because that's the method you're calling on the server :)

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Feb 16, 2016

Contributor

Rebase with the current master (due to Cable configuration changes).
Anything else?

Contributor

palkan commented Feb 16, 2016

Rebase with the current master (due to Cable configuration changes).
Anything else?

@rafaelfranca

This comment has been minimized.

Show comment
Hide comment
Member

rafaelfranca commented Feb 17, 2016

@rails-bot rails-bot assigned matthewd and unassigned eileencodes Feb 17, 2016

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 Apr 24, 2016

Member

@palkan Would you be willing to rebase this, and resolve the remaining conflicts? Getting good Action Cable testing infrastructure setup for 5.0 (when Action Cable officially launches) is very important :)

Member

maclover7 commented Apr 24, 2016

@palkan Would you be willing to rebase this, and resolve the remaining conflicts? Getting good Action Cable testing infrastructure setup for 5.0 (when Action Cable officially launches) is very important :)

@@ -0,0 +1,124 @@
+module ActionCable
+ # Provides helper methods for testing Action Cable broadcasting
+ module TestHelper

This comment has been minimized.

@maclover7

maclover7 Apr 24, 2016

Member

Can we modify this to create an ActionCable::TestCase, similar to how Action Mailer has ActionMailer::TestCase, Active Job ActiveJob::TestCase, etc.?

@maclover7

maclover7 Apr 24, 2016

Member

Can we modify this to create an ActionCable::TestCase, similar to how Action Mailer has ActionMailer::TestCase, Active Job ActiveJob::TestCase, etc.?

@jeremy jeremy added this to the 5.0.0 milestone Apr 24, 2016

@jeremy jeremy assigned jeremy and unassigned matthewd Apr 24, 2016

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Apr 25, 2016

Contributor

@maclover7 Done!

Contributor

palkan commented Apr 25, 2016

@maclover7 Done!

+ # <tt>ActionCable::TestHelper</tt> it makes a great tool to test your Rails application.
+ #
+ # To use the test adapter set adapter value to +test+ in your +cable.yml+.
+ class Test < Base

This comment has been minimized.

@maclover7

maclover7 Apr 25, 2016

Member

Can you add :nodoc: after the Base?

@maclover7

maclover7 Apr 25, 2016

Member

Can you add :nodoc: after the Base?

This comment has been minimized.

@palkan

palkan Apr 25, 2016

Contributor

Done!)

@palkan

palkan Apr 25, 2016

Contributor

Done!)

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 Apr 25, 2016

Member

One more comment, then LGTM 👍

Member

maclover7 commented Apr 25, 2016

One more comment, then LGTM 👍

@jeremy jeremy modified the milestones: 5.0.1, 5.0.0 Apr 26, 2016

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Apr 27, 2016

Contributor

Failures and conflicts fixed.

Contributor

palkan commented Apr 27, 2016

Failures and conflicts fixed.

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 Apr 27, 2016

Member

Sorry, one last thing -- can you squash your commits? After that, I think we should be ready to ship.

Member

maclover7 commented Apr 27, 2016

Sorry, one last thing -- can you squash your commits? After that, I think we should be ready to ship.

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Apr 27, 2016

Contributor

Sure) Done!

Contributor

palkan commented Apr 27, 2016

Sure) Done!

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 Apr 30, 2016

Member

Hmm, looks like your commits didn't squash properly -- you still have three commits instead of one :)

Member

maclover7 commented Apr 30, 2016

Hmm, looks like your commits didn't squash properly -- you still have three commits instead of one :)

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan May 4, 2016

Contributor

Oh, sorry for that(
Squashed.

Contributor

palkan commented May 4, 2016

Oh, sorry for that(
Squashed.

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 May 4, 2016

Member

Opening and closing to trigger Travis build.

Member

maclover7 commented May 4, 2016

Opening and closing to trigger Travis build.

@maclover7 maclover7 closed this May 4, 2016

@maclover7 maclover7 reopened this May 4, 2016

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan May 6, 2016

Contributor

Finally, It's green)

Contributor

palkan commented May 6, 2016

Finally, It's green)

@maclover7

This comment has been minimized.

Show comment
Hide comment
@maclover7

maclover7 Jul 9, 2016

Member

Hi @h8rry! As denoted by the 5.0.1 milestone, this PR will be merged in, and will ship with Rails (and Action Cable) v5.0.1.

cc release manager @matthewd

Member

maclover7 commented Jul 9, 2016

Hi @h8rry! As denoted by the 5.0.1 milestone, this PR will be merged in, and will ship with Rails (and Action Cable) v5.0.1.

cc release manager @matthewd

@matthewd

This comment has been minimized.

Show comment
Hide comment
@matthewd

matthewd Jul 9, 2016

Member

will ship with Rails (and Action Cable) v5.0.1

I'll leave the milestone for now, but to me, this is highly doubtful.

The change does seem sufficiently isolated (and important) to justify backporting it to the 5.0 series -- which is what the 5.0.1 milestone meant in April: "5.0, but not 5.0.0" -- but I don't think we'll be ready to lock in an API in time for 5.0.1.

Member

matthewd commented Jul 9, 2016

will ship with Rails (and Action Cable) v5.0.1

I'll leave the milestone for now, but to me, this is highly doubtful.

The change does seem sufficiently isolated (and important) to justify backporting it to the 5.0 series -- which is what the 5.0.1 milestone meant in April: "5.0, but not 5.0.0" -- but I don't think we'll be ready to lock in an API in time for 5.0.1.

carpeliam added a commit to carpeliam/pingpongreg that referenced this pull request Jul 17, 2016

Send reservations to redux store upon any reservation modification
- other actions that create/remove reservations are now superfluous, should be removed
- unit test for subscribeToSocketEvents is really more of an integration test, mocking at the websocket level - implementation heavily inspired by https://github.com/rails/rails/tree/master/actioncable/test/javascript
- unsure as to how to test reservations_relay_job or tables_channel, as ActionCable testing hasn't made it into a Rails release yet. See rails/rails#23211

carpeliam added a commit to carpeliam/pingpongreg that referenced this pull request Jul 17, 2016

Send reservations to redux store upon any reservation modification
- other actions that create/remove reservations are now superfluous, should be removed
- unit test for subscribeToSocketEvents is really more of an integration test, mocking at the websocket level - implementation heavily inspired by https://github.com/rails/rails/tree/master/actioncable/test/javascript
- unsure as to how to test reservations_relay_job or tables_channel, as ActionCable testing hasn't made it into a Rails release yet. See rails/rails#23211

[#122429431]
actioncable/test/stubs/test_server.rb
@@ -1,6 +1,7 @@
require 'ostruct'
class TestServer
+ include ActionCable::Server::Broadcasting
include ActionCable::Server::Connections
include ActionCable::Server::Broadcasting

This comment has been minimized.

@aglushkov

aglushkov Jul 19, 2016

ActionCable::Server::Broadcasting already included

@aglushkov

aglushkov Jul 19, 2016

ActionCable::Server::Broadcasting already included

This comment has been minimized.

@palkan

palkan Jul 19, 2016

Contributor

Thanks

@palkan

palkan Jul 19, 2016

Contributor

Thanks

+ # ActionCable.server.broadcast 'messages', { text: 'hello' }
+ # end
+ #
+ # assert_broadcasts 'messages', 2

This comment has been minimized.

@aglushkov

aglushkov Jul 19, 2016

forgotten 'do'

@aglushkov

aglushkov Jul 19, 2016

forgotten 'do'

@Preen

This comment has been minimized.

Show comment
Hide comment
@Preen

Preen Oct 26, 2016

Any news on this? =)

Preen commented Oct 26, 2016

Any news on this? =)

@guilleiguaran

This comment has been minimized.

Show comment
Hide comment
@guilleiguaran

guilleiguaran Nov 22, 2016

Member

Maybe 5.1.0 is a more realistic milestone for this?

Member

guilleiguaran commented Nov 22, 2016

Maybe 5.1.0 is a more realistic milestone for this?

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Nov 22, 2016

Member

@dhh @jeremy @matthewd @eileencodes since Action Cable has been out for some time, I'm curious to hear if there's test practices to extract from Basecamp for Action Cable? Are you still just testing through the app?

Another thing, since proper system testing is being integrated in #26703, does it make sense to have Channel unit tests? Or should we find a way to spin up a WebSocket server (like we do in the Action Cable tests) to run things through the wire?

Member

kaspth commented Nov 22, 2016

@dhh @jeremy @matthewd @eileencodes since Action Cable has been out for some time, I'm curious to hear if there's test practices to extract from Basecamp for Action Cable? Are you still just testing through the app?

Another thing, since proper system testing is being integrated in #26703, does it make sense to have Channel unit tests? Or should we find a way to spin up a WebSocket server (like we do in the Action Cable tests) to run things through the wire?

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Nov 22, 2016

Contributor

@kaspth

... does it make sense to have Channel unit tests?

I still believe that system tests are not enough. How can you test, for example, authentication/authorization? Consider a very simple (though realistic) example:

class AdminChannel < ApplicationCable::Channel
  def subscribed
    reject unless current_user.admin?
  end
end

I can't see how to cover rejection case with only system tests.

I see two options to test channels:

  • unit testing with connection and channel kinda mocks
  • "run things through the wire" using smth like action_cable_client.

The first one looks like controller tests and the second one – like request tests.

Contributor

palkan commented Nov 22, 2016

@kaspth

... does it make sense to have Channel unit tests?

I still believe that system tests are not enough. How can you test, for example, authentication/authorization? Consider a very simple (though realistic) example:

class AdminChannel < ApplicationCable::Channel
  def subscribed
    reject unless current_user.admin?
  end
end

I can't see how to cover rejection case with only system tests.

I see two options to test channels:

  • unit testing with connection and channel kinda mocks
  • "run things through the wire" using smth like action_cable_client.

The first one looks like controller tests and the second one – like request tests.

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Nov 23, 2016

Member

At Basecamp we're testing cable through system tests. But agree it'd be nice to have a way to test the channels directly. Will need to serious stubbing out to make that happen, though. Anyway, happy to see that.

Member

dhh commented Nov 23, 2016

At Basecamp we're testing cable through system tests. But agree it'd be nice to have a way to test the channels directly. Will need to serious stubbing out to make that happen, though. Anyway, happy to see that.

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Nov 23, 2016

Contributor

@dhh
I have some ideas on the implementation/API. Please, take a look at this:

class ChatChannelTest < ActionCable::ChannelTest
  def connection
    # connection stub used by channels 
    @connection ||= connection_stub(user: @user)
  end  

  test "should subscribe" do
    subscribe room_id: 1
    
    # checks that subscription was confirmed
    assert_subscribed

    # checks that was subscribed to the stream
    assert_streamed_from "chat_1"
  end

  test "reject if id is missing" do
    subscribe
    assert_rejected
  end

  test "send message" do
    subscribe room_id: 1
    perform :send, text: 'hi1'

    assert_broadcasted "chat_1", text: 'hi1', uid: @user.id
  end

  test "ping" do
    subscribe room_id: 1
    perform :ping

    assert_received "pong"
  end
end

What do you think?

Contributor

palkan commented Nov 23, 2016

@dhh
I have some ideas on the implementation/API. Please, take a look at this:

class ChatChannelTest < ActionCable::ChannelTest
  def connection
    # connection stub used by channels 
    @connection ||= connection_stub(user: @user)
  end  

  test "should subscribe" do
    subscribe room_id: 1
    
    # checks that subscription was confirmed
    assert_subscribed

    # checks that was subscribed to the stream
    assert_streamed_from "chat_1"
  end

  test "reject if id is missing" do
    subscribe
    assert_rejected
  end

  test "send message" do
    subscribe room_id: 1
    perform :send, text: 'hi1'

    assert_broadcasted "chat_1", text: 'hi1', uid: @user.id
  end

  test "ping" do
    subscribe room_id: 1
    perform :ping

    assert_received "pong"
  end
end

What do you think?

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Nov 23, 2016

Member

I generally don't love a big proliferation of custom asserts. I think we can get there without, like:

class ChatChannelTest < ActionCable::ChannelTest
  delegate :subscribe, :subscriptions, :broadcasts, to: :connection

  def connection
    # connection stub used by channels 
    @connection ||= connection_stub(user: @user)
  end  

  test "should subscribe" do
    subscription = subscribe room_id: 1
    
    # checks that subscription was confirmed
    assert subscription.present?

    # checks that was subscribed to the stream
    assert_equal "subscribed", subscription.received.last
  end

  test "reject if id is missing" do
    assert_no_difference -> { subscriptions.count } do
      subscribe
    end
  end

  test "send message" do
    subscription = subscribe room_id: 1
    subscription.perform :send, text: 'hi1'

    assert_equal { text: 'hi1', uid: @user.id }, broadcasts["chat_1"].last
  end

  test "ping" do
    subscription = subscribe room_id: 1
    subscription.perform :ping

    assert_equal "pong", subscription.received.last
  end
end
Member

dhh commented Nov 23, 2016

I generally don't love a big proliferation of custom asserts. I think we can get there without, like:

class ChatChannelTest < ActionCable::ChannelTest
  delegate :subscribe, :subscriptions, :broadcasts, to: :connection

  def connection
    # connection stub used by channels 
    @connection ||= connection_stub(user: @user)
  end  

  test "should subscribe" do
    subscription = subscribe room_id: 1
    
    # checks that subscription was confirmed
    assert subscription.present?

    # checks that was subscribed to the stream
    assert_equal "subscribed", subscription.received.last
  end

  test "reject if id is missing" do
    assert_no_difference -> { subscriptions.count } do
      subscribe
    end
  end

  test "send message" do
    subscription = subscribe room_id: 1
    subscription.perform :send, text: 'hi1'

    assert_equal { text: 'hi1', uid: @user.id }, broadcasts["chat_1"].last
  end

  test "ping" do
    subscription = subscribe room_id: 1
    subscription.perform :ping

    assert_equal "pong", subscription.received.last
  end
end
@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Nov 24, 2016

Contributor

@dhh @kaspth
ActionCable::Channel::TestCase basic implementation added. Waiting for your feedback!

Contributor

palkan commented Nov 24, 2016

@dhh @kaspth
ActionCable::Channel::TestCase basic implementation added. Waiting for your feedback!

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Nov 24, 2016

Member

I'll let @kaspth look at what's there now. Then I can look again when the API is closer to what proposed above.

Member

dhh commented Nov 24, 2016

I'll let @kaspth look at what's there now. Then I can look again when the API is closer to what proposed above.

actioncable/CHANGELOG.md
+* Provide testing utils (ActionCable::TestCase and ActionCable::Channel::TestCase).
+
+ *Vladimir Dementyev*
+

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

Perhaps we should leave the changelog entry out for now until we have the API settled.

@kaspth

kaspth Nov 27, 2016

Member

Perhaps we should leave the changelog entry out for now until we have the API settled.

+
+module ActionCable
+ module Channel
+ class NonInferrablChannelError < ::StandardError

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

Inferrable

Do we need to fully qualify the StandardError here?

@kaspth

kaspth Nov 27, 2016

Member

Inferrable

Do we need to fully qualify the StandardError here?

+ def add(channel, params)
+ identifier = {
+ channel: channel.name
+ }.merge(params).to_json

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

We're duplicating the knowledge of how to create an identifier in the test case. Let's see if we can't share the logic.

@kaspth

kaspth Nov 27, 2016

Member

We're duplicating the knowledge of how to create an identifier in the test case. Let's see if we can't share the logic.

+
+ # Stub `sream_from` to track streams for the channel
+ module TestStreams
+ def stream_from(broadcasting, callback = nil, coder: nil)

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

def stream_from(broadcasting, *)?

@kaspth

kaspth Nov 27, 2016

Member

def stream_from(broadcasting, *)?

+ end
+ end
+
+ # Stub `sream_from` to track streams for the channel

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

stream_from

@kaspth

kaspth Nov 27, 2016

Member

stream_from

@@ -0,0 +1,207 @@
+require "active_support/test_case"
+require "active_support/core_ext/hash/indifferent_access"

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

We also need to require the top level here with this at the top: require "active_support".

@kaspth

kaspth Nov 27, 2016

Member

We also need to require the top level here with this at the top: require "active_support".

+ end
+ end
+
+ # Superclass for ActionCable channels functional tests. Functional tests allow you to

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

When not referring to the constant, it's Action Cable with a space.

@kaspth

kaspth Nov 27, 2016

Member

When not referring to the constant, it's Action Cable with a space.

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

We've also moved away from the functional term in controller tests. So let's just say channel tests.

@kaspth

kaspth Nov 27, 2016

Member

We've also moved away from the functional term in controller tests. So let's just say channel tests.

+ end
+
+ # Superclass for ActionCable channels functional tests. Functional tests allow you to
+ # test a single channel per method.

This comment has been minimized.

@kaspth

kaspth Nov 27, 2016

Member

test a single channel per method what does that mean?

@kaspth

kaspth Nov 27, 2016

Member

test a single channel per method what does that mean?

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Nov 27, 2016

Member

I tried reviewing this, but ran out of steam. There's too much code here with too broad of a reach for my taste. Instead of solving the whole Action Cable unit testing story in one go (and over these many many months), can we try paring it down?

This PR would then just set up the bare minimum to get unit testing up and running. Base classes and stubbing, but no custom assertions.

Then we can see if the assertions make sense later on. Though I appreciate all the extra work you've put into this!

Member

kaspth commented Nov 27, 2016

I tried reviewing this, but ran out of steam. There's too much code here with too broad of a reach for my taste. Instead of solving the whole Action Cable unit testing story in one go (and over these many many months), can we try paring it down?

This PR would then just set up the bare minimum to get unit testing up and running. Base classes and stubbing, but no custom assertions.

Then we can see if the assertions make sense later on. Though I appreciate all the extra work you've put into this!

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Nov 27, 2016

Contributor

@kaspth
Thanks for the review!

... can we try paring it down?

This PR contains 2 features:

  • transmissions testing (i.e. testing ActionCable.server.broadcast calls) the same way we test jobs queuing;
  • channels unit-testing.

I think, it's possible to extract unit-testing into a separate PR. Would you like me to do that?

Contributor

palkan commented Nov 27, 2016

@kaspth
Thanks for the review!

... can we try paring it down?

This PR contains 2 features:

  • transmissions testing (i.e. testing ActionCable.server.broadcast calls) the same way we test jobs queuing;
  • channels unit-testing.

I think, it's possible to extract unit-testing into a separate PR. Would you like me to do that?

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Nov 27, 2016

Member

Yes, exactly 😄

Member

kaspth commented Nov 27, 2016

Yes, exactly 😄

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Mar 6, 2017

Contributor

Is there any chance to get this PR merged and released as a part of 5.1?

Contributor

palkan commented Mar 6, 2017

Is there any chance to get this PR merged and released as a part of 5.1?

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Mar 6, 2017

Member

@palkan unfortunately no, the betas are our feature freeze point.

Additionally, I'm going to have to bow out of reviewing these PRs I'm too exhausted at the moment to give this the attention it needs. Let's revisit once 5.2 is completely out the door.

Your dedication to this is admirable and appreciated 👏

Member

kaspth commented Mar 6, 2017

@palkan unfortunately no, the betas are our feature freeze point.

Additionally, I'm going to have to bow out of reviewing these PRs I'm too exhausted at the moment to give this the attention it needs. Let's revisit once 5.2 is completely out the door.

Your dedication to this is admirable and appreciated 👏

Add ActionCable testing utils
- Add Action Cable test adapter and test helper
- Add ActionCable::TestCase
@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Jul 6, 2017

Contributor

Hi, @kaspth! Maybe it's time to re-visit and prepare for 5.2?

Contributor

palkan commented Jul 6, 2017

Hi, @kaspth! Maybe it's time to re-visit and prepare for 5.2?

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Jul 6, 2017

Member

Hey! Unfortunately I'm already booked up for 5.2 with a GSoC project and improving things I've directly contributed to 😊

Member

kaspth commented Jul 6, 2017

Hey! Unfortunately I'm already booked up for 5.2 with a GSoC project and improving things I've directly contributed to 😊

@Undo1

This comment has been minimized.

Show comment
Hide comment
@Undo1

Undo1 Aug 26, 2017

This is a highly important issue. Is anyone else able to push this forward? This is a pretty big part of an application to be untestable.

Undo1 commented Aug 26, 2017

This is a highly important issue. Is anyone else able to push this forward? This is a pretty big part of an application to be untestable.

@kesha-antonov

This comment has been minimized.

Show comment
Hide comment
@kesha-antonov

kesha-antonov Aug 26, 2017

Contributor

Agreed. Need this for testing too 👍

Contributor

kesha-antonov commented Aug 26, 2017

Agreed. Need this for testing too 👍

@dhh

This comment has been minimized.

Show comment
Hide comment
@dhh

dhh Aug 26, 2017

Member
Member

dhh commented Aug 26, 2017

@Undo1 Undo1 referenced this pull request in NullVoxPopuli/action_cable_client Oct 3, 2017

Open

Add a way to actually test the action cable connection (with dummy app?) #23

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Oct 24, 2017

Contributor

Extracted into a separate gem – https://github.com/palkan/action-cable-testing.

@dhh @kaspth I'm not closing this one and #27191, 'cause I hope to get them reviewed+merged one day.

Contributor

palkan commented Oct 24, 2017

Extracted into a separate gem – https://github.com/palkan/action-cable-testing.

@dhh @kaspth I'm not closing this one and #27191, 'cause I hope to get them reviewed+merged one day.

@Preen

This comment has been minimized.

Show comment
Hide comment
@Preen

Preen Oct 31, 2017

@palkan Great work. Thanks.

Preen commented Oct 31, 2017

@palkan Great work. Thanks.

@kaspth

This comment has been minimized.

Show comment
Hide comment
@kaspth

kaspth Nov 12, 2017

Member

Sounds good. Let's keep it in the gem for now ✌️

Member

kaspth commented Nov 12, 2017

Sounds good. Let's keep it in the gem for now ✌️

@Schwad

This comment has been minimized.

Show comment
Hide comment
@Schwad

Schwad Nov 16, 2017

@kaspth @dhh @rafaelfranca this PR was whipped together in January of 16 and it has been in a nearly 'ready-to-roll' state it seems since Q2 of that year.

Going back and reading through the work that @palkan has put into this there are nine separate instances where requests were made by members of the team that were then immediately implemented without argument or complaint.

It was only after the better part of two years of this being open that he then put this into a gem.

As someone who is passionate about open source contributions and the community, tell me what I can do to help get this PR on to some sort of timeline. Can we give it a flag or label to make it a priority for the 5.3 milestone queue? 😃

Schwad commented Nov 16, 2017

@kaspth @dhh @rafaelfranca this PR was whipped together in January of 16 and it has been in a nearly 'ready-to-roll' state it seems since Q2 of that year.

Going back and reading through the work that @palkan has put into this there are nine separate instances where requests were made by members of the team that were then immediately implemented without argument or complaint.

It was only after the better part of two years of this being open that he then put this into a gem.

As someone who is passionate about open source contributions and the community, tell me what I can do to help get this PR on to some sort of timeline. Can we give it a flag or label to make it a priority for the 5.3 milestone queue? 😃

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Nov 16, 2017

Contributor

Very good points, @Schwad!

I've already tried to find the answers. With no luck.

I would dare to say that Action Cable has no maintainers.

Maybe, "maintainer" is not the right word, 'cause, hopefully, we still have deps updates, small bug fixes and docs updates–some kind of maintenance. But we don't have any new functionality (even such crucial as testing and a lot more, see PRs and issues).

I had to extract two PRs into a gem. What should I do next time I want to improve Action Cable, propose a PR or write yet another gem? I favor the latter one now.

P.S. OSS that stops evolution gradually dies (Matz).

Contributor

palkan commented Nov 16, 2017

Very good points, @Schwad!

I've already tried to find the answers. With no luck.

I would dare to say that Action Cable has no maintainers.

Maybe, "maintainer" is not the right word, 'cause, hopefully, we still have deps updates, small bug fixes and docs updates–some kind of maintenance. But we don't have any new functionality (even such crucial as testing and a lot more, see PRs and issues).

I had to extract two PRs into a gem. What should I do next time I want to improve Action Cable, propose a PR or write yet another gem? I favor the latter one now.

P.S. OSS that stops evolution gradually dies (Matz).

@matthewd

This comment has been minimized.

Show comment
Hide comment
@matthewd

matthewd Nov 16, 2017

Member

Maybe the gem will have helped with this: have people actually used this in real applications and found it useful? As @dhh has noted, it's solving a problem Basecamp isn't experiencing.

Does anyone have any representative real-world examples of how their tests look when using it?

Anyway, this seems to be a far less intimidating pile of code than it was last time I saw it. I'll try to have a look -- though the gem seemed like a perfectly fine idea to me.

But we don't have any new [Action Cable] functionality

Yep. It's doing its job, and people are spending their limited volunteer time elsewhere. Sorry 🤷🏻‍♂️

Member

matthewd commented Nov 16, 2017

Maybe the gem will have helped with this: have people actually used this in real applications and found it useful? As @dhh has noted, it's solving a problem Basecamp isn't experiencing.

Does anyone have any representative real-world examples of how their tests look when using it?

Anyway, this seems to be a far less intimidating pile of code than it was last time I saw it. I'll try to have a look -- though the gem seemed like a perfectly fine idea to me.

But we don't have any new [Action Cable] functionality

Yep. It's doing its job, and people are spending their limited volunteer time elsewhere. Sorry 🤷🏻‍♂️

@matthewd matthewd assigned matthewd and unassigned jeremy Nov 16, 2017

@Preen

This comment has been minimized.

Show comment
Hide comment
@Preen

Preen Nov 16, 2017

Yep. It's doing its job, and people are spending their limited volunteer time elsewhere. Sorry

Not @palkan. So dont feel sorry. Just feel sorry its not been merged into Rails. :)

Preen commented Nov 16, 2017

Yep. It's doing its job, and people are spending their limited volunteer time elsewhere. Sorry

Not @palkan. So dont feel sorry. Just feel sorry its not been merged into Rails. :)

@Schwad

This comment has been minimized.

Show comment
Hide comment
@Schwad

Schwad Nov 17, 2017

Thank you so much for being willing to have a look @matthewd :) If there's anything I can do to help on this please let me know.

Schwad commented Nov 17, 2017

Thank you so much for being willing to have a look @matthewd :) If there's anything I can do to help on this please let me know.

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Nov 17, 2017

Contributor

First of all, thank you, @matthewd!

Does anyone have any representative real-world examples of how their tests look when using it?

A few examples from the wild: one, and two, and three.

There are several reasons to write unit/functional cable tests:

  • Access control (who has access to the channel? who can perform action and with what arguments? who can connect to the server?)–similar to controller tests
  • Frontend-less applications–no other way to test channels, because no system tests at all

So, system tests (the official testing technique for cable) don't solve all the problems. And also, due to the well-known reasons (slow, hard to maintain, etc.), not everyone writes system tests.

I'll try to have a look -- though the gem seemed like a perfectly fine idea to me.

I agree–having this functionality in a separate gem makes sense. It's much more easier to react on bugs/feature requests and so on. And as long as Action Cable itself is not evolving, there should not be any compatibility issues.

I'd like to propose adding the information about the gem to the guides and, maybe, even include it to the default Gemfile. @matthewd WDYT?

it's solving a problem Basecamp isn't experiencing.

Looks like Basecamp is the only Rails application. Oh, there are also Shopify and Github)

Also, "not experiencing" != "doesn't exist and impossible".

It's doing its job

Unfortunately, IMO it's not doing it well. It could be much better. I've already mentioned some problems here (e.g. #26999), I'm speaking about Action Cable and its problems from time to time (see here, for example) and I have a lot of ideas on how to improve it but... "It's doing its job". I've heard this before.

Nevertheless, I'm still here and willing to help.

Contributor

palkan commented Nov 17, 2017

First of all, thank you, @matthewd!

Does anyone have any representative real-world examples of how their tests look when using it?

A few examples from the wild: one, and two, and three.

There are several reasons to write unit/functional cable tests:

  • Access control (who has access to the channel? who can perform action and with what arguments? who can connect to the server?)–similar to controller tests
  • Frontend-less applications–no other way to test channels, because no system tests at all

So, system tests (the official testing technique for cable) don't solve all the problems. And also, due to the well-known reasons (slow, hard to maintain, etc.), not everyone writes system tests.

I'll try to have a look -- though the gem seemed like a perfectly fine idea to me.

I agree–having this functionality in a separate gem makes sense. It's much more easier to react on bugs/feature requests and so on. And as long as Action Cable itself is not evolving, there should not be any compatibility issues.

I'd like to propose adding the information about the gem to the guides and, maybe, even include it to the default Gemfile. @matthewd WDYT?

it's solving a problem Basecamp isn't experiencing.

Looks like Basecamp is the only Rails application. Oh, there are also Shopify and Github)

Also, "not experiencing" != "doesn't exist and impossible".

It's doing its job

Unfortunately, IMO it's not doing it well. It could be much better. I've already mentioned some problems here (e.g. #26999), I'm speaking about Action Cable and its problems from time to time (see here, for example) and I have a lot of ideas on how to improve it but... "It's doing its job". I've heard this before.

Nevertheless, I'm still here and willing to help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment