Add Socket Mode support to the SDK #879

seratch · 2020-11-23T09:15:52Z

Summary

This pull request is still work in progress. We may change the APIs and/or implementations before releasing this.

This pull request adds all required functionalities for Socket Mode support, such as WSS URL retrieval, bidirectional communications with the Slack server over a WebSocket connection, maintaining WS connections, and relaying only envelope messages to Bolt framework or something similar.

In the initial version, the SDK is going to support the following major libraries for the low-level WebSocket client layer.

websocket-client: https://github.com/websocket-client/websocket-client
websockets: https://websockets.readthedocs.io/
aiohttp: https://docs.aiohttp.org/en/stable/client_quickstart.html#websockets

TODO:

autobahn: https://github.com/crossbario/autobahn-python
tornado: https://www.tornadoweb.org/en/stable/websocket.html
ws4py: https://ws4py.readthedocs.io/en/latest/sources/clienttutorial/
(others): https://github.com/search?l=Python&o=desc&q=websocket&s=stars&type=Repositories

The BaseSocketModeClient and AsyncBaseSocketModeClient provide good abstraction and the interface never depends on any of these specific library's interface. We accept the difference of a bit detailed configuration in the subtypes of the interface. For instance, websockets library does not support proxy settings at this moment. As aiohttp does not have OPEN message events, the implementation for this library does not offer on_open_listeners. These differences never appear in the base class (interface) level.

In this pull request, I have not implemented any decorators for envelope types (events_api, interactive, slash_commands) yet. If we receive developers' need to have such mechanism not only in Bolt for Python but also in slack_sdk.socket_mode module, we may consider adding the feature. But, in general, we highly recommend using Bolt for Python along with this SDK for complex app development.

related change: slackapi/bolt-python#154

Others I should mention are:

The GA release date is not yet publicly announced
Documentation is out of scope in this PR

Category (place an `x` in each of the `[ ]`)

slack_sdk.socket_mode (New one!)

Requirements (place an `x` in each `[ ]`)

I've read and understood the Contributing Guidelines and have done my best effort to follow them.
I've read and agree to the Code of Conduct.
I've run python setup.py validate after making the changes.

codecov · 2020-11-23T09:18:30Z

Codecov Report

Merging #879 (845fbab) into main (1c3878a) will decrease coverage by 11.47%.
The diff coverage is 0.17%.

@@             Coverage Diff             @@
##             main     #879       +/-   ##
===========================================
- Coverage   76.28%   64.80%   -11.48%     
===========================================
  Files          35       44        +9     
  Lines        3226     3799      +573     
===========================================
+ Hits         2461     2462        +1     
- Misses        765     1337      +572

Impacted Files	Coverage Δ
slack_sdk/socket_mode/aiohttp/__init__.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/async_client.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/async_listeners.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/client.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/listeners.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/request.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/response.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/websocket_client/__init__.py	`0.00% <0.00%> (ø)`
slack_sdk/socket_mode/websockets/__init__.py	`0.00% <0.00%> (ø)`
slack_sdk/web/legacy_client.py	`35.68% <33.33%> (-0.02%)`	⬇️
... and 9 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc39395...f6a9cc7. Read the comment docs.

seratch

comments for reviewers @stevengill

seratch · 2020-11-23T09:16:54Z

integration_tests/samples/socket_mode/bolt_adapter/aiohttp.py

+        # TODO: remove this once Bolt allow skipping this
+        if self.app._signing_secret is None:
+            self.app._signing_secret = "just-for-workaround"
+        from slack_sdk.signature import SignatureVerifier
+
+        self.signature_verifier = SignatureVerifier(
+            signing_secret=self.app._signing_secret
+        )


these lines will be removed once slackapi/bolt-python#154 is shipped

seratch · 2020-11-23T09:18:06Z

integration_tests/samples/socket_mode/bolt_adapter/aiohttp.py

+        listener = SocketModeAdapter(self.app).listener
+        self.client.socket_mode_request_listeners.append(listener)
+
+    async def connect_async(self):


note: it's not technically possible to have asyncio.run here (in this class). so, the only thing this module can do for developers is provide async method this way.

seratch · 2020-11-23T09:18:29Z

integration_tests/samples/socket_mode/bolt_adapter/aiohttp.py

+    async def close_async(self):
+        await self.client.close()
+
+    async def start_async(self):


refer to the examples in this PR for usage

seratch · 2020-11-23T09:19:21Z

integration_tests/samples/socket_mode/bolt_adapter/aiohttp_async.py

+from slack_bolt.response import BoltResponse
+
+
+class AsyncSocketModeAdapter:


don't worry, once we merge these changes into bolt-python library, I will commonize most of the code.

seratch · 2020-11-23T09:20:16Z

integration_tests/samples/socket_mode/bolt_adapter/aiohttp.py

+from slack_bolt.response import BoltResponse
+
+
+class SocketModeAdapter:


eventually, this class will be placed under slack_bolt.adapter. that's why I named this as an adapter

seratch · 2020-11-23T09:35:29Z

slack_sdk/socket_mode/aiohttp/__init__.py

+    app_token: str
+    wss_uri: Optional[str]
+    auto_reconnect_enabled: bool
+    web_socket_message_queue: Queue


I intentionally use web_socket_ and socket_mode_ prefix for field / method names to make the code clear but most of them look very long. We may want to remove web_socket_ as much as possible as it's obvious.

seratch · 2020-11-23T09:37:03Z

slack_sdk/socket_mode/async_client.py

+            _: Future[None] = asyncio.ensure_future(
+                self.run_web_socket_message_listeners(message, raw_message)
+            )


not to process messages sequentially, using future.

seratch · 2020-11-23T09:37:32Z

slack_sdk/socket_mode/async_client.py

+                if request is not None:
+                    for listener in self.socket_mode_request_listeners:
+                        try:
+                            await listener(self, request)


the listener for Bolt is one of these listeners. you can add more listeners as necessary

seratch · 2020-11-23T09:38:50Z

slack_sdk/socket_mode/websocket_client/__init__.py

+        concurrency: int = 10,
+        trace_enabled: bool = False,
+        http_proxy_host: Optional[str] = None,
+        http_proxy_port: Optional[int] = None,
+        http_proxy_auth: Optional[Tuple[str, str]] = None,
+        proxy_type: Optional[str] = None,
+        on_open_listeners: Optional[List[Callable[[WebSocketApp], None]]] = None,
+        on_message_listeners: Optional[
+            List[Callable[[WebSocketApp, str], None]]
+        ] = None,
+        on_error_listeners: Optional[
+            List[Callable[[WebSocketApp, Exception], None]]
+        ] = None,
+        on_close_listeners: Optional[List[Callable[[WebSocketApp], None]]] = None,


these are only available for websocket-client.

seratch · 2020-11-23T09:39:35Z

slack_sdk/socket_mode/websocket_client/__init__.py

+                        f"(error: {type(e).__name__}, message: {e})"
+                    )
+
+        self.current_app_executor.submit(run_current_session)


not to block the current thread - WebSocketApp doesn't provide a way to establish WS connection without blocking the current thread.

seratch · 2020-11-24T09:36:20Z

I will be publishing 3.2.0a1 (pre release version) including this change for getting more feedback from Socket Mode beta testers. Let me know if other maintainers have concerns or suggestions.

seratch · 2020-11-25T02:16:11Z

published an alpha version (pre-release): https://pypi.org/project/slack-sdk/3.2.0a1/

seratch · 2020-12-01T13:45:34Z

We will continue working on this task at #883

seratch added enhancement M-T: A feature request for new functionality web-client Version: 3x oauth labels Nov 23, 2020

seratch requested a review from stevengill November 23, 2020 09:15

seratch self-assigned this Nov 23, 2020

seratch commented Nov 23, 2020

View reviewed changes

seratch added this to the 3.2.0 milestone Nov 23, 2020

seratch force-pushed the socket-mode branch from 845fbab to c299005 Compare November 24, 2020 09:21

seratch changed the title ~~WIP: Add Socket Mode support to the SDK~~ Add Socket Mode support to the SDK Nov 28, 2020

seratch force-pushed the socket-mode branch from e44a21c to 2fa97ba Compare November 29, 2020 01:28

seratch added 7 commits November 30, 2020 16:14

Add Socket Mode implementation (aiohttp, websockets, websocket-client)

d7e0865

Remove web_socket_ prefix from message related modules

b414675

version 3.2.0a1

7eb77b9

Remove print debug

f341788

version 3.2.0a2

8599015

Add built-in Socket Mode client, update examples

20fd93f

version 3.2.0a3

4ec70d6

seratch force-pushed the socket-mode branch from 669cf91 to 4ec70d6 Compare November 30, 2020 07:15

Apply more changes to built-in client

f6a9cc7

seratch closed this Dec 1, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Socket Mode support to the SDK #879

Add Socket Mode support to the SDK #879

seratch commented Nov 23, 2020 •

edited

codecov bot commented Nov 23, 2020 •

edited

seratch left a comment

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch Nov 23, 2020

seratch commented Nov 24, 2020

seratch commented Nov 25, 2020

seratch commented Dec 1, 2020

		from slack_bolt.response import BoltResponse


		class AsyncSocketModeAdapter:

		from slack_bolt.response import BoltResponse


		class SocketModeAdapter:

Add Socket Mode support to the SDK #879

Add Socket Mode support to the SDK #879

Conversation

seratch commented Nov 23, 2020 • edited

Summary

Category (place an x in each of the [ ])

Requirements (place an x in each [ ])

codecov bot commented Nov 23, 2020 • edited

Codecov Report

seratch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

seratch commented Nov 24, 2020

seratch commented Nov 25, 2020

seratch commented Dec 1, 2020

seratch commented Nov 23, 2020 •

edited

Category (place an `x` in each of the `[ ]`)

Requirements (place an `x` in each `[ ]`)

codecov bot commented Nov 23, 2020 •

edited