Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to make simultaneous Slack API calls without reinitializing the client? #497

Closed
2 of 6 tasks
hbandlamudi opened this issue Aug 8, 2019 · 12 comments
Closed
2 of 6 tasks
Assignees
Labels
area:concurrency Issues and PRs related to concurrency bug M-T: A confirmed bug report. Issues are confirmed when the reproduction steps are documented question M-T: User needs support to use the project rtm-client Version: 2x web-client
Milestone

Comments

@hbandlamudi
Copy link

hbandlamudi commented Aug 8, 2019

  • bug
  • enhancement (feature request)
  • question
  • documentation related
  • testing related
  • discussion
  • [x ] I've read and understood the Contributing guidelines and have done my best effort to follow them.
  • [x ] I've read and agree to the Code of Conduct.
  • [x ] I've searched for any related issues and avoided creating a duplicate issue.

Reproducible in:

slackclient version: 2.1.0

python version: 3.7

I created a Slack app (using Flask) using version 1.3.1 and it could handle simultaneous Slack API calls without any issue. Since then, I've tried upgrading to slackclient==2.1.0 and see that when simultaneous calls are made, one of them will fail with the a RunTimeError that the This event loop is already running.

In order to further investigate, I created a simple Flask app which can handle simultaneous calls. The code below works without any issue however if I try to move the loop and slack WebClient initialization outside of the entry() method, simultaneous calls fail. Do these variables need to be reinitialized on each request or is there another way to only initialize them once yet be able to handle concurrent requests?

from flask_api import status
import asyncio
import slack

app = Flask(__name__)
bot_slack_token = 'xoxb-xxxxxxxxxx'

response_headers = {
    'Content-Type': 'application/json'
}

async def send_async_message(channel, text, client):
    response = await client.chat_postMessage(channel=channel, text=text)

    assert response["ok"]
    assert response["message"]["text"] == "ok"


@app.route("/")
def entry():
    asyncio.set_event_loop(asyncio.new_event_loop())
    loop = asyncio.get_event_loop()

    client = slack.WebClient(
        token=bot_slack_token,
        run_async=True
    )

    loop.run_until_complete(send_async_message(channel='#hello', text="ok", client=client)) 

    return make_response("", status.HTTP_200_OK, response_headers)


if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')



@SeverinAlexB
Copy link

Same problem here

@jeffbuswell
Copy link

jeffbuswell commented Aug 15, 2019

I'm hitting this issue too. I was able to temporarily work around the problem using nest_asyncio, but a long-term solution would be appreciated.

@hbandlamudi
Copy link
Author

@jeffbuswell Could you post an example of how to use nest_asyncio in this situation?

@jeffbuswell
Copy link

jeffbuswell commented Aug 16, 2019

@hbandlamudi Using your example, it would look something like this:

from flask_api import status
import asyncio
import slack
import nest_asyncio

app = Flask(__name__)
bot_slack_token = 'xoxb-xxxxxxxxxx'
nest_asyncio.apply()

response_headers = {
    'Content-Type': 'application/json'
}

def send_message(channel, text, client):
    response = client.chat_postMessage(channel=channel, text=text)

    assert response["ok"]
    assert response["message"]["text"] == "ok"

@app.route("/")
def entry():
    client = slack.WebClient(
        token=bot_slack_token
    )

    send_message(channel='#hello', text="ok", client=client)

    return make_response("", status.HTTP_200_OK, response_headers)


if __name__ == '__main__':
    app.run(debug=True, host='0.0.0.0')

@hbandlamudi
Copy link
Author

@jeffbuswell Thank you for your script. When I tried to use it and initialized client outside of the method, simultaneous calls still fail with RuntimeError: There is no current event loop in thread 'Thread-XX'. so even with nest_asyncio the client would still have to reinitialized at each request.

@RodneyU215 RodneyU215 self-assigned this Aug 20, 2019
@RodneyU215 RodneyU215 added Priority: High question M-T: User needs support to use the project Version: 2x labels Aug 20, 2019
@RodneyU215
Copy link
Contributor

Hi @hbandlamudi, thanks for opening this issue! If you're looking to make asynchronous requests I wouldn't currently recommend using Flask. Instead I'd use something more async friendly. If you like Flask's API I would use Quart. It uses the exact same API as Flask, but is built for async. Below is a working example of what this would look like:

# example.py
import slack
from quart import Quart

app = Quart(__name__)
bot_slack_token = "xoxb-1234...etc"
client = slack.WebClient(token=bot_slack_token, run_async=True)

async def send_async_message(channel, text, client):
    response = await client.chat_postMessage(channel=channel, text=text)

    assert response["ok"]
    assert response["message"]["text"] == "ok"

@app.route("/")
async def notify():
    await send_async_message(channel="#general", text="ok", client=client)
    return "OK"

if __name__ == "__main__":
    app.run(debug=False)
$ pip install slackclient
$ pip install quart
$ python example.py 

You could also consider aiohttp's web server, but requires slightly more work to get started.

@Marc-Girard
Copy link

Marc-Girard commented Aug 28, 2019

Quart looks like a great alternative to Flask but it looks like it is Python 3.7 only. My production environment is 3.6 unfortunately.

From the web site:

Quart is only compatible with Python 3.7 or higher and can be installed using pipenv or your favorite python package manager.

@justdave
Copy link

I'm running into this as well, but just using the built-in event loop from RTMClient, not an app framework. The nested_asyncio thing fixed it for me as well, but that feels like a hack and not a fix...

@RodneyU215 RodneyU215 added the area:concurrency Issues and PRs related to concurrency label Feb 8, 2020
@nbeuchat
Copy link

nbeuchat commented Apr 9, 2020

Same issue here, I am trying to run the Slack client synchronously with client = slack.WebClient(token=bot_slack_token, run_async=False).

I had to switch back to version 1.3, hopefully, there will be a solution soon

@seratch seratch assigned seratch and unassigned RodneyU215 Apr 10, 2020
@seratch seratch added bug M-T: A confirmed bug report. Issues are confirmed when the reproduction steps are documented rtm-client web-client labels Apr 10, 2020
seratch added a commit to seratch/python-slack-sdk that referenced this issue Apr 22, 2020
@seratch
Copy link
Member

seratch commented Apr 22, 2020

👋 Sorry for the confusion mentioned here. I've created a small test app showing a possible option at this point. https://github.com/slackapi/python-slackclient/blob/master/integration_tests/samples/issues/issue_497.py

As you see, with version 2.0.0 - 2.5.0, using shared WebClient & event_loop. run_until_complete(coro) in Flask (or any similar) applications doesn't work for simultaneous requests. The only workaround is creating a WebClient instance per request as I did in the above test app.

I'm working on addressing this issue (and related others) right now. For the near future, WebClient with run_async=False (the default) will be available for sharing among requests. The fixed version will be the next minor version - 2.6.0. If everything goes well, I'll release the new version by the end of this month.

@seratch seratch added this to the 2.6.0 milestone Apr 22, 2020
seratch added a commit to seratch/python-slack-sdk that referenced this issue Apr 27, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue Apr 28, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue Apr 28, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue Apr 30, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 11, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 11, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 11, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 11, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 11, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 11, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 11, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 13, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
seratch added a commit to seratch/python-slack-sdk that referenced this issue May 14, 2020
* slackapi#530 Fixed by changing _execute_in_thread to be a coroutine
* slackapi#569 Resolved by removing a blocking loop (while future.running())
* slackapi#645 WebClient(run_async=False) no longer depends on asyncio by default
* slackapi#633 WebClient(run_async=False) doesn't internally depend on aiohttp
* slackapi#631 When run_async=True, RTM listner can be a normal function and WebClient is free from the event loop
* slackapi#630 WebClient no longer depends on aiohttp when run_async=False
* slackapi#497 Fixed when run_async=False / can be closed as we don't support run_async=True for this use case (in Flask)
@seratch
Copy link
Member

seratch commented May 15, 2020

Let me close this issue now as #662 resolved this for the run_async=False mode.

@seratch seratch closed this as completed May 15, 2020
@seratch
Copy link
Member

seratch commented May 15, 2020

👋 slackclient 2.6.0rc1 is out. The pre-release version contains fixes for your issue described here.
https://pypi.org/project/slackclient/2.6.0rc1/

One week later from now, we'll be releasing version 2.6.0 to PyPI.

If you have a chance, could you try the release candidate version out and let us know your feedback? Thank you very much for being patient with this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:concurrency Issues and PRs related to concurrency bug M-T: A confirmed bug report. Issues are confirmed when the reproduction steps are documented question M-T: User needs support to use the project rtm-client Version: 2x web-client
Projects
None yet
Development

No branches or pull requests

8 participants