SocketIO no longer working after updating libraries

[denisvitez]

Hi,

so I was successfully using the Flask-SocketIO in my flask application that was communicating with the Vue.js application. I have now updated python packages to their latest versions and this seemed to have broken the SocketIO implementation inside my project.

Old package versions:

Flask==1.1.2
Flask-SocketIO==5.0.1
python-engineio==4.0.0
python-socketio==5.0.4
redis==3.5.0
Werkzeug==1.0.1

New package versions:

Flask==2.1.2
Flask-SocketIO==5.2.0
python-engineio==4.3.3
python-socketio==5.7.0
redis==4.3.4
Werkzeug==2.1.2

Socket.IO is initialized as:

socketio.init_app(
        flask_app,
        message_queue='redis://',
        cors_allowed_origins='*',
        logger=True,
        engineio_logger=True
    )

So the error that was shown after running the application was:
The WebSocket transport is not available, you must install a WebSocket server that is compatible with your async mode to enable it. See the documentation for details. (further occurrences of this error will be logged with level INFO)

I have both eventlet and gevent installed as dependencies but that didn't seem to work, so I had to manually set gevent as async mode:

socketio.init_app(
        flask_app,
        message_queue='redis://',
        cors_allowed_origins='*',
        logger=True,
        engineio_logger=True,
        async_mode='gevent'
    )

So after that I started to get the following error:
RuntimeError: Redis requires a monkey patched socket library to work with gevent

So I added the monkey patch for gevent:

from gevent import monkey
monkey.patch_all(subprocess=True)

After that I started to get a new error:
RuntimeError: You need to use the gevent-websocket server. See the Deployment section of the documentation for more information.

After some searching I found a post where it was said that flask run is not working correctly, so I started the application "manually":
socketio.run(app, host='0.0.0.0')

This got rid of the problem in logs. But even though there are no more errors in the logs and emit is logged, no message will get to my client.

9RmoW9vBp8ZoK6I1AAAA: Sending packet OPEN data {'sid': '9RmoW9vBp8ZoK6I1AAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
redis backend initialized.
9RmoW9vBp8ZoK6I1AAAA: Received packet MESSAGE data 0
9RmoW9vBp8ZoK6I1AAAA: Sending packet MESSAGE data 4"Unable to connect"
9RmoW9vBp8ZoK6I1AAAA: Received request to upgrade to websocket
9RmoW9vBp8ZoK6I1AAAA: Upgrade to websocket successful
emitting event "updateOutput" to all [/]
pubsub message: emit

Does anyone have any ideas what could be causing the messages to not work, or if I somehow screwed up the setup of the Flask-SocketIO with the newer version?

[miguelgrinberg]

@denisvitez For all the problems except the final one, you could have saved a lot of time if you read the docs. Those are all basic problems that occur because you are doing things in ways that are incorrect and/or unsupported.

The last error (unable to connect) could be related to a change that went into the 5.7.0 release. Can I ask you to try 5.6.0 and report if that addresses this particular problem? Thank you.

[miguelgrinberg]

Adding namespaces='*' is not really a great solution, but more of a workaround. The issue is that your client is attempting to connect to a namespace that is not implemented by your server. The correct solution is to change the client to only connect to the namespace(s) that your server understands.

[tumbleshack]

Perhaps I'm misinterpreting the server logs, but it looks like the client is opening a connection and emitting an event on /test

To clarify the problem observed: when using the built-in flask server, the client and server connect (with handlers executed) on the /test namespace. When running the app with gunicorn, however, the handlers don't execute.

Until adding the namespace=*, the connection would also fail.

[miguelgrinberg]

@tumbleshack you haven't shared a failure log, so I cannot comment on that. I can only comment on the description you provided. Switching between Flask and Gunicorn was not mentioned earlier. This has to be a bug in how your application is structured, but again, you have not provided the relevant code, so I cannot comment on that.

[tumbleshack]

Thanks for your quick replies @miguelgrinberg - your advice in this issue looks like the closest problem to what I have, but I'm not quite sure how to apply your suggestion to class Namespaces.

Here's my setup. To be clear, I expect to observe a log from app.logger.info("my_event", data) after the log received event "my_event" from XceICgUgJukvQho2AAAD [/test] in the trace above.

backend/app/__init__.py

from apiflask import APIFlask
import os

def load_config(app):
    app.config.from_object('backend.config.default')
    app.config.from_object('backend.instance.default')

    # Load the file specified by the APP_RUN_CONFIG environment variable
    # Variables defined here will override those in the default configuration
    env_config_name = os.environ.get('APP_RUN_CONFIG', 'development')
    env_config_module = 'backend.config.' + env_config_name
    instance_config_module = 'backend.instance.' + env_config_name
    app.config.from_object(env_config_module)
    # Load the configuration from the instance folder
    app.config.from_object(instance_config_module)

app = APIFlask(__name__, instance_relative_config=True)
load_config(app)

backend/__init__.py

from flask_security import Security, SQLAlchemySessionUserDatastore
from .database import init_db, db_session
from .models.user import User, Role
from flask_wtf.csrf import CSRFProtect
from flask_cors import CORS
from .api import api_blueprint
from flask_socketio import SocketIO, Namespace, emit
from .app import app

csrf = CSRFProtect(app)

# Allow request from frontend domains
CORS(
    app,
    supports_credentials=True,  # needed for cross domain cookie support
    resources="/*",
    allow_headers="*",
    origins=app.config['CORS_ORIGINS'],
)

# Setup Flask-Security
user_datastore = SQLAlchemySessionUserDatastore(db_session, User, Role)
app.security = Security(app, user_datastore)
    
init_db() # helper only calls Base.metadata.create_all(bind=engine)

class SecretNamespace(Namespace):
    def on_connect(self):
        app.logger.info("connected!")
    
    def on_disconnect(self):
        app.logger.info("disconnected!")
        pass

    def on_my_event(self, data):
        app.logger.info("my_event", data)
        emit('my_response', {"data": "ping response"}, broadcast=False)
        app.logger.info("emitted my_response")


socket_app = SocketIO(app, cors_allowed_origins=app.config['CORS_ORIGINS'], logger=True, engineio_logger=True, log_output=True)
socket_app.on_namespace(SecretNamespace('/test'))
socket_app.init_app(app, namespaces='*')
app.register_blueprint(api_blueprint, url_prefix='/api')

Run command gunicorn --worker-class eventlet -w 1 backend:app -b 0.0.0.0:5000

---

Re: namespace='*' topic, if I replace socket_app.init_app(app, namespaces='*') with socket_app.init_app(app), then I observe the following when the client tries to connect

BBW8DCozbfrT3C1rAAAA: Sending packet OPEN data {'sid': 'BBW8DCozbfrT3C1rAAAA', 'upgrades': ['websocket'], 'pingTimeout': 20000, 'pingInterval': 25000}
BBW8DCozbfrT3C1rAAAA: Received packet MESSAGE data 0/test,
BBW8DCozbfrT3C1rAAAA: Sending packet MESSAGE data 4/test,"Unable to connect"
BBW8DCozbfrT3C1rAAAA: Received request to upgrade to websocket
BBW8DCozbfrT3C1rAAAA: Upgrade to websocket successful

[miguelgrinberg]

@tumbleshack the issue is in how you initialize the Flask-SocketIO extension. You can either initialize it entirely in the constructor (by passing app as first argument), or else you can initialize it in to phases, first by using an empty constructor, then passing app and other arguments to init_app(). You have done an incorrect mix of the two methods, and for that reason your namespace isn't correctly registered. Try this instead:

socket_app = SocketIO()
socket_app.on_namespace(SecretNamespace('/test'))
socket_app.init_app(app, cors_allowed_origins=app.config['CORS_ORIGINS'], logger=True, engineio_logger=True, log_output=True)
app.register_blueprint(api_blueprint, url_prefix='/api')