Centrifugo on Production Server

[pythoneast]

Hello!

I am using Centrifugo in my Django project as a tool for publishing notifications.
All services (also Centrifugo) are in Docker containers on the same server.
I use Nginx as a proxy server for Centrifugo.
Centrifugo is available on endpoint https://my-domain.io/centrifugo
I try to publish a notification with publish method, but get an error:

from cent import Client, CentException
url = "https://my_domain.io/centrifugo"
api_key = "my_api_key"
client = Client(url, api_key=api_key, timeout=1)
channel = 'channel'

try:
    client.publish('channel', {"value": "My message"})
except CentException as e:
    print(e)

HTTPSConnectionPool(host='my-domain.io', port=443): Max retries exceeded with url: /centrifugo/api (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f088e765130>, 'Connection to my-domain.io timed out. (connect timeout=1)'))

Also I try to publish with python requests package and get almost the same error:

import json
import requests


command = {
    "method": "publish",
    "params": {
        "channel": "channel",
        "data": {
            "value": "My message!"
        }
    }
}


api_key = "my-api-key"
data = json.dumps(command)
headers = {'Content-type': 'application/json', 'Authorization': 'apikey ' + api_key}
resp = requests.post("https://my-domain.io/centrifugo/api", data=data, headers=headers)

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 159, in _new_conn
    conn = connection.create_connection(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 976, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 308, in connect
    conn = self._new_conn()
  File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 171, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f088e765970>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 724, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='my-domain.io', port=443): Max retries exceeded with url: /centrifugo/api (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f088e765970>: Failed to establish a new connection: [Errno 110] Connection timed out'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='my-domain.io', port=443): Max retries exceeded with url: /centrifugo/api (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f088e765970>: Failed to establish a new connection: [Errno 110] Connection timed out'))

But when I run the same commands from my local server, everything works as expected.
What am I doing wrong? Should I use a separate server (droplet) for Centrifugo?
Thank you in advance!

[FZambia]

I believe that the reason not in Centrifugo itself but in your setup configuration. Though I can't say exactly where the problem comes from without the possibility to investigate this. This looks like missing network connectivity. Try to reduce problem scope. If you are familiar with tcpdump – it can give you insights.

[pythoneast]

Thank you @FZambia for your help.
I'll keep trying to resolve the problem. I suspect that the problem in my web server configurations.

[FZambia]

Welcome, maybe also try to use curl or similar when making publish request (since this library uses requests internally so it's not unexpected you got the same error when using requests manually).

curl --header "Content-Type: application/json" \
  --header "Authorization: apikey <YOUR_API_KEY>" \
  --request POST \
  --data '{"method": "publish", "params": {"channel": "channel", "data": {"value": "test"}}}' \
  http://localhost:8000/api

[pythoneast]

I've connected to my external server via ssh and tested sending request to publish a notification in terminal. I could send a request for publishing a notification via curl successfully.

The problem occurs when I enter a Docker container for Django backend inside my external server. Neither Cent library nor Requests package could send a request for publishing a notification. As I understood the problem is in Docker Network.

[pythoneast]

@FZambia I solved my problem. The reason was that docker did not send the request directly to https://my-domain/centrifugo/api, since this is the host of the server itself. I solved the problem by creating a separate network in docker and setting static ips for containers. After that, I simply sent requests for publication using the static ip of the container with Centrifugo.
I apologize for disturbing you. The problem was not with Centrifugo itself, but with the Docker settings.
Thanks for your help and a great open source product like Centrifugo.

[FZambia]

Glad you solved it, thanks for kind words 👍