feat: connect to redis sentinel for redis queue

[revant]

Please provide enough information so that others can review your pull request:

Use redis sentinel for redis queue connection.
It is setup for production only.
Uses extra keys from common_site_config.json

Explain the details for making this change. What existing problem does the pull request solve?

At the moment we can connect to single redis master for queue. With this we can connect to sentinel cluster.

https://redis.io/docs/management/sentinel

To try it with frappe_docker/devcontainer make following changes to .devcontainer/docker-compose.yml

version: "3.7"
services:
  mariadb:
    image: docker.io/mariadb:10.6
    command:
      - --character-set-server=utf8mb4
      - --collation-server=utf8mb4_unicode_ci
      - --skip-character-set-client-handshake
      - --skip-innodb-read-only-compressed # Temporary fix for MariaDB 10.6
    environment:
      - MYSQL_ROOT_PASSWORD=123
    volumes:
      - mariadb-data:/var/lib/mysql

  redis-cache:
    image: redis:alpine

  redis-queue:
    image: redis:alpine


## Change begins here ##
  redis-queue-sentinel:
    image: docker.io/bitnami/redis-sentinel:7.2
    environment:
      - REDIS_SENTINEL_PASSWORD=sentinelpassword
      - REDIS_MASTER_HOST=redis-queue-master
      - REDIS_MASTER_PASSWORD=masterpassword

  redis-queue-master:
    image: docker.io/bitnami/redis:7.2
    environment:
      - REDIS_PASSWORD=masterpassword
## Change ends here ##

  frappe:
    image: docker.io/frappe/bench:latest
    command: sleep infinity
    environment:
      - SHELL=/bin/bash
    volumes:
      - ..:/workspace:cached
      - ${HOME}/.ssh:/home/frappe/.ssh
    working_dir: /workspace/development
    ports:
      - 8000-8005:8000-8005
      - 9000-9005:9000-9005

volumes:
  mariadb-data:

add redis_cache_* keys to common_site_config.json

{
 "background_workers": 1,
 "db_host": "mariadb",
 "db_type": "mariadb",
 "developer_mode": 1,
 "file_watcher_port": 6788,
 "frappe_user": "frappe",
 "gunicorn_workers": 25,
 "live_reload": true,
 "rebase_on_pull": false,
 "redis_socketio": "redis://redis-queue:6379",
 "redis_cache": "redis://redis-cache:6379",
 "redis_queue": "redis://redis-queue:6379",
 "redis_queue_sentinel_enabled": 1,
 "redis_queue_sentinels": ["redis-queue-sentinel:26379"],
 "redis_queue_sentinel_password": "sentinelpassword",
 "redis_queue_master_service": "mymaster",
 "redis_queue_master_password": "masterpassword",
 "restart_supervisor_on_update": false,
 "restart_systemd_on_update": false,
 "serve_default_site": true,
 "shallow_clone": true,
 "socketio_port": 9000,
 "use_redis_auth": false,
 "webserver_port": 8000
}

Note: Nothing has changed to the redis_queue key. It will be used by socketio.

Screenshots/GIFs

Docs: https://frappeframework.com/app/wiki-page-patch/372939da4e

[revant]

• I added Webhook on Note. It uses background workers.
• triggered creation of notes in a loop.
• stopped the redis-queue-master
• The background jobs were not affected
• Started the master again.
• System was not affected.

frappe.enqueue() does get_queue() which in turn does get_redis_conn(). This ensures we always get master from sentinels and connect to it.

For start_worker() rq cli is doing same thing. https://github.com/rq/rq/blob/bab0061da18761da772e3244d9efedacefb6fad4/rq/cli/helpers.py#L48-L63

I believe it is working as expected.