Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: connect to redis sentinel for redis queue (backport #25506) #25557

Merged

Conversation

mergify[bot]
Copy link
Contributor

@mergify mergify bot commented Mar 20, 2024

Please provide enough information so that others can review your pull request:

Use redis sentinel for redis queue connection.
It is setup for production only.
Uses extra keys from common_site_config.json

Explain the details for making this change. What existing problem does the pull request solve?

At the moment we can connect to single redis master for queue. With this we can connect to sentinel cluster.

https://redis.io/docs/management/sentinel

To try it with frappe_docker/devcontainer make following changes to .devcontainer/docker-compose.yml

version: "3.7"
services:
  mariadb:
    image: docker.io/mariadb:10.6
    command:
      - --character-set-server=utf8mb4
      - --collation-server=utf8mb4_unicode_ci
      - --skip-character-set-client-handshake
      - --skip-innodb-read-only-compressed # Temporary fix for MariaDB 10.6
    environment:
      - MYSQL_ROOT_PASSWORD=123
    volumes:
      - mariadb-data:/var/lib/mysql

  redis-cache:
    image: redis:alpine

  redis-queue:
    image: redis:alpine


## Change begins here ##
  redis-queue-sentinel:
    image: docker.io/bitnami/redis-sentinel:7.2
    environment:
      - REDIS_SENTINEL_PASSWORD=sentinelpassword
      - REDIS_MASTER_HOST=redis-queue-master
      - REDIS_MASTER_PASSWORD=masterpassword

  redis-queue-master:
    image: docker.io/bitnami/redis:7.2
    environment:
      - REDIS_PASSWORD=masterpassword
## Change ends here ##

  frappe:
    image: docker.io/frappe/bench:latest
    command: sleep infinity
    environment:
      - SHELL=/bin/bash
    volumes:
      - ..:/workspace:cached
      - ${HOME}/.ssh:/home/frappe/.ssh
    working_dir: /workspace/development
    ports:
      - 8000-8005:8000-8005
      - 9000-9005:9000-9005

volumes:
  mariadb-data:

add redis_cache_* keys to common_site_config.json

{
 "background_workers": 1,
 "db_host": "mariadb",
 "db_type": "mariadb",
 "developer_mode": 1,
 "file_watcher_port": 6788,
 "frappe_user": "frappe",
 "gunicorn_workers": 25,
 "live_reload": true,
 "rebase_on_pull": false,
 "redis_socketio": "redis://redis-queue:6379",
 "redis_cache": "redis://redis-cache:6379",
 "redis_queue": "redis://redis-queue:6379",
 "redis_queue_sentinel_enabled": 1,
 "redis_queue_sentinels": ["redis-queue-sentinel:26379"],
 "redis_queue_sentinel_password": "sentinelpassword",
 "redis_queue_master_service": "mymaster",
 "redis_queue_master_password": "masterpassword",
 "restart_supervisor_on_update": false,
 "restart_systemd_on_update": false,
 "serve_default_site": true,
 "shallow_clone": true,
 "socketio_port": 9000,
 "use_redis_auth": false,
 "webserver_port": 8000
}

Note: Nothing has changed to the redis_queue key. It will be used by socketio.

Screenshots/GIFs

Docs: https://frappeframework.com/app/wiki-page-patch/372939da4e


This is an automatic backport of pull request #25506 done by [Mergify](https://mergify.com).

@mergify mergify bot requested a review from a team as a code owner March 20, 2024 01:36
@mergify mergify bot requested review from ankush and removed request for a team March 20, 2024 01:36
@mergify mergify bot assigned revant Mar 20, 2024
@github-actions github-actions bot added the add-test-cases Add test case to validate fix or enhancement label Mar 20, 2024
@akhilnarang akhilnarang merged commit 82a33c3 into version-15-hotfix Mar 20, 2024
19 checks passed
@akhilnarang akhilnarang deleted the mergify/bp/version-15-hotfix/pr-25506 branch March 20, 2024 13:02
frappe-pr-bot pushed a commit that referenced this pull request Mar 27, 2024
# [15.19.0](v15.18.2...v15.19.0) (2024-03-27)

### Bug Fixes

* add missing arg while invoking _download_multi_pdf ([f82ed9f](f82ed9f))
* allow page length 2500 ([#25062](#25062)) ([75b1f6c](75b1f6c))
* **Contact form:** make email translatable ([800cfee](800cfee))
* **Contact form:** make title and options translatable ([b1dbd79](b1dbd79))
* **Contact form:** translate internal notification to system language ([f67e5f1](f67e5f1))
* DB Query distinct handling with full table name ([#25594](#25594)) ([4fe0498](4fe0498))
* diff after converting to html to text ([#25582](#25582)) ([#25584](#25584)) ([bd7372d](bd7372d))
* Mark totals row correctly for print ([#25629](#25629)) ([#25631](#25631)) ([725018f](725018f))
* only add title field in search if it exists ([#25634](#25634)) ([#25636](#25636)) ([b2260c1](b2260c1))
* redirect after login, todo filters (backport [#25521](#25521)) ([#25561](#25561)) ([d9f9aa4](d9f9aa4)), closes [#25455](#25455) [#25455](#25455)
* respect `null` as number value ([#25639](#25639)) ([5b64ac9](5b64ac9))
* set list row height to 40px ([#25619](#25619)) ([#25625](#25625)) ([af406f9](af406f9))
* **test:** update RQ serialization test ([c764282](c764282))
* Translate form and workflow builder (backport [#25482](#25482)) ([#25547](#25547)) ([01f469f](01f469f))
* **webhook:** `r` is referenced here before its initialized ([a60bb4e](a60bb4e)), closes [#21064](#21064)
* **workspace:** Don't allow Welcome as default workspace ([819a24c](819a24c))

### Features

* allow setting a custom rate limit for `login via email link` feature ([c7af627](c7af627))
* connect to redis sentinel for redis queue ([#25557](#25557)) ([82a33c3](82a33c3))
* drop sentry's RQ integration ([1eab176](1eab176))
* **workspace:** Allow user to choose a default workspace ([bfa1a83](bfa1a83))

### Performance Improvements

* Faster "show title in link field" on list view ([#25597](#25597)) ([#25622](#25622)) ([f62a30c](f62a30c))
* remove useless sorting on docstatus ([#25571](#25571)) ([#25591](#25591)) ([5e5853e](5e5853e))

### Reverts

* Revert "fix: escape text types before setting disp area (#25520) (#25523)" (#25603) ([bbf55df](bbf55df)), closes [#25520](#25520) [#25523](#25523) [#25603](#25603)
@frappe-pr-bot
Copy link
Collaborator

🎉 This PR is included in version 15.19.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 11, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
add-test-cases Add test case to validate fix or enhancement released
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants