Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Redis takes too much Resources! #25

Closed
vknabel opened this issue May 14, 2020 · 1 comment
Closed

Redis takes too much Resources! #25

vknabel opened this issue May 14, 2020 · 1 comment
Assignees
@vknabel
Labels
app:server bug Something isn't working
Projects
Development
Milestone
Interne Produktiv...

Comments

@vknabel
Copy link
Owner

vknabel commented May 14, 2020 •
edited 

CONTAINER ID        NAME                                      CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
f822a7c7870e        puffery-vapor_puffery-queues_1            0.04%               4.168MiB / 985.4MiB   0.42%               103kB / 310kB       73.8MB / 0B         4
2b5252e7ec66        puffery-vapor_inbound-middleware_1        0.00%               15.14MiB / 985.4MiB   1.54%               637kB / 664kB       142MB / 0B          12
a96cb3b9778d        puffery-vapor_puffery_1                   0.01%               6.109MiB / 985.4MiB   0.62%               825kB / 136kB       113MB / 0B          4
b8f0a56f4d15        puffery-vapor_redis_1                     95.68%              274.1MiB / 985.4MiB   27.81%              17.2MB / 194kB      116MB / 8.89MB      17
f0d06a1b27b1        db                                        0.01%               6.594MiB / 985.4MiB   0.67%               49.1kB / 72.6kB     110MB / 1.42MB      8

docker-compose.yml

version: "3.7"
services:
  redis:
    image: redis
    restart: always
    ports:
      - "6379:6379"
    volumes:
      - ./data/redis:/data
    command: ["redis-server", "--appendonly", "yes"]

Logs

redis_1               | 1:C 14 May 2020 20:19:06.081 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1               | 1:C 14 May 2020 20:19:06.082 # Redis version=5.0.8, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1               | 1:C 14 May 2020 20:19:06.082 # Configuration loaded
redis_1               | 1:M 14 May 2020 20:19:06.083 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
redis_1               | 1:M 14 May 2020 20:19:06.083 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
redis_1               | 1:M 14 May 2020 20:19:06.083 # Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
redis_1               | 1:M 14 May 2020 20:19:06.085 * Running mode=standalone, port=6379.
redis_1               | 1:M 14 May 2020 20:19:06.085 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1               | 1:M 14 May 2020 20:19:06.085 # Server initialized
redis_1               | 1:M 14 May 2020 20:19:06.085 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1               | 1:M 14 May 2020 20:19:06.085 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1               | 1:M 14 May 2020 20:19:06.086 * Reading RDB preamble from AOF file...
redis_1               | 1:M 14 May 2020 20:19:06.086 * Reading the remaining AOF tail...
redis_1               | 1:M 14 May 2020 20:19:06.086 * DB loaded from append only file: 0.000 seconds
redis_1               | 1:M 14 May 2020 20:19:06.086 * Ready to accept connections
redis_1               | 1:signal-handler (1589487577) Received SIGTERM scheduling shutdown...
redis_1               | 1:M 14 May 2020 20:19:37.386 # User requested shutdown...
redis_1               | 1:M 14 May 2020 20:19:37.388 * Calling fsync() on the AOF file.
redis_1               | 1:M 14 May 2020 20:19:37.389 # Redis is now ready to exit, bye bye...
redis_1               | 1:C 14 May 2020 20:20:05.953 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1               | 1:C 14 May 2020 20:20:05.953 # Redis version=5.0.8, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1               | 1:C 14 May 2020 20:20:05.953 # Configuration loaded
redis_1               | 1:M 14 May 2020 20:20:05.954 # You requested maxclients of 10000 requiring at least 10032 max file descriptors.
redis_1               | 1:M 14 May 2020 20:20:05.954 # Server can't set maximum open files to 10032 because of OS error: Operation not permitted.
redis_1               | 1:M 14 May 2020 20:20:05.954 # Current maximum open files is 4096. maxclients has been reduced to 4064 to compensate for low ulimit. If you need higher maxclients increase 'ulimit -n'.
redis_1               | 1:M 14 May 2020 20:20:05.958 * Running mode=standalone, port=6379.
redis_1               | 1:M 14 May 2020 20:20:05.958 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis_1               | 1:M 14 May 2020 20:20:05.958 # Server initialized
redis_1               | 1:M 14 May 2020 20:20:05.958 # WARNING overcommit_memory is set to 0! Background save may fail under low memory condition. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1               | 1:M 14 May 2020 20:20:05.958 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis_1               | 1:M 14 May 2020 20:20:05.958 * Reading RDB preamble from AOF file...
redis_1               | 1:M 14 May 2020 20:20:05.958 * Reading the remaining AOF tail...
redis_1               | 1:M 14 May 2020 20:20:05.958 * DB loaded from append only file: 0.000 seconds
redis_1               | 1:M 14 May 2020 20:20:05.958 * Ready to accept connections
redis_1               | 1:S 14 May 2020 20:25:38.317 * Before turning into a replica, using my master parameters to synthesize a cached master: I may be able to synchronize with the new master with just a partial transfer.
redis_1               | 1:S 14 May 2020 20:25:38.324 * REPLICAOF 51.83.171.41:8886 enabled (user request from 'id=5 addr=134.209.57.77:42058 fd=10 name= age=1 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=46 qbuf-free=32722 obl=0 oll=0 omem=0 events=r cmd=slaveof')
redis_1               | 1:S 14 May 2020 20:25:39.205 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:39.205 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:39.229 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:39.254 * Master replied to PING, replication can continue...
redis_1               | 1:S 14 May 2020 20:25:39.307 * Trying a partial resynchronization (request 064d39fd8ac30b1632587e120229ead5298114ab:1).
redis_1               | 1:S 14 May 2020 20:25:39.342 * Full resync from master: ZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZZ:1
redis_1               | 1:S 14 May 2020 20:25:39.342 * Discarding previously cached master state.
redis_1               | 1:S 14 May 2020 20:25:39.344 * MASTER <-> REPLICA sync: receiving 55656 bytes from master
redis_1               | 1:S 14 May 2020 20:25:39.413 * MASTER <-> REPLICA sync: Flushing old data
redis_1               | 1:S 14 May 2020 20:25:39.414 * MASTER <-> REPLICA sync: Loading DB in memory
redis_1               | 1:S 14 May 2020 20:25:39.414 # Wrong signature trying to load DB from file
redis_1               | 1:S 14 May 2020 20:25:39.414 # Failed trying to load the MASTER synchronization DB from disk
redis_1               | 1:S 14 May 2020 20:25:39.415 * Background append only file rewriting started by pid 16
redis_1               | 1:S 14 May 2020 20:25:39.439 * AOF rewrite child asks to stop sending diffs.
redis_1               | 16:C 14 May 2020 20:25:39.439 * Parent agreed to stop sending diffs. Finalizing AOF...
redis_1               | 16:C 14 May 2020 20:25:39.440 * Concatenating 0.00 MB of AOF diff received from parent.
redis_1               | 16:C 14 May 2020 20:25:39.440 * SYNC append only file rewrite performed
redis_1               | 16:C 14 May 2020 20:25:39.441 * AOF rewrite: 0 MB of memory used by copy-on-write
redis_1               | 1:S 14 May 2020 20:25:39.507 * Background AOF rewrite terminated with success
redis_1               | 1:S 14 May 2020 20:25:39.507 * Residual parent diff successfully flushed to the rewritten AOF (0.00 MB)
redis_1               | 1:S 14 May 2020 20:25:39.507 * Background AOF rewrite finished successfully
redis_1               | 1:S 14 May 2020 20:25:40.212 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:40.212 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:40.242 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:40.273 # Error condition on socket for SYNC: Broken pipe
redis_1               | 1:S 14 May 2020 20:25:41.217 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:41.217 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:41.242 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:41.270 # Error condition on socket for SYNC: Broken pipe
redis_1               | 1:S 14 May 2020 20:25:42.219 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:42.220 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:42.240 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:42.262 # Error condition on socket for SYNC: Broken pipe
redis_1               | 1:S 14 May 2020 20:25:43.224 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:43.225 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:43.248 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:43.271 # Error condition on socket for SYNC: Broken pipe
redis_1               | 1:S 14 May 2020 20:25:44.228 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:44.229 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:44.248 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:44.268 # Error condition on socket for SYNC: Connection reset by peer
redis_1               | 1:S 14 May 2020 20:25:45.232 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:45.232 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:45.254 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:45.275 # Error reply to PING from master: '-Reading from master: Resource temporarily unavailable'
redis_1               | 1:S 14 May 2020 20:25:46.236 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:46.236 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:46.256 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:46.277 # Error condition on socket for SYNC: Broken pipe
redis_1               | 1:S 14 May 2020 20:25:46.711 * Module 'system' loaded from ./red2.so
redis_1               | 1:S 14 May 2020 20:25:47.239 * Connecting to MASTER 51.83.171.41:8886
redis_1               | 1:S 14 May 2020 20:25:47.239 * MASTER <-> REPLICA sync started
redis_1               | 1:S 14 May 2020 20:25:47.261 * Non blocking connect for SYNC fired the event.
redis_1               | 1:S 14 May 2020 20:25:47.283 # Error condition on socket for SYNC: Connection reset by peer
redis_1               | 1:M 14 May 2020 20:25:47.475 # Setting secondary replication ID to 064d39fd8ac30b1632587e120229ead5298114ab, valid up to offset: 1. New replication ID is 49a489c1cc8e6493a261afee0cda2d4a1de3be8c
redis_1               | 1:M 14 May 2020 20:25:47.476 * MASTER MODE enabled (user request from 'id=5 addr=134.209.57.77:42058 fd=10 name= age=10 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=34 qbuf-free=32734 obl=0 oll=0 omem=0 events=r cmd=slaveof')
redis_1               | 1:M 14 May 2020 20:25:52.705 * Module system unloaded
@vknabel vknabel added bug Something isn't working app:server labels May 14, 2020
@vknabel vknabel added this to Discussion in Development via automation May 14, 2020
@vknabel vknabel added this to the Interne Produktivnutzung milestone May 14, 2020
@vknabel vknabel moved this from Discussion to To do in Development May 14, 2020
@vknabel vknabel self-assigned this May 14, 2020
@vknabel
Copy link
Owner Author

vknabel commented May 14, 2020

So viel zum Thema „ich gehe jetzt schlafen und mach das issue auf. Ich kann ja morgen nachschauen.“

Der Fix stand in den logs. 👍

@vknabel vknabel closed this as completed May 14, 2020
Development automation moved this from To do to Done May 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@vknabel vknabel

Labels
app:server bug Something isn't working
Projects
Development
  
Done
Milestone
Interne Produktivnutzung
Development

No branches or pull requests
1 participant
@vknabel