Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Redis 5.0.5 crashed by signal: 11 #11851

Open
lneuhaus opened this issue Feb 27, 2023 · 0 comments
Open

[BUG] Redis 5.0.5 crashed by signal: 11 #11851

lneuhaus opened this issue Feb 27, 2023 · 0 comments

Comments

@lneuhaus
Copy link

lneuhaus commented Feb 27, 2023

Describe the bug
My redis instance died from signal 11. This was in regular operation and happened unexpectedly. The redis commands that I sent at the time can only have been typical, but I do not have enough context from logfiles to trace this exactly.

To reproduce
I am unable to reproduce this.

Expected behavior
No crash.

Additional information
I am running redis in a docker container, using the image redis:5.0.5.

Bug report by redis:


=== REDIS BUG REPORT START: Cut & paste starting from here ===
7:M 26 Feb 2023 13:42:15.741 # Redis 5.0.5 crashed by signal: 11
7:M 26 Feb 2023 13:42:15.741 # Crashed running the instruction at: 0x5560f7ae987c
7:M 26 Feb 2023 13:42:15.741 # Accessing address: 0x2000
7:M 26 Feb 2023 13:42:15.741 # Failed assertion: <no assertion failed> (<no file>:0)

------ STACK TRACE ------
EIP:
redis-server 0.0.0.0:6379(dictFind+0x5c)[0x5560f7ae987c]

Backtrace:
redis-server 0.0.0.0:6379(logStackTrace+0x32)[0x5560f7b31112]
redis-server 0.0.0.0:6379(sigsegvHandler+0x9e)[0x5560f7b317ee]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7fa76b132730]
redis-server 0.0.0.0:6379(dictFind+0x5c)[0x5560f7ae987c]
redis-server 0.0.0.0:6379(zsetAdd+0x68)[0x5560f7b19208]
redis-server 0.0.0.0:6379(zaddGenericCommand+0x2ff)[0x5560f7b19b5f]
redis-server 0.0.0.0:6379(call+0xa7)[0x5560f7aed2b7]
redis-server 0.0.0.0:6379(processCommand+0x51e)[0x5560f7aedb5e]
redis-server 0.0.0.0:6379(processInputBuffer+0x171)[0x5560f7afdd51]
redis-server 0.0.0.0:6379(aeProcessEvents+0x101)[0x5560f7ae72e1]
redis-server 0.0.0.0:6379(aeMain+0x2b)[0x5560f7ae76eb]
redis-server 0.0.0.0:6379(main+0x4b9)[0x5560f7ae4569]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x7fa76af8309b]
redis-server 0.0.0.0:6379(_start+0x2a)[0x5560f7ae47aa]

------ INFO OUTPUT ------
# Server
redis_version:5.0.5
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:442b43d467cd2b03
redis_mode:standalone
os:Linux 5.15.0-60-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:8.3.0
process_id:7
run_id:c75e1ae6c1384702eec8adee7fe290f170c49d7d
tcp_port:6379
uptime_in_seconds:128129
uptime_in_days:1
hz:10
configured_hz:10
lru_clock:16474551
executable:/data/redis/redis-server
config_file:/tmp/redis.conf

# Clients
connected_clients:28
client_recent_max_input_buffer:2
client_recent_max_output_buffer:0
blocked_clients:0

# Memory
used_memory:281273920
used_memory_human:268.24M
used_memory_rss:290750464
used_memory_rss_human:277.28M
used_memory_peak:416032232
used_memory_peak_human:396.76M
used_memory_peak_perc:67.61%
used_memory_overhead:1335920
used_memory_startup:791344
used_memory_dataset:279938000
used_memory_dataset_perc:99.81%
allocator_allocated:281355200
allocator_active:281792512
allocator_resident:290021376
total_system_memory:67168899072
total_system_memory_human:62.56G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.00
allocator_frag_bytes:437312
allocator_rss_ratio:1.03
allocator_rss_bytes:8228864
rss_overhead_ratio:1.00
rss_overhead_bytes:729088
mem_fragmentation_ratio:1.03
mem_fragmentation_bytes:9477808
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:539360
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0

# Persistence
loading:0
rdb_changes_since_last_save:2380
rdb_bgsave_in_progress:0
rdb_last_save_time:1677418833
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:1585152
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0

# Stats
total_connections_received:12720
total_commands_processed:23802075
instantaneous_ops_per_sec:215
total_net_input_bytes:2223140058
total_net_output_bytes:1252510295
instantaneous_input_kbps:20.02
instantaneous_output_kbps:11.67
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
evicted_keys:0
keyspace_hits:16984861
keyspace_misses:3122678
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:13893
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0

# Replication
role:master
connected_slaves:0
master_replid:a7bb7e80d76a12122710f58fb209e604a6452a9a
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:496.779755
used_cpu_user:339.369515
used_cpu_sys_children:43.685153
used_cpu_user_children:345.674342

# Commandstats
cmdstat_get:calls=12813,usec=15100,usec_per_call=1.18
cmdstat_zremrangebyrank:calls=103,usec=982882,usec_per_call=9542.54
cmdstat_type:calls=104,usec=123,usec_per_call=1.18
cmdstat_ping:calls=12687,usec=6909,usec_per_call=0.54
cmdstat_info:calls=281018,usec=5457531,usec_per_call=19.42
cmdstat_zadd:calls=3221343,usec=38385310,usec_per_call=11.92
cmdstat_keys:calls=166572,usec=3051145,usec_per_call=18.32
cmdstat_zrange:calls=19979305,usec=59381379,usec_per_call=2.97
cmdstat_set:calls=12813,usec=18512,usec_per_call=1.44
cmdstat_zrangebyscore:calls=115317,usec=205252,usec_per_call=1.78

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=104,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=10 addr=127.0.0.1:33190 fd=14 name= age=128124 idle=128110 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=11 addr=127.0.0.1:33192 fd=15 name= age=128124 idle=5746 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=188 addr=127.0.0.1:44826 fd=21 name= age=126393 idle=52265 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=3 addr=127.0.0.1:33138 fd=7 name= age=128128 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=189 qbuf-free=32579 obl=0 oll=0 omem=0 events=r cmd=zadd
id=6378 addr=127.0.0.1:36482 fd=27 name= age=63947 idle=51313 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=4 addr=127.0.0.1:33146 fd=8 name= age=128128 idle=128118 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=17 addr=127.0.0.1:41600 fd=20 name= age=128118 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=6326 addr=127.0.0.1:46188 fd=23 name= age=64443 idle=60456 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=610 addr=127.0.0.1:32776 fd=22 name= age=122141 idle=9281 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=info
id=6613 addr=127.0.0.1:36156 fd=33 name= age=61695 idle=50836 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=6383 addr=127.0.0.1:37364 fd=28 name= age=63903 idle=50985 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=6377 addr=127.0.0.1:36474 fd=26 name= age=63947 idle=51292 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=5 addr=127.0.0.1:33148 fd=9 name= age=128128 idle=3 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=6376 addr=127.0.0.1:36462 fd=24 name= age=63948 idle=50978 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=6 addr=127.0.0.1:33162 fd=10 name= age=128128 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=info
id=7 addr=127.0.0.1:33166 fd=11 name= age=128128 idle=51 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=12 addr=127.0.0.1:41550 fd=16 name= age=128121 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=keys
id=6499 addr=127.0.0.1:58272 fd=31 name= age=62798 idle=62798 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=6530 addr=127.0.0.1:38410 fd=34 name= age=62520 idle=50996 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=6501 addr=127.0.0.1:58300 fd=32 name= age=62792 idle=50972 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=6386 addr=127.0.0.1:42948 fd=30 name= age=63893 idle=50979 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd
id=13 addr=127.0.0.1:41560 fd=17 name= age=128121 idle=5745 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=14 addr=127.0.0.1:41576 fd=18 name= age=128121 idle=128065 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=5991 addr=127.0.0.1:34198 fd=25 name= age=67806 idle=67806 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=keys
id=15 addr=127.0.0.1:41578 fd=19 name= age=128121 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=set
id=8 addr=127.0.0.1:33170 fd=12 name= age=128124 idle=5744 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zrange
id=9 addr=127.0.0.1:33174 fd=13 name= age=128124 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=zrange
id=6384 addr=127.0.0.1:37380 fd=29 name= age=63903 idle=50979 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=zadd

------ CURRENT CLIENT INFO ------
id=3 addr=127.0.0.1:33138 fd=7 name= age=128128 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=189 qbuf-free=32579 obl=0 oll=0 omem=0 events=r cmd=zadd
argv[0]: 'ZADD'
argv[1]: 'parameter_0_1_0:json:xy_alignment_procedure:last_status'
argv[2]: '1677418935741435272'
argv[3]: '�Gc�|a͈"ok: Completed(4.1s) XY alignment done, aligned: {'x': True, 'y': True}"''
7:M 26 Feb 2023 13:42:15.744 # key 'parameter_0_1_0:json:xy_alignment_procedure:last_status' found in DB containing the following object:
7:M 26 Feb 2023 13:42:15.744 # Object type: 3
7:M 26 Feb 2023 13:42:15.744 # Object encoding: 7
7:M 26 Feb 2023 13:42:15.744 # Object refcount: 1
7:M 26 Feb 2023 13:42:15.744 # Sorted set size: 88089
7:M 26 Feb 2023 13:42:15.744 # Skiplist level: 8

------ REGISTERS ------
7:M 26 Feb 2023 13:42:15.744 # 
RAX:00007fa7500ac700 RBX:0000000000002000
RCX:c753438103502070 RDX:0000000000008c04
RDI:00005560f7b656c7 RSI:2898c89f8ea2a9f7
RBP:00007fa7607ff560 RSP:00007ffe4cdc5820
R8 :00005560f7c1a290 R9 :227d65757254203a
R10:00007fa75c4ef2f3 R11:0000000000000000
R12:00007fa75c4ef2a3 R13:00007fa7607ff570
R14:41b8c8aeef9c8c04 R15:00007fa7607ff5b0
RIP:00005560f7ae987c EFL:0000000000010206
CSGSFS:002b000000000033
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc582f) -> 0000000000000000
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc582e) -> 0000000000000000
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc582d) -> 0000000000000000
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc582c) -> 0000000000000000
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc582b) -> 0000000000000001
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc582a) -> 0000000000000000
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5829) -> 000000006aa09a88
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5828) -> 43b74763a27c61ce
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5827) -> 00005560f7b19208
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5826) -> 00007fa7607a4450
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5825) -> 0000000000000000
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5824) -> 0000000000000000
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5823) -> 00007ffe4cdc5908
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5822) -> 00007fa75c4ef2a3
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5821) -> 00007ffe4cdc5904
7:M 26 Feb 2023 13:42:15.744 # (00007ffe4cdc5820) -> 0000000000000010

------ FAST MEMORY TEST ------
7:M 26 Feb 2023 13:42:15.744 # Bio thread for job type #0 terminated
7:M 26 Feb 2023 13:42:15.745 # Bio thread for job type #1 terminated
7:M 26 Feb 2023 13:42:15.745 # Bio thread for job type #2 terminated
*** Preparing to test memory region 5560f7c5e000 (2248704 bytes)
*** Preparing to test memory region 5560f9579000 (135168 bytes)
*** Preparing to test memory region 7fa74a400000 (216006656 bytes)
*** Preparing to test memory region 7fa7572fd000 (296747008 bytes)
*** Preparing to test memory region 7fa768dfe000 (8388608 bytes)
*** Preparing to test memory region 7fa7695ff000 (8388608 bytes)
*** Preparing to test memory region 7fa769e00000 (8388608 bytes)
*** Preparing to test memory region 7fa76a600000 (8388608 bytes)
*** Preparing to test memory region 7fa76af59000 (24576 bytes)
*** Preparing to test memory region 7fa76b11c000 (16384 bytes)
*** Preparing to test memory region 7fa76b13d000 (16384 bytes)
*** Preparing to test memory region 7fa76b2d3000 (8192 bytes)
*** Preparing to test memory region 7fa76b300000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------
Symbol: dictFind (base: 0x5560f7ae9820)
Module: redis-server 0.0.0.0:6379 (base 0x5560f7ab2000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x5560f7ae9820 -D -b binary -m i386:x86-64 /tmp/dump.bin
------
7:M 26 Feb 2023 13:42:16.949 # dump of function (hexdump of 220 bytes):
415741564155415455534883ec08488b4748480347280f848f00000048837f50ff4989f44889fd740b48837f58000f848c000000488b45004c89e74c8d6d104c8d7d50ff104989c6498b5510498b45004c21f2488b1cd04885db743f488b134939d47514eb47662e0f1f840000000000488b134c39e27435488b4500488b40184885c0740d488b7d084c89e6ffd085c0751b488b5b104885db75d548837d50ff74094983c5204d39ef759d31db4883c4084889d85b5d415c415d415e415fc390be01000000e8e6f5ffffe965ffffff904883ec08e827ffffff4885c0
Function at 0x5560f7ae8ed0 is dictRehash
Function at 0x5560f7ae9820 is dictFind

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

       Please report the crash by opening an issue on github:

           http://github.com/antirez/redis/issues

  Suspect RAM error? Use redis-server --test-memory to verify it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant