This repository has been archived by the owner on Jan 24, 2023. It is now read-only.
Redis 5.0.3 crashed by signal: 11 (Redistimeseries.so with compaction policy set) #79
Labels
bug
Something isn't working
Comments
|
@udayshankarv can you open it on https://github.com/RedisTimeSeries/RedisTimeSeries ? this repo is used only for the python client. |
|
Also, @udayshankarv do you know which version are you using? |
|
thank you @danni-m , Redis server v=5.0.3 Name: redistimeseries |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Redis frequently crashing when compaction policy is enabled.
loadmodule /opt/redis/redistimeseries.so DUPLICATE_POLICY Last COMPACTION_POLICY sum:300s:90d;count:300s:90d;sum:1800s:180d;count:1800s:180d
When the module is loaded without COMPACTION_POLICY, It is a seamless.
=== REDIS BUG REPORT START: Cut & paste starting from here ===
14861:M 29 Oct 2020 07:45:30.469 # Redis 5.0.3 crashed by signal: 11
14861:M 29 Oct 2020 07:45:30.469 # Crashed running the instruction at: 0x7fa241f7cb05
14861:M 29 Oct 2020 07:45:30.469 # Accessing address: (nil)
14861:M 29 Oct 2020 07:45:30.469 # Failed assertion: (:0)
------ STACK TRACE ------
EIP:
/opt/redis/redistimeseries.so(Uncompressed_UpsertSample+0x125)[0x7fa241f7cb05]
Backtrace:
/usr/bin/redis-server 0.0.0.0:6379(logStackTrace+0x44)[0x55df2b955c54]
/usr/bin/redis-server 0.0.0.0:6379(sigsegvHandler+0xb5)[0x55df2b9563d5]
/lib64/libpthread.so.0(+0x12dd0)[0x7fa2413e5dd0]
/opt/redis/redistimeseries.so(Uncompressed_UpsertSample+0x125)[0x7fa241f7cb05]
/opt/redis/redistimeseries.so(SeriesUpsertSample+0xda)[0x7fa241f85ada]
/opt/redis/redistimeseries.so(SeriesUpsertSample+0x3a8)[0x7fa241f85da8]
/opt/redis/redistimeseries.so(+0xcad2)[0x7fa241f80ad2]
/opt/redis/redistimeseries.so(TSDB_add+0x162)[0x7fa241f82d42]
/usr/bin/redis-server 0.0.0.0:6379(RedisModuleCommandDispatcher+0x66)[0x55df2b983946]
/usr/bin/redis-server 0.0.0.0:6379(call+0x93)[0x55df2b90e853]
/usr/bin/redis-server 0.0.0.0:6379(processCommand+0x49e)[0x55df2b90fcfe]
/usr/bin/redis-server 0.0.0.0:6379(processInputBuffer+0x181)[0x55df2b91fb01]
/usr/bin/redis-server 0.0.0.0:6379(aeProcessEvents+0x119)[0x55df2b9081f9]
/usr/bin/redis-server 0.0.0.0:6379(aeMain+0x2b)[0x55df2b90865b]
/usr/bin/redis-server 0.0.0.0:6379(main+0x3de)[0x55df2b90523e]
/lib64/libc.so.6(__libc_start_main+0xf3)[0x7fa2410346a3]
/usr/bin/redis-server 0.0.0.0:6379(_start+0x2e)[0x55df2b90558e]
------ INFO OUTPUT ------
Server
redis_version:5.0.3
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:28849dbea6f07cc8
redis_mode:standalone
os:Linux 4.18.0-80.el8.x86_64 x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:8.3.1
process_id:14861
run_id:4dfba04838528974a4243b5231d8363e1c62a677
tcp_port:6379
uptime_in_seconds:1
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:10136410
executable:/usr/bin/redis-server
config_file:/etc/redis.conf
Clients
connected_clients:1
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
Memory
used_memory:30728624
used_memory_human:29.31M
used_memory_rss:36397056
used_memory_rss_human:34.71M
used_memory_peak:30728624
used_memory_peak_human:29.31M
used_memory_peak_perc:100.56%
used_memory_overhead:1120656
used_memory_startup:791304
used_memory_dataset:29607968
used_memory_dataset_perc:98.90%
allocator_allocated:30647824
allocator_active:30879744
allocator_resident:36335616
total_system_memory:8193073152
total_system_memory_human:7.63G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.01
allocator_frag_bytes:231920
allocator_rss_ratio:1.18
allocator_rss_bytes:5455872
rss_overhead_ratio:1.00
rss_overhead_bytes:61440
mem_fragmentation_ratio:1.19
mem_fragmentation_bytes:5838216
mem_not_counted_for_evict:522
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:49694
mem_aof_buffer:522
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
Persistence
loading:0
rdb_changes_since_last_save:12046
rdb_bgsave_in_progress:0
rdb_last_save_time:1603971929
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
aof_current_size:2840711
aof_base_size:2839964
aof_pending_rewrite:0
aof_buffer_length:0
aof_rewrite_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0
Stats
total_connections_received:1
total_commands_processed:4
instantaneous_ops_per_sec:0
total_net_input_bytes:969
total_net_output_bytes:48
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
evicted_keys:0
keyspace_hits:9
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
Replication
role:master
connected_slaves:0
master_replid:7cd9b1523a1994fde5ed6227465506d441c0a7bd
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
CPU
used_cpu_sys:0.021736
used_cpu_user:0.213450
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
Commandstats
cmdstat_info:calls=1,usec=18,usec_per_call=18.00
cmdstat_ts.add:calls=3,usec=494,usec_per_call=164.67
Cluster
cluster_enabled:0
Keyspace
db0:keys=5340,expires=0,avg_ttl=0
------ CLIENT LIST OUTPUT ------
id=10913 addr=172.17.21.2:59224 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=245 qbuf-free=32523 obl=0 oll=0 omem=0 events=r cmd=ts.add
------ CURRENT CLIENT INFO ------
id=10913 addr=172.17.21.2:59224 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=245 qbuf-free=32523 obl=0 oll=0 omem=0 events=r cmd=ts.add
argv[0]: 'TS.ADD'
argv[1]: 'billing:2088'
argv[2]: '1602565827000'
argv[3]: '1'
argv[4]: 'RETENTION'
argv[5]: '86400000'
argv[6]: 'LABELS'
argv[7]: 'agentname'
argv[8]: 'Laura Jackson'
argv[9]: 'agentskillname'
argv[10]: 'SK1589'
argv[11]: 'region'
argv[12]: 'N. Virginia'
argv[13]: 'server'
argv[14]: 'Server1'
argv[15]: 'shift'
argv[16]: '1'
14861:M 29 Oct 2020 07:45:30.470 # key 'billing:2088' found in DB containing the following object:
14861:M 29 Oct 2020 07:45:30.470 # Object type: 5
14861:M 29 Oct 2020 07:45:30.470 # Object encoding: 0
14861:M 29 Oct 2020 07:45:30.470 # Object refcount: 1
------ REGISTERS ------
14861:M 29 Oct 2020 07:45:30.470 #
RAX:00007fa23d660380 RBX:0000000000000000
RCX:00007fa240000000 RDX:0000000000000002
RDI:00007fffc394abf0 RSI:00007fffc394abcc
RBP:00007fa23d660380 RSP:00007fffc394ab70
R8 :00000175205d8040 R9 :0000000000000000
R10:00007fa241f8f838 R11:00007fa24119df20
R12:0000000000000002 R13:00000175205d8040
R14:ffffffffffffffff R15:00007fffc394ac88
RIP:00007fa241f7cb05 EFL:0000000000010246
CSGSFS:002b000000000033
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab7f) -> 000055df2b982105
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab7e) -> 00007fa23d75e150
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab7d) -> 000055df2c5746b0
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab7c) -> 00007fa23d660380
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab7b) -> 000000002b915e7a
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab7a) -> 0000000000000078
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab79) -> 00007fa240c16100
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab78) -> 0000000000000001
14861:M 29 Oct 2020 07:45:30.470 # (00007fffc394ab77) -> 3ff0000000000000
14861:M 29 Oct 2020 07:45:30.471 # (00007fffc394ab76) -> 0000000000000078
14861:M 29 Oct 2020 07:45:30.471 # (00007fffc394ab75) -> 00007fa241f85ada
14861:M 29 Oct 2020 07:45:30.471 # (00007fffc394ab74) -> ffffffffffffffff
14861:M 29 Oct 2020 07:45:30.471 # (00007fffc394ab73) -> 00000175205d8040
14861:M 29 Oct 2020 07:45:30.471 # (00007fffc394ab72) -> 0000000000000002
14861:M 29 Oct 2020 07:45:30.471 # (00007fffc394ab71) -> 00007fa241f8e780
14861:M 29 Oct 2020 07:45:30.471 # (00007fffc394ab70) -> 000055df2c5747f0
------ FAST MEMORY TEST ------
14861:M 29 Oct 2020 07:45:30.471 # Bio thread for job type #0 terminated
14861:M 29 Oct 2020 07:45:30.471 # Bio thread for job type #1 terminated
14861:M 29 Oct 2020 07:45:30.471 # Bio thread for job type #2 terminated
*** Preparing to test memory region 55df2bc82000 (2252800 bytes)
*** Preparing to test memory region 55df2c496000 (1622016 bytes)
*** Preparing to test memory region 7fa23c400000 (2097152 bytes)
*** Preparing to test memory region 7fa23c780000 (36175872 bytes)
*** Preparing to test memory region 7fa23eb05000 (2621440 bytes)
*** Preparing to test memory region 7fa23ed86000 (8388608 bytes)
*** Preparing to test memory region 7fa23f587000 (8388608 bytes)
*** Preparing to test memory region 7fa23fd88000 (8388608 bytes)
*** Preparing to test memory region 7fa240800000 (8388608 bytes)
*** Preparing to test memory region 7fa2413cf000 (16384 bytes)
*** Preparing to test memory region 7fa2415ef000 (16384 bytes)
*** Preparing to test memory region 7fa241f8f000 (32768 bytes)
*** Preparing to test memory region 7fa241fac000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
------ DUMPING CODE AROUND EIP ------
Symbol: Uncompressed_UpsertSample (base: 0x7fa241f7c9e0)
Module: /opt/redis/redistimeseries.so (base 0x7fa241f74000)
$ xxd -r -p /tmp/dump.hex /tmp/dump.bin
$ objdump --adjust-vma=0x7fa241f7c9e0 -D -b binary -m i386:x86-64 /tmp/dump.bin
14861:M 29 Oct 2020 07:45:30.765 # dump of function (hexdump of 421 bytes):
4156415541545553488b6f10c706000000004c8b07448b4d10490fbfd94885db0f84ff000000488b450831d2488b084939c87720e9f60000000f1f80000000004863ca48c1e104488b0c084c39c10f837c0000004883c2014839da75e34989dd49c1e5044c39c174764989f6488b75184489ca4989fc4889f148c1e9044839ca751e4889c7488b05741101004883c61048897518ff108b5510488945084989d14a8d34284839da7757f3410f6f0c244183c10131c00f110e44894d1041c706010000005b5d415c415d415ec30f1f40004989d54889d349c1e5044c39c1758a4c01e8f20f104708f20f11400831c05b5d415c415d415ec3660f1f840000000000488d7b014829da48c1e70448c1e204488d3c38e8d8faffff488b7508448b4d104c01eeeb84488b0425000000000f0b74b14c8945004531ed31dbe92affffff9041554989fdbf0100000041544989d4555389f3be200000004883ec08488b052d100100ff104889c54c892889581c83e3017407418b5d1083eb01895d084d85e4741931ffe867f9fffff30f6f00410f110424488b401049894424104883c4084889e85b5d41
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
The text was updated successfully, but these errors were encountered: