Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kamailio 5.2.3(commit c36229) crash at free_cell_helper #3379

Closed
binbin6363 opened this issue Feb 24, 2023 · 1 comment
Closed

kamailio 5.2.3(commit c36229) crash at free_cell_helper #3379

binbin6363 opened this issue Feb 24, 2023 · 1 comment

Comments

@binbin6363
Copy link

Description

kamailio 5.2.3 crash at free_cell_helper

Troubleshooting

Reproduction

This cannot be reproduced at will.

Debugging Data

[root@VM-110-141-centos /data/coredump]# gdb -c core_kamailio_1676550458 /usr/local/services/kamailio/sbin/kamailio
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.tl2
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /usr/local/services/kamailio/sbin/kamailio...done.
[New LWP 4452]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Missing separate debuginfo for /usr/local/services/kamailio/lib64/kamailio/libjansson.so.4
Try: yum --enablerepo='*debug*' install /usr/lib/debug/.build-id/fc/cf02c89115bf860ddba32e8b703205ffeeabf1.debug
Core was generated by `/usr/local/services/kamailio/sbin/kamailio -f /usr/local/services/kamailio/etc/'.
Program terminated with signal 11, Segmentation fault.
#0  0x00007f8ede8e113a in free_cell_helper (dead_cell=0x7f8eb7dcb1b8, silent=0, fname=0x7f8ede9ece02 "timer.c", fline=689) at h_table.c:182
182     h_table.c: No such file or directory.
Missing separate debuginfos, use: debuginfo-install bash-4.2.46-34.tl2.3.x86_64 cyrus-sasl-lib-2.1.26-23.el7.x86_64 glibc-2.17-323.tl2.x86_64 keyutils-libs-1.5.8-3.el7.x86_64 krb5-libs-1.15.1-51.tl2.x86_64 libcom_err-1.42.9-19.tl2.x86_64 libcurl-7.29.0-59.tl2.1.x86_64 libevent-2.0.21-4.el7.x86_64 libgcc-4.8.5-44.tl2.1.x86_64 libidn-1.28-4.el7.x86_64 libselinux-2.5-15.tl2.x86_64 libssh2-1.8.0-4.tl2.x86_64 libstdc++-4.8.5-44.tl2.1.x86_64 mariadb-libs-5.5.68-1.tl2.x86_64 nspr-4.32.0-1.tl2.x86_64 nss-3.67.0-4.tl2.x86_64 nss-softokn-freebl-3.67.0-3.tl2.x86_64 nss-util-3.67.0-1.tl2.x86_64 openldap-2.4.44-24.tl2.x86_64 openssl-libs-1.0.2k-22.tl2.1.x86_64 pcre-8.32-17.el7.x86_64 python-libs-2.7.5-90.tl2.1.x86_64 zlib-1.2.7-19.tl2.x86_64
(gdb) bt full
#0  0x00007f8ede8e113a in free_cell_helper (dead_cell=0x7f8eb7dcb1b8, silent=0, fname=0x7f8ede9ece02 "timer.c", fline=689) at h_table.c:182
        b = 0x7f8ede8decc4 <futex_release+29> "\211E\374\203}\374\002\017\224\300\017\266\300H\205\300t3H\213E\350\307\004$"
        i = 1
        rpl = 0x7fffb5166580
        tt = 0x7fffb51665a0
        foo = 0x100000000
        cbs = 0x7f8e00000001
        cbs_tmp = 0x7f8e00000001
        __FUNCTION__ = "free_cell_helper"
#1  0x00007f8ede99381e in wait_handler (ti=367281959, wait_tl=0x7f8eb7dcb240, data=0x7f8eb7dcb1b8) at timer.c:689
        p_cell = 0x7f8eb7dcb1b8
        ret = 3080640456
        unlinked = 0
        rcount = 1
        __FUNCTION__ = "wait_handler"
#2  0x0000000000487769 in timer_list_expire (t=367281959, h=0x7f8eb7a318a8, slow_l=0x7f8eb7a34d70, slow_mark=42793) at core/timer.c:874
        tl = 0x7f8eb7dcb240
        ret = 0
#3  0x0000000000487bca in timer_handler () at core/timer.c:939
        saved_ticks = 367281959
        run_slow_timer = 0
        i = 809
        __FUNCTION__ = "timer_handler"
#4  0x0000000000488074 in timer_main () at core/timer.c:978
No locals.
#5  0x0000000000425a31 in main_loop () at main.c:1727
        i = 8
        pid = 0
        si = 0x0
        si_desc = "udp receiver child=7 sock=11.177.110.141:35560\000(175.27.32.249:15060)\000\177\000\000\223U{\000\000\000\000\000-\000\000\000\000\000\000\000\000\000\000 \000\000\000\000\000\000\000\004\000\000\000\000\257*\245\346\216\177\000\000\360\260y\000\000\000\000\000\000\227\265\342\216\177\000"
        nrprocs = 8
        woneinit = 1
        __FUNCTION__ = "main_loop"
#6  0x000000000042c72d in main (argc=13, argv=0x7fffb5166bd8) at main.c:2696
        cfg_stream = 0xfdc010
        c = -1
        r = 0
        tmp = 0x7fffb5168757 ""
        tmp_len = 0
        port = 0
        proto = 2496
        options = 0x76a700 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
        ret = -1
        seed = 99772630
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 0
        n_lst = 0x7fffb5166a90
        p = 0x7f8ee6a00d2d <intel_check_word+173> "H\205\300t\025\017\266P\003\017\266\312D9\371\017\204\316"
        st = {st_dev = 21, st_ino = 32759713, st_nlink = 2, st_mode = 16877, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1671615993, 
            tv_nsec = 68240759}, st_mtim = {tv_sec = 1671620803, tv_nsec = 306084978}, st_ctim = {tv_sec = 1671620803, tv_nsec = 306084978}, __unused = {0, 0, 0}}
        __FUNCTION__ = "main"
(gdb) info locals
cfg_stream = 0xfdc010
c = -1
r = 0
tmp = 0x7fffb5168757 ""
tmp_len = 0
port = 0
proto = 2496
options = 0x76a700 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:x:X:Y:"
ret = -1
seed = 99772630
rfd = 4
debug_save = 0
debug_flag = 0
dont_fork_cnt = 0
n_lst = 0x7fffb5166a90
p = 0x7f8ee6a00d2d <intel_check_word+173> "H\205\300t\025\017\266P\003\017\266\312D9\371\017\204\316"
st = {st_dev = 21, st_ino = 32759713, st_nlink = 2, st_mode = 16877, st_uid = 0, st_gid = 0, __pad0 = 0, st_rdev = 0, st_size = 40, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1671615993, 
    tv_nsec = 68240759}, st_mtim = {tv_sec = 1671620803, tv_nsec = 306084978}, st_ctim = {tv_sec = 1671620803, tv_nsec = 306084978}, __unused = {0, 0, 0}}
__FUNCTION__ = "main"
(gdb) list
177     in h_table.c
(gdb)

Log Messages

kamailio.log

Feb 16 20:27:33 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4434]: INFO: {10 OPTIONS 3c7f4fb43ea4086d-4451@11.177.110.141} <script>: keepalive-host is up: OPTIONS sip:9.137.223.152:50180 (<null>)
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4425]: ALERT: <core> [main.c:756]: handle_sigs(): child process 4452 exited by a signal 11
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4425]: ALERT: <core> [main.c:759]: handle_sigs(): core was generated
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4425]: INFO: <core> [main.c:781]: handle_sigs(): terminating due to SIGCHLD
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4470]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4469]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4462]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4466]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4467]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4468]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4464]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4449]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4460]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4465]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4461]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4440]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4459]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4429]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4457]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4430]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4456]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4442]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4432]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4458]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4437]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4444]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4463]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4439]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4454]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4434]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4448]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4451]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4436]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4447]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4450]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4435]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4431]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4441]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4446]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4428]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4445]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4427]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4453]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4438]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4455]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4433]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4443]: INFO: <core> [main.c:836]: sig_usr(): signal 15 received
Feb 16 20:27:41 VM-110-141-centos /usr/local/services/kamailio/sbin/kamailio[4425]: CRITICAL: <core> [core/mem/q_malloc.c:137]: qm_debug_check_frag(): BUG: qm: fragm. 0x7f8eb7d04778 (address 0x7f8eb7d047b0) end overwritten (1, abcdefed)! Memory allocator was called from dialplan: dp_db.c:687. Fragment marked by dialplan: dp_db.c:400. Exec from core/mem/q_malloc.c:504.

messages

[Thu Feb 16 20:27:44 2023] kamailio[4452]: segfault at 7f8e00000021 ip 00007f8ede8e113a sp 00007fffb51664a0 error 4 in tm.so[7f8ede8c5000+13a000]

SIP Traffic

Possible Solutions

Additional Information

  • Kamailio Version - output of kamailio -v
[root@VM-110-141-centos /usr/local/services/kamailio/sbin]# ./kamailio -version
version: kamailio 5.2.3 (x86_64/linux) c36229
flags: STATS: Off, USE_TCP, USE_TLS, USE_SCTP, TLS_HOOKS, USE_RAW_SOCKS, DISABLE_NAGLE, USE_MCAST, DNS_IP_HACK, SHM_MEM, SHM_MMAP, PKG_MALLOC, Q_MALLOC, F_MALLOC, TLSF_MALLOC, DBG_SR_MEMORY, USE_FUTEX, FAST_LOCK-ADAPTIVE_WAIT, USE_DNS_CACHE, USE_DNS_FAILOVER, USE_NAPTR, USE_DST_BLACKLIST, HAVE_RESOLV_RES
ADAPTIVE_WAIT_LOOPS=1024, MAX_RECV_BUFFER_SIZE 262144 MAX_URI_SIZE 1024, BUF_SIZE 65535, DEFAULT PKG_SIZE 8MB
poll method support: poll, epoll_lt, epoll_et, sigio_rt, select.
id: c36229 
compiled on 16:50:57 Jul 16 2019 with gcc 4.8.5
  • Operating System:
[root@VM-110-141-centos /data/coredump]# uname -a
Linux VM-110-141-centos 4.14.105-1-tlinux3-0023 #1 SMP Mon Jan 10 18:17:33 CST 2022 x86_64 x86_64 x86_64 GNU/Linux
@henningw
Copy link
Contributor

Thanks for the report. The Kamailio version you use is long end of life. Please update to a supported version, like one of the latest 5.5.x or 5.6.x versions.

@henningw henningw closed this as completed Mar 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@henningw @binbin6363