Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

libradiusclient - rc_send_server - Segfault with core dump #614

Closed
phil-lavin opened this issue May 11, 2016 · 7 comments
Closed

libradiusclient - rc_send_server - Segfault with core dump #614

phil-lavin opened this issue May 11, 2016 · 7 comments

Comments

@phil-lavin
Copy link
Contributor

Kamailio 4.3 fails to start after building a previously working sourcetree/config from Debian 7 onto a new Debian 8 server.

Log file is:

May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: INFO: tls [tls_domain.c:674]: set_verification(): TLSc<default>: Server MAY present invalid certificate
May 11 16:46:19 ua-proxy-00 kernel: [12060.056386] kamailio[360]: segfault at 207 ip 00007f07d23c8cbf sp 00007fff0b92bdd8 error 4
May 11 16:46:19 ua-proxy-00 kernel: [12060.056774] kamailio[358]: segfault at 207 ip 00007f07d23c8cbf sp 00007fff0b92bdd8 error 4 in libc-2.19.so[7f07d2337000+1a2000]
May 11 16:46:19 ua-proxy-00 kernel: [12060.058171]  in libc-2.19.so[7f07d2337000+1a2000]
May 11 16:46:19 ua-proxy-00 kernel: [12060.059921] kamailio[362]: segfault at 207 ip 00007f07d23c8cbf sp 00007fff0b92bdd8 error 4 in libc-2.19.so[7f07d2337000+1a2000]
May 11 16:46:19 ua-proxy-00 kernel: [12060.094333] kamailio[367]: segfault at 207 ip 00007f07d23c8cbf sp 00007fff0b92bdd8 error 4 in libc-2.19.so[7f07d2337000+1a2000]
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[623]: INFO: ctl [io_listener.c:214]: io_listen_loop(): io_listen_loop:  using epoll_lt io watch method (config)
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:728]: handle_sigs(): child process 358 exited by a signal 11
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:731]: handle_sigs(): core was not generated
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:728]: handle_sigs(): child process 360 exited by a signal 11
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:731]: handle_sigs(): core was not generated
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:728]: handle_sigs(): child process 362 exited by a signal 11
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:731]: handle_sigs(): core was not generated
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:728]: handle_sigs(): child process 367 exited by a signal 11
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: ALERT: <core> [main.c:731]: handle_sigs(): core was not generated
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[312]: INFO: <core> [main.c:743]: handle_sigs(): terminating due to SIGCHLD
May 11 16:46:19 ua-proxy-00 /opt/kamailio/sbin/kamailio[681]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received

gdb output is:

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/opt/kamailio/sbin/kamailio -f /opt/kamailio/etc/kamailio/kamailio.cfg -P /var/'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f6d5f577af6 in atomic_get (v=0x14) at ../../mem/../atomic/atomic_common.h:74
74              return atomic_get_int(&(v->val));
(gdb) bt full
#0  0x00007f6d5f577af6 in atomic_get (v=0x14) at ../../mem/../atomic/atomic_common.h:74
No locals.
#1  0x00007f6d5f578fd7 in ht_slot_lock (ht=0x7f6d61c9fad8, idx=0) at ht_api.c:150
        mypid = 17992
#2  0x00007f6d5f5719a0 in ht_db_save_table (ht=0x7f6d61c9fad8, dbtable=0x7f6d61c9faf0) at ht_db.c:443
        db_cols = {0x7f6d5f79f980 <ht_db_name_column>, 0x7f6d5f79f990 <ht_db_ktype_column>, 0x7f6d5f79f9a0 <ht_db_vtype_column>, 0x7f6d5f79f9b0 <ht_db_value_column>, 0x7f6d5f79f9c0 <ht_db_expires_column>}
        db_vals = {{type = 16777216, nul = 50, free = 351883413, val = {int_val = 4284224, ll_val = 4284224, double_val = 2.1166878974885686e-317, time_val = 4284224,
              string_val = 0x415f40 <_start> "1\355I\211\321^H\211\342H\203\344\360PTI\307\300 /n", str_val = {s = 0x415f40 <_start> "1\355I\211\321^H\211\342H\203\344\360PTI\307\300 /n", len = -1839950320}, blob_val = {
                s = 0x415f40 <_start> "1\355I\211\321^H\211\342H\203\344\360PTI\307\300 /n", len = -1839950320}, bitmap_val = 4284224}}, {type = 2455015904, nul = 32766, free = 1786366069, val = {int_val = 1803868645,
              ll_val = 140107932031461, double_val = 6.9222515926605253e-310, time_val = 140107932031461,
              string_val = 0x7f6d6b84d9e5 <db_mysql_val2str> "UH\211\345SH\203\354XH\211}\270H\211u\260H\211U\250H\211M\240H\213M\240H\213U\250H\213u\260H\213E\270H\211\307\350\372w\376\377\211E\354\203", <incomplete sequence \354>,
              str_val = {s = 0x7f6d6b84d9e5 <db_mysql_val2str> "UH\211\345SH\203\354XH\211}\270H\211u\260H\211U\250H\211M\240H\213M\240H\213U\250H\213u\260H\213E\270H\211\307\350\372w\376\377\211E\354\203", <incomplete sequence \354>,
                len = 17}, blob_val = {
                s = 0x7f6d6b84d9e5 <db_mysql_val2str> "UH\211\345SH\203\354XH\211}\270H\211u\260H\211U\250H\211M\240H\213M\240H\213U\250H\213u\260H\213E\270H\211\307\350\372w\376\377\211E\354\203", <incomplete sequence \354>,
                len = 17}, bitmap_val = 1803868645}}, {type = DB1_INT, nul = 0, free = 0, val = {int_val = 0, ll_val = 0, double_val = 0, time_val = 0, string_val = 0x0, str_val = {s = 0x0, len = 1806418832}, blob_val = {s = 0x0,
                len = 1806418832}, bitmap_val = 0}}, {type = 2455016976, nul = 32766, free = 1806462152, val = {int_val = -1839951280, ll_val = 140731353436752, double_val = 6.9530527025841608e-310, time_val = 140731353436752,
              string_val = 0x7ffe92549250 "p\222T\222\376\177", str_val = {s = 0x7ffe92549250 "p\222T\222\376\177", len = 32}, blob_val = {s = 0x7ffe92549250 "p\222T\222\376\177", len = 32}, bitmap_val = 2455016016}}, {
            type = 1803893696, nul = 32621, free = 351883413, val = {int_val = -1839951312, ll_val = 140731353436720, double_val = 6.9530527025825798e-310, time_val = 140731353436720,
              string_val = 0x7ffe92549230 "\200\222T\222\376\177", str_val = {s = 0x7ffe92549230 "\200\222T\222\376\177", len = 1803893774}, blob_val = {s = 0x7ffe92549230 "\200\222T\222\376\177", len = 1803893774},
              bitmap_val = 2455015984}}}
        it = 0x7f6d61c9faf0
        tmp = {s = 0x3214f95095 <error: Cannot access memory at address 0x3214f95095>, len = 0}
        i = 0
        now = 1462979679
        ncols = 4284224
        __FUNCTION__ = "ht_db_save_table"
#3  0x00007f6d5f58155a in ht_db_sync_tables () at ht_api.c:941
        ht = 0x7f6d61c9fad8
        __FUNCTION__ = "ht_db_sync_tables"
#4  0x00007f6d5f5876c9 in destroy () at htable.c:266
No locals.
#5  0x000000000054a1a7 in destroy_modules () at sr_module.c:805
        t = 0x7f6d6bad04b8
        foo = 0x7f6d6bacf7c0
        __FUNCTION__ = "destroy_modules"
#6  0x0000000000530ef1 in cleanup (show_status=0) at main.c:513
        memlog = 32766
        __FUNCTION__ = "cleanup"
#7  0x00000000005324d6 in shutdown_children (sig=15, show_status=0) at main.c:655
        __FUNCTION__ = "shutdown_children"
#8  0x0000000000542326 in main (argc=13, argv=0x7ffe92549618) at main.c:2568
        cfg_stream = 0x1f5d010
        c = -1
        r = 1824835416
        tmp = 0x7ffe92549e5e ""
        tmp_len = 32621
        port = 1824785872
        proto = 0
        options = 0x727dc0 ":f:cm:M:dVIhEeb:l:L:n:vKrRDTN:W:w:t:u:g:P:G:SQ:O:a:A:"
        ret = -1
        seed = 2855621674
        rfd = 4
        debug_save = 0
        debug_flag = 0
        dont_fork_cnt = 0
        n_lst = 0xf63d4e2e
        p = 0x7f6d6cc40000 ""
        st = {st_dev = 15, st_ino = 77938, st_nlink = 2, st_mode = 16877, st_uid = 1008, st_gid = 1009, __pad0 = 0, st_rdev = 0, st_size = 80, st_blksize = 4096, st_blocks = 0, st_atim = {tv_sec = 1462977193, tv_nsec = 888000000},
          st_mtim = {tv_sec = 1462979531, tv_nsec = 400000000}, st_ctim = {tv_sec = 1462979531, tv_nsec = 400000000}, __glibc_reserved = {0, 0, 0}}
        __FUNCTION__ = "main"
(gdb)

Feels like a race condition of some sort as enabling full debug logs (debug=3) actually fixes the problem...
@miconda
Copy link
Member

miconda commented May 11, 2016

This is a backtrace from shut down procedure. There must be something else stopping Kamailio. If it is at startup, look for error messages in syslog before.

@phil-lavin
Copy link
Contributor Author

Nothing relevant that I can see in the log. Here's the full thing.

Can you see anything relevant?

May 11 17:16:36 ua-proxy-00 kamailio: INFO: tls [tls_init.c:403]: init_tls_compression(): tls: init_tls: disabling compression...
May 11 17:16:36 ua-proxy-00 kamailio: INFO: <core> [sctp_core.c:75]: sctp_core_check_support(): SCTP API not enabled - if you want to use it, load sctp module
May 11 17:16:36 ua-proxy-00 kamailio: INFO: <core> [tcp_main.c:4657]: init_tcp(): using epoll_lt as the io watch method (auto detected)
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: rr [../outbound/api.h:54]: ob_load_api(): unable to import bind_ob - maybe module is not loaded
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: rr [rr_mod.c:174]: mod_init(): outbound module not available
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: usrloc [hslot.c:51]: ul_init_locks(): locks array size 1024
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_mod.c:362]: mod_init(): With ECDH-Support!
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_mod.c:365]: mod_init(): With Diffie Hellman
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: permissions [parse_config.c:251]: parse_config_file(): file not found: /opt/kamailio/etc/kamailio/permissions.allow
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: permissions [permissions.c:608]: mod_init(): default allow file (/opt/kamailio/etc/kamailio/permissions.allow) not found => empty rule set
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: permissions [parse_config.c:251]: parse_config_file(): file not found: /opt/kamailio/etc/kamailio/permissions.deny
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: permissions [permissions.c:617]: mod_init(): default deny file (/opt/kamailio/etc/kamailio/permissions.deny) not found => empty rule set
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: pua [pua.c:447]: db_restore(): the query returned no result
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: WARNING: pua_dialoginfo [pua_dialoginfo.c:637]: mod_init(): pubruri_caller_dlg_var is not set - restore on restart disabled
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: WARNING: pua_dialoginfo [pua_dialoginfo.c:640]: mod_init(): pubruri_callee_dlg_var is not set - restore on restart disabled
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: pike [pike.c:105]: pike_init(): PIKE - initializing
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: pike [ip_tree.c:81]: init_lock_set(): probing 256 set size
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: NOTICE: regex [regex_mod.c:181]: mod_init(): 'file' parameter is not set, group matching disabled
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_init.c:591]: init_tls_h(): tls: _init_tls_h:  compiled  with  openssl  version "OpenSSL 1.0.1k 8 Jan 2015" (0x100010bf), kerberos support: off, compression: on
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_init.c:599]: init_tls_h(): tls: init_tls_h: installed openssl library version "OpenSSL 1.0.1k 8 Jan 2015" (0x100010bf), kerberos support: off,  zlib compression: off#012 compiler: -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m64 -DL_ENDIAN -DTERMIO -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -Wl,-z,relro -Wa,--noexecstack -Wall -DMD32_REG_T=int -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: WARNING: tls [tls_init.c:648]: init_tls_h(): tls: openssl bug #1491 (crash/mem leaks on low memory) workarround disabled
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [cfg/cfg_ctx.c:608]: cfg_set_now(): INFO: cfg_set_now(): tls.low_mem_threshold1 has been changed to 0
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [cfg/cfg_ctx.c:608]: cfg_set_now(): INFO: cfg_set_now(): tls.low_mem_threshold2 has been changed to 0
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:150]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:200]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:150]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:200]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:150]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:200]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:150]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:200]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:150]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:200]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:150]: probe_max_receive_buffer(): SO_RCVBUF is initially 212992
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [udp_server.c:200]: probe_max_receive_buffer(): SO_RCVBUF is finally 425984
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:276]: fill_missing(): TLSs<default>: tls_method=12
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:288]: fill_missing(): TLSs<default>: certificate='/opt/kamailio/etc/kamailio/certs/synety.pem'
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:295]: fill_missing(): TLSs<default>: ca_list='/opt/kamailio/etc/kamailio/certs/ca.pem'
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:302]: fill_missing(): TLSs<default>: crl='(null)'
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:306]: fill_missing(): TLSs<default>: require_certificate=0
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:313]: fill_missing(): TLSs<default>: cipher_list='(null)'
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:320]: fill_missing(): TLSs<default>: private_key='/opt/kamailio/etc/kamailio/certs/synety.key'
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:324]: fill_missing(): TLSs<default>: verify_certificate=0
May 11 17:16:36 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:327]: fill_missing(): TLSs<default>: verify_depth=9
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:671]: set_verification(): TLSs<default>: No client certificate required and no checks performed
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:276]: fill_missing(): TLSc<default>: tls_method=12
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:288]: fill_missing(): TLSc<default>: certificate='(null)'
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:295]: fill_missing(): TLSc<default>: ca_list='(null)'
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:302]: fill_missing(): TLSc<default>: crl='(null)'
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:306]: fill_missing(): TLSc<default>: require_certificate=0
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:313]: fill_missing(): TLSc<default>: cipher_list='(null)'
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:320]: fill_missing(): TLSc<default>: private_key='(null)'
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:324]: fill_missing(): TLSc<default>: verify_certificate=0
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:327]: fill_missing(): TLSc<default>: verify_depth=9
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: tls [tls_domain.c:674]: set_verification(): TLSc<default>: Server MAY present invalid certificate
May 11 17:16:37 ua-proxy-00 kernel: [13878.510359] kamailio[8835]: segfault at 207 ip 00007f411749ccbf sp 00007ffc74a09ed8 error 4 in libc-2.19.so[7f411740b000+1a2000]
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[9082]: INFO: ctl [io_listener.c:214]: io_listen_loop(): io_listen_loop:  using epoll_lt io watch method (config)
May 11 17:16:37 ua-proxy-00 kernel: [13878.560866] kamailio[8839]: segfault at 207 ip 00007f411749ccbf sp 00007ffc74a09ed8 error 4 in libc-2.19.so[7f411740b000+1a2000]
May 11 17:16:37 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: ALERT: <core> [main.c:728]: handle_sigs(): child process 8835 exited by a signal 11
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: ALERT: <core> [main.c:731]: handle_sigs(): core was not generated
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9148]: CRITICAL: <core> [pass_fd.c:275]: receive_fd(): EOF on 58
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9148]: CRITICAL: <core> [pass_fd.c:275]: receive_fd(): EOF on 60
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: ALERT: <core> [main.c:728]: handle_sigs(): child process 8839 exited by a signal 11
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: ALERT: <core> [main.c:731]: handle_sigs(): core was not generated
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [main.c:743]: handle_sigs(): terminating due to SIGCHLD
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9096]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9136]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9061]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9138]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8802]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9140]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9067]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9148]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9142]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9071]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9144]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8795]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9120]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9077]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8797]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9122]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8798]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9084]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9124]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8801]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9019]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9090]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9126]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8803]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9128]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8806]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9053]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9072]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8921]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9146]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9130]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8814]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9132]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8817]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8800]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9134]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8820]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9108]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8824]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8828]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9110]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9112]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8832]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8836]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9114]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9116]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8840]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8844]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9118]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8848]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9098]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8852]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9100]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9085]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8856]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9102]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9106]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9104]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8860]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9088]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8864]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9094]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8927]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8872]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9082]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8876]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8868]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8886]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8890]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9049]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8894]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8883]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8880]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9045]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9057]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9041]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9037]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9031]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9027]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9020]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9023]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9075]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9013]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9009]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9078]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8898]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9062]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9065]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8902]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9068]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9005]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8906]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8910]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9058]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9001]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9054]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8914]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8997]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9050]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8918]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9046]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8928]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8993]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9042]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8932]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8989]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9038]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8936]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9035]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8940]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8983]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9032]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8979]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9028]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9006]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8975]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9024]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8796]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9010]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9002]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8971]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8998]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8990]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9014]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8967]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8994]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8963]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9017]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8799]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8959]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8805]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8987]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8809]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8984]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8955]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8813]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8980]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8819]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8951]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8976]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8823]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8972]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8947]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8827]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8968]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8943]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8960]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8905]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8939]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8956]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8909]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8952]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8901]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8964]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8948]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8913]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8897]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8944]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8917]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8924]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8893]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8922]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8931]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8889]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8935]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8885]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8831]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8875]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8810]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8879]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8863]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8871]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8843]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8855]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8847]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[9092]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8867]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8851]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8859]: INFO: <core> [main.c:798]: sig_usr(): signal 15 received
May 11 17:16:38 ua-proxy-00 /opt/kamailio/sbin/kamailio[8785]: INFO: <core> [sctp_core.c:53]: sctp_core_destroy(): SCTP API not initialized

@miconda
Copy link
Member

miconda commented May 11, 2016

You should have many core files -- enable corefile per process and take the backtrace from each of them.

@phil-lavin
Copy link
Contributor Author

Ah, thanks Daniel. This is more useful.

#0  __strncpy_sse2 (s1=0x7ffc980a968f "", s2=0x208 <error: Cannot access memory at address 0x208>, n=48) at ./strncpy.c:41
        n4 = 12
        c = <optimized out>
        s = 0x7ffc980a9690 ""
#1  0x00007f0ffc54bbc9 in rc_send_server () from /usr/lib/x86_64-linux-gnu/libfreeradius-client.so.2
No symbol table info available.
#2  0x00007f0ffc54ad9c in rc_aaa () from /usr/lib/x86_64-linux-gnu/libfreeradius-client.so.2
No symbol table info available.
#3  0x00007f0ffc54b036 in rc_auth () from /usr/lib/x86_64-linux-gnu/libfreeradius-client.so.2
No symbol table info available.
#4  0x00007f0ffc760071 in radius_authorize_sterman (_msg=0x7f1006ab7360, _cred=0x7f1006a3a040, _method=0x7f1006ab7388, _user=0x7ffc980ad980) at sterman.c:395
        msg = '\000' <repeats 4095 times>
        send = 0x222fb30
        received = 0x0
        service = 15
        method = {
          s = 0xa18800 <buf> "REGISTER sip:register.staging.cloudcall.com SIP/2.0\r\nVia: SIP/2.0/UDP 172.31.17.79:1024;branch=z9hG4bK-5yotxhq2gic2;rport\r\nFrom: \"Staging 1050\" <sip:441164961050@register.staging.cloudcall.com>;tag=kh"..., len = 8}
        user = {s = 0x7f1006a3a178 "441164961050", len = 12}
        user_name = {s = 0xd4fd0 <error: Cannot access memory at address 0xd4fd0>, len = 1328360}
        ruri = 0x7f1006a3a090
        extra_cnt = 2
        offset = 31
        i = 2
        __FUNCTION__ = "radius_authorize_sterman"
#5  0x00007f0ffc763288 in authorize (_msg=0x7f1006ab7360, _realm=0x7f1006a531f8, _uri_user=0x0, _hftype=HDR_AUTHORIZATION_T) at authorize.c:175
        res = 32764
        ret = NOT_AUTHENTICATED
        h = 0x7f1006a6c8a8
        cred = 0x7f1006a3a038
        uri_user = 0x7f1006ac6818
        user = {s = 0x7f1006a3a178 "441164961050", len = 12}
        domain = {s = 0x7f10069cec78 "register.staging.cloudcall.com", len = 30}
        pv_val = {rs = {s = 0x0, len = 1242592893}, ri = 4284224, flags = 0}
        __FUNCTION__ = "authorize"
#6  0x00007f0ffc763a67 in radius_www_authorize_1 (_msg=0x7f1006ab7360, _realm=0x7f1006a531f8 "\250\366\245\006\020\177", _s2=0x0) at authorize.c:242
No locals.
#7  0x00000000005883d9 in do_action (h=0x7ffc980ae080, a=0x7f1006a5f480, msg=0x7f1006ab7360) at action.c:1059

@phil-lavin
Copy link
Contributor Author

phil-lavin commented May 11, 2016 •
edited

Looks like a bug with the Debian 8 libfreeradius-client-dev package... or at least how Kamailio works with it. Compiling the latest from https://github.com/FreeRADIUS/freeradius-client has fixed the problem.

I'm going to do some more digging.

@phil-lavin phil-lavin changed the title Segfault with core dump libradiusclient - rc_send_server - Segfault with core dump May 11, 2016
@miconda
Copy link
Member

miconda commented May 11, 2016

OK -- thanks for following up, let us know if you find something new. There were no relevant changes in our code related to radius modules.

@phil-lavin
Copy link
Contributor Author

I downloaded the source package from the Debian repo and compiled with symbols. The error traced back to an incorrect Radius client configuration. I've done some more testing by removing some random directives from the config and it's surprisingly easy to make it segfault.

Poor error handling from libradiusclient... but not a Kamailio issue.

Thanks for the help :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@miconda @phil-lavin