Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

issue notifying subscribers with transport tcp/tls #11

Closed
lazedo opened this issue Dec 20, 2014 · 14 comments
Closed

issue notifying subscribers with transport tcp/tls #11

lazedo opened this issue Dec 20, 2014 · 14 comments
Labels

Comments

@lazedo
Copy link
Contributor

lazedo commented Dec 20, 2014

Hi,

i noticed a weird behavior in 4.2 branch with this kamailio.cfg (snippet)

!substdef "!UDP_SIP!udp:MY_IP_ADDRESS:5060!g"

!substdef "!TCP_SIP!tcp:MY_IP_ADDRESS:5060!g"

listen=UDP_SIP
listen=TCP_SIP

# Routing Logic

route
{
# log the basic info regarding this call
xlog("L_INFO", "$ci|start|recieved $oP request $rm $ou");
.....
}

when clients using TCP send a SUBSCRIBE we get log info that it is a UDP operation.
when we send the first notify we get tcp errors.
for reference, i'm using Bria 4 in testing.

Dec 20 16:49:29 kamailio-05 kamailio[5085]: INFO: <script>: YzYwYWVlOTFiMGU0YmNjM2IyMDA1NmQ4NjdiMGEwZDY|start|recieved UDP request SUBSCRIBE sip:2135@sip.com
Dec 20 16:49:29 kamailio-05 kamailio[5072]: INFO: <script>: received dialog update for sip:2135@sip.com
Dec 20 16:49:29 kamailio-05 kamailio[5072]: INFO: presence [notify.c:1604]: send_notify_request(): NOTIFY sip:user_7z7b6s@sip.com via sip:user_7z7b6s@192.168.16.12:62285;transpop on behalf of sip:2134@sip.com for event dialog
Dec 20 16:49:29 kamailio-05 kamailio[5072]: CRITICAL: <core> [pass_fd.c:157]: send_all(): send on 0 failed: Socket operation on non-socket
Dec 20 16:49:29 kamailio-05 kamailio[5072]: ERROR: <core> [tcp_main.c:2303]: tcpconn_send_put(): failed to get fd(write):Socket operation on non-socket (88)
Dec 20 16:49:29 kamailio-05 kamailio[5072]: ERROR: tm [../../forward.h:227]: msg_send(): tcp_send failed
Dec 20 16:49:29 kamailio-05 kamailio[5072]: ERROR: tm [uac.c:598]: send_prepared_request_impl(): t_uac: Attempt to send to precreated request failed

@lazedo
Copy link
Contributor Author

lazedo commented Dec 20, 2014

the same thing happens with TLS connections.

one side effect is that the subscription doesn't get updated to active and gets cleaned by timer proc.

Dec 20 18:20:32 kamailio-05 kamailio[6089]: INFO: <script>: received dialog update for sip:2134@sip.com
Dec 20 18:20:32 kamailio-05 kamailio[6089]: CRITICAL: <core> [pass_fd.c:157]: send_all(): send on 0 failed: Socket operation on non-socket
Dec 20 18:20:32 kamailio-05 kamailio[6089]: ERROR: <core> [tcp_main.c:2303]: tcpconn_send_put(): failed to get fd(write):Socket operation on non-socket (88)
Dec 20 18:20:32 kamailio-05 kamailio[6089]: ERROR: tm [../../forward.h:247]: msg_send(): tcp_send failed
Dec 20 18:20:32 kamailio-05 kamailio[6089]: ERROR: tm [uac.c:598]: send_prepared_request_impl(): t_uac: Attempt to send to precreated request failed
Dec 20 18:20:32 kamailio-05 kamailio[6089]: INFO: presence [notify.c:1604]: send_notify_request(): NOTIFY sip:user_7z7b6s@sip.com via sip:user_7z7b6s@192.168.16.12:15778;transport=tls on behalf of sip:2134@sip.com for event dialog

@miconda
Copy link
Member

miconda commented Dec 22, 2014

$oP is the protocol specified in the transport parameter of original R-URI. If you want to take the value from the ip layer, use $pr.

You have to paste here the error messages you get in order to be able to assist further.

@lazedo
Copy link
Contributor Author

lazedo commented Dec 22, 2014

thanks about the $pr.
the error is in the message (it doesn't show but if you try to edit you'll see it).

@linuxmaniac
Copy link
Member

JFTR, I edited @lazedo comments using markdown syntax in order to see the logs. Just my 2 cents

@lazedo lazedo changed the title wrong determination of udp/tcp request issue notifying subscribers with transport tcp/tls Dec 22, 2014
@miconda
Copy link
Member

miconda commented Dec 22, 2014

Can you get the ngrep for the SUBSCRIBE as received by Kamailio? Apparently something is related to what is forced as outgoing protocol.

@lazedo
Copy link
Contributor Author

lazedo commented Dec 22, 2014

@miconda
Copy link
Member

miconda commented Dec 22, 2014

It doesn't look like ngrep output. Better attach the pcap than a long paste from wireshark that is hard to spot the requests and analyze.

@kamailio-sync
Copy link

pcap file


From: sr-dev [sr-dev-bounces@lists.sip-router.org] on behalf of Daniel-Constantin Mierla [notifications@github.com]
Sent: Monday, December 22, 2014 3:09 PM
To: Kamailio Devel List
Subject: Re: [sr-dev] [kamailio] issue notifying subscribers with transport tcp/tls (#11)

It doesn't look like ngrep output. Better attach the pcap than a long paste from wireshark that is hard to spot the requests and analyze.


Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-67902787.

@miconda
Copy link
Member

miconda commented Dec 22, 2014

The error message is about inter-process communication in Kamailio, not with the network. Do you see the NOTIFY on the net? If you do 'kamctl ps', do you see tcp workers and tcp main attendant processes?

I would need all logs messages with debug=3 to be able to troubleshoot further.

@kamailio-sync
Copy link

Hi,

the notify_watchers is being called from a background process that was forked like this pid=fork_process(i+2, "AMQP Consumer", 0);
could this be the problem ? should it be pid=fork_process(i+2, "AMQP Consumer", 1); ?

also, is there a reason for send_notify_request in presence/notify.c to use tmb.t_request_within instead of tmb.t_request?

set_uac_req(&uac_r, &met, &str_hdr, notify_body, td, TMCB_LOCAL_COMPLETED,
p_tm_callback, (void*)cb_param);
result = tmb.t_request_within(&uac_r);


From: sr-dev [sr-dev-bounces@lists.sip-router.org] on behalf of Daniel-Constantin Mierla [notifications@github.com]
Sent: Monday, December 22, 2014 3:42 PM
To: Kamailio Devel List
Cc: kamailio-sync
Subject: Re: [sr-dev] [kamailio] issue notifying subscribers with transport tcp/tls (#11)

The error message is about inter-process communication in Kamailio, not with the network. Do you see the NOTIFY on the net? If you do 'kamctl ps', do you see tcp workers and tcp main attendant processes?

I would need all logs messages with debug=3 to be able to troubleshoot further.


Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-67905766.

@miconda
Copy link
Member

miconda commented Dec 23, 2014

Yes, you have to set last parameter to 1 for fork_process().

t_request_within() sends a request within an existing dialog (subscribe dialog in this case).

@lazedo
Copy link
Contributor Author

lazedo commented Dec 23, 2014

Hi Daniel,

what about when there isn't a subscribe dialog ? or does the SUBSCRIBE keep
a reference to the dialog ? in this case the NOTIFY is sent when other
extension is ringing (BLF).

thanks

On Tue, Dec 23, 2014 at 9:07 AM, Daniel-Constantin Mierla <
notifications@github.com> wrote:

Yes, you have to set last parameter to 1 for fork_process().

t_request_within() sends a request within an existing dialog (subscribe
dialog in this case).


Reply to this email directly or view it on GitHub
#11 (comment).

@miconda
Copy link
Member

miconda commented Dec 23, 2014

presence module sends notify only to active subscribe dialogs (to active watchers) -- it doesn't know where to send the notify if there was no subscribe before it (it is how sip presence specs are).

@lazedo
Copy link
Contributor Author

lazedo commented Dec 23, 2014

thank you for helping debug this. the problem was on the background process forked without the tcp option. the strange thing is that it works with udp.

guillemj added a commit to guillemj/kamailio that referenced this issue Nov 14, 2019
The code enabled by SIG_DEBUG inside signal handlers is
async-signal-unsafe (see the signal-safetey(7) man page), as it calls
at least syslog(3) and stdio functions.

During a daemon shutdown via a SIGTERM, we got a crash in isatty(3) due
to stack exhaustion from infinite nested calls within stdio coming from
the syslog(3) in src/main.c:sig_usr():

  #0  0x00007f691903fe6d in __isatty (fd=2) at ../sysdeps/posix/isatty.c:24
        term = <error reading variable term (Cannot access memory at address 0x7fffa44f4fb0)>
  kamailio#1  0x00007f6918fc37bb in local_isatty (fd=<optimized out>)
  kamailio#2  __GI__IO_file_doallocate (fp=0x7f6919110680 <_IO_2_1_stderr_>)
  kamailio#3  0x00007f6918fd1700 in __GI__IO_doallocbuf (
  kamailio#4  0x00007f6918fd0988 in _IO_new_file_overflow (
  kamailio#5  0x00007f6918fcfa8f in _IO_new_file_xsputn (n=8, data=<optimized out>,
  kamailio#6  _IO_new_file_xsputn (f=0x7f6919110680 <_IO_2_1_stderr_>,
  7  0x00007f6918fa41fb in _IO_vfprintf_internal (
  kamailio#8  0x00007f6918fc32e8 in locked_vfxprintf (
  kamailio#9  0x00007f6918fc3469 in __fxprintf (fp=0x7f6919110680 <_IO_2_1_stderr_>,
  kamailio#10 0x00007f6918fd3a50 in __malloc_assert (
  kamailio#11 0x00007f6918fd5e6f in sysmalloc (nb=nb@entry=4112,
  kamailio#12 0x00007f6918fd72c9 in _int_malloc (
  kamailio#13 0x00007f6918fd83e3 in __GI___libc_malloc (bytes=bytes@entry=4096)

  [ next calls nested until stack exhaustion ]

  #41030 0x00007f6918fc371c in __GI__IO_file_doallocate (
  #41031 0x00007f6918fd1700 in __GI__IO_doallocbuf (
  #41032 0x00007f6918fd0988 in _IO_new_file_overflow (
  #41033 0x00007f6918fcfa8f in _IO_new_file_xsputn (n=8, data=<optimized out>,
  #41034 _IO_new_file_xsputn (f=0x7f6919110680 <_IO_2_1_stderr_>,
  #41035 0x00007f6918fa41fb in _IO_vfprintf_internal (
  #41036 0x00007f6918fc32e8 in locked_vfxprintf (
  #41037 0x00007f6918fc3469 in __fxprintf (fp=0x7f6919110680 <_IO_2_1_stderr_>,
  #41038 0x00007f6918fd3a50 in __malloc_assert (
  #41039 0x00007f6918fd5e6f in sysmalloc (nb=nb@entry=4112,
  #41040 0x00007f6918fd72c9 in _int_malloc (
  #41041 0x00007f6918fd83e3 in __GI___libc_malloc (bytes=bytes@entry=4096)
  #41042 0x00007f6918fc371c in __GI__IO_file_doallocate (
  #41043 0x00007f6918fd1700 in __GI__IO_doallocbuf (
  #41044 0x00007f6918fd0988 in _IO_new_file_overflow (
  #41045 0x00007f6918fcfa8f in _IO_new_file_xsputn (n=8, data=<optimized out>,
  #41046 _IO_new_file_xsputn (f=0x7f6919110680 <_IO_2_1_stderr_>,
  #41047 0x00007f6918fa41fb in _IO_vfprintf_internal (
  #41048 0x00007f6918fc32e8 in locked_vfxprintf (
  #41049 0x00007f6918fc3469 in __fxprintf (fp=0x7f6919110680 <_IO_2_1_stderr_>,
  #41050 0x00007f6918fd3a50 in __malloc_assert (
  #41051 0x00007f6918fd5e6f in sysmalloc (nb=nb@entry=8208,
  #41052 0x00007f6918fd72c9 in _int_malloc (
  #41053 0x00007f6918fd91a2 in __libc_calloc (n=n@entry=1,
  #41054 0x00007f6918fcc0b3 in __GI___open_memstream (
  #41055 0x00007f69190477e0 in __GI___vsyslog_chk (pri=189, flag=-1,
  #41056 0x00007f6919047d4c in __syslog (pri=<optimized out>,
  #41057 0x0000563a58b01c08 in sig_usr (signo=<optimized out>) at main.c:857
  #41058 sig_usr (signo=<optimized out>) at main.c:816
  #41059 <signal handler called>

While this is certainly not easy to trigger, it's still not correct to
leave enabled in production builds. It might be an acceptable tradeoff
as a debugging aid, though.

So we switch the the code to off by default, and swap the meaning of the
macro to enable the unsafe debugging, to match.
nickvsnetworking pushed a commit to nickvsnetworking/kamailio that referenced this issue Aug 24, 2022
# This is the 1st commit message:

ss7ops: updated with relocation of lib/srutils to core/utils

# This is the commit message #2:

tmrec: updated with relocation of lib/srutils to core/utils

# This is the commit message #3:

topos: updated with relocation of lib/srutils to core/utils

# This is the commit message #4:

uac_redirect: updated with relocation of lib/srutils to core/utils

# This is the commit message kamailio#5:

usrloc: updated with relocation of lib/srutils to core/utils

# This is the commit message kamailio#6:

pkg: obs spec - removed lib/srutils from rpm pacaging

# This is the commit message kamailio#7:

core: crypto, utils - updated comments referencing srutils after relocation

# This is the commit message kamailio#8:

corex: declare nio specific globals in c file

# This is the commit message kamailio#9:

Makefile.defs: add -lm to linker flags

- on some linux distros is not linked by default

# This is the commit message kamailio#10:

pipelimit: make error log debug in rpc command when pipe not found

- error code is returned to rpc command

# This is the commit message kamailio#11:

tls: proper match of config profile by server id

# This is the commit message kamailio#12:

http_client: typo in variable name for network interface

# This is the commit message kamailio#13:

core: dprint - new flags for long engine json

- a: prefix attribute names with application name
- A: do not prefix message attribute with application name (when 'a' is
set)

# This is the commit message kamailio#14:

sipdump: small whitespace fixes

# This is the commit message kamailio#15:

sipdump: option to run event_route[sipdump:msg] on rcv/snd messages

- new modparam mode to control the behaviour, its values can be built
from flags:
  - 1 - write to files (the existing behaviour), default
  - 2 - execute event route
- inside event route, $sipdump(...) variable is available to get the
buffer content and other related attributes
- new modparam event_callback to set the KEMI function to be run instead
of the event route

# This is the commit message kamailio#16:

sipdump: docs for mode and event_callback parameters

# This is the commit message kamailio#17:

sipdump: docs for event route sipdump:msg

# This is the commit message kamailio#18:

modules: readme files regenerated - sipdump ... [skip ci]

# This is the commit message kamailio#19:

sipdump: kemi functions to return buf and tag values

# This is the commit message kamailio#20:

seas: docs - removed mi_fifo from examples

# This is the commit message kamailio#21:

modules: readme files regenerated - seas ... [skip ci]

# This is the commit message kamailio#22:

textops: docs - proper example for append_body_part() with headers

# This is the commit message kamailio#23:

modules: readme files regenerated - textops ... [skip ci]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants