Opensips crash on CANCEL on unanswerd call. (2.1-rc2) #484
Comments
|
In |
|
Hello, Can you privately send me the script you are using and describe how you are able to reproduce this ? ( email at vladpaiu@opensips.org if possible ) Best Regards, |
|
@avfedorov , any chances to grab your script for reproducing this crash ? |
|
I send all to Vlad. |
|
Hello, The shm_free checks whether the pointer is inside the SHM memory, and in your case it is entirely out of the memory block, so it looks like a memory corruption. Furthermore, sip_msg_cloner is called with updatable=0 only for replies, but in your case the crash is related to a request. The source of the crash appears to be a memory corruption - follow the steps at http://www.opensips.org/Documentation/TroubleShooting-OutOfMem in order to enable memory debugging ( see the make menuconfig section ) and reproduce your scenario - OpenSIPS should crash and generate a new core file. Please post the new backtrace after doing this. Best Regards, |
|
I add some debug to t_msgbuilder.h:free_faked_req inline static void free_faked_req(struct sip_msg *faked_req, struct cell *t)
{
LM_DBG("free_faked_req 1 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->new_uri.s) {
pkg_free(faked_req->new_uri.s);
faked_req->new_uri.s = NULL;
}
LM_DBG("free_faked_req 2 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->dst_uri.s) {
pkg_free(faked_req->dst_uri.s);
faked_req->dst_uri.s = NULL;
}
LM_DBG("free_faked_req 3 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->path_vec.s) {
pkg_free(faked_req->path_vec.s);
faked_req->path_vec.s = NULL;
}
LM_DBG("free_faked_req 4 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->set_global_address.s) {
pkg_free(faked_req->set_global_address.s);
faked_req->set_global_address.s = NULL;
}
LM_DBG("free_faked_req 5 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->set_global_port.s) {
pkg_free(faked_req->set_global_port.s);
faked_req->set_global_port.s = NULL;
}
LM_DBG("free_faked_req 6 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
/* SDP in not cloned into SHM, so if we have one, it means the SDP
* was parsed in the fake environment, so we have to free it */
if (faked_req->sdp)
free_sdp(&(faked_req->sdp));
LM_DBG("free_faked_req 7 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->multi) {
free_multi_body(faked_req->multi);
faked_req->multi = NULL;
}
LM_DBG("free_faked_req 8 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->msg_cb) {
msg_callback_process(faked_req, MSG_DESTROY, NULL);
}
LM_DBG("free_faked_req 9 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
/* free all types of lump that were added in failure handlers */
del_notflaged_lumps( &(faked_req->add_rm), LUMPFLAG_SHMEM );
LM_DBG("free_faked_req 10 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
del_notflaged_lumps( &(faked_req->body_lumps), LUMPFLAG_SHMEM );
LM_DBG("free_faked_req 11 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
del_nonshm_lump_rpl( &(faked_req->reply_lump) );
LM_DBG("free_faked_req 12 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->add_rm && faked_req->add_rm != t->uas.request->add_rm)
shm_free(faked_req->add_rm);
LM_DBG("free_faked_req 13 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->body_lumps && faked_req->body_lumps != t->uas.request->body_lumps)
shm_free(faked_req->body_lumps);
LM_DBG("free_faked_req 14 faked_req->req_lump=%p, t->uas.request->reply_lump=%p\n",faked_req->reply_lump,t->uas.request->reply_lump);
if (faked_req->reply_lump && faked_req->reply_lump != t->uas.request->reply_lump)
shm_free(faked_req->reply_lump);
clean_msg_clone( faked_req, t->uas.request, t->uas.end_request);
}And I get in log this: Now it is crashed on Vlad, I send you log with memory debug. |
|
I am also seeing this problem in a certain scenario, the output from opensips in logs is: Jun 30 15:54:52 lab-pett-opensips2 scp1a[13568]: CRITICAL:tm:free_faked_req:
It seems you have hit a programming bug. |
|
Managed to get a core dump for this: |
|
Hello, Thanks for the backtrace, Pete. Will coordinate with you today to login on the box where the crash occurs and further debug the core file. Best Regards, |
|
Some further info @vladpaiu - this seems very connected to failure route. If there is a failure route defined (even one that does only a simple xlog) then opensips will crash on failure response. I feel sure it must be related to script somehow too as I have other 2.1 scripts which exhibit no crashing behaviour |
|
Hi @avfedorov , @petekelly - I'm taking over this issue. Thanks and regards, |
The path_vec (array holding the PATH string) was not properly pushed into the faked requests before calling the failure route. This was generating a mixture of shm/pkg memory when the faked request had to be freed. This crash was trigger only if PATH support was used and requests were looked up with PATH header. Reported by @avfedorov and @petekelly. Closes #484 Many thanks to @avfedorov for his details information and to @petekelly for support in investigating and testing.
|
@avfedorov and @petekelly many thanks for the information and support you provided in regards to this bug. This fix will be part of the tomorrow minor release OpenSIPS 2.1.1 . Thanks and regards, |
The text was updated successfully, but these errors were encountered: