Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

5.7 crash due to rtpengine module bencode, on aarch64 (raspberry pi) #3444

Closed
smititelu opened this issue May 9, 2023 · 8 comments
Closed

5.7 crash due to rtpengine module bencode, on aarch64 (raspberry pi) #3444

smititelu opened this issue May 9, 2023 · 8 comments

Comments

@smititelu
Copy link
Contributor

Hi, I've built today the 5.7 debs for kamailio together with latest rtpengine master branch debs (commit 13a7e1d), for raspberry pi and tried to give it a try:

root@pisip:~# sudo dpkg -l | grep kama
ii  kamailio                             5.7.0~rc0                        armhf        very fast, dynamic and configurable SIP server
ii  kamailio-dbg:armhf                   5.7.0~rc0                        armhf        very fast and configurable SIP server [debug symbols]
ii  kamailio-presence-modules:armhf      5.7.0~rc0                        armhf        SIP presence modules for Kamailio
ii  kamailio-tls-modules:armhf           5.7.0~rc0                        armhf        TLS support for the Kamailio SIP server (authentication, transport)
ii  kamailio-websocket-modules:armhf     5.7.0~rc0                        armhf        WebSocket module for the Kamailio SIP server
root@pisip:~# sudo dpkg -l | grep rtpe
ii  ngcp-rtpengine                       11.4.0.0+0~mr11.4.0.0            all          NGCP RTP/media proxy - meta package
ii  ngcp-rtpengine-daemon                11.4.0.0+0~mr11.4.0.0            armhf        proxy for RTP and media streams used in NGCP, userspace part
ii  ngcp-rtpengine-iptables              11.4.0.0+0~mr11.4.0.0            armhf        IPtables extension module for the kernel-space NGCP media proxy
ii  ngcp-rtpengine-kernel-dkms           11.4.0.0+0~mr11.4.0.0            all          IPtables kernel module for the NGCP media proxy - DKMS
ii  ngcp-rtpengine-recording-daemon      11.4.0.0+0~mr11.4.0.0            armhf        recording daemon for RTP and media streams
ii  ngcp-rtpengine-utils                 11.4.0.0+0~mr11.4.0.0            all          scripts and Perl modules for NGCP rtpengine
root@pisip:~# sudo dpkg -l | grep bencode
ii  libbencode-perl                      1.501-1.1                        all          Perl module for BitTorrent serialisation format
root@pisip:~# uname -a
Linux ... 6.1.25-v8+ #1647 SMP PREEMPT Wed Apr 26 17:39:08 BST 2023 aarch64 GNU/Linux
root@pisip:~# cat /etc/issue
Raspbian GNU/Linux 11 \n \l

Kamailio crashes on startup, only when rtpengine module is loaded and rtp proxies are queried:

GNU gdb (Raspbian 10.1-1.7) 10.1.90.20210103-git
Copyright (C) 2021 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<https://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from kamailio...
Reading symbols from /usr/lib/debug/.build-id/b1/3b2e37e49071a519b33ce95a0b555c21a2d8f4.debug...

warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
[New LWP 5455]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
Core was generated by `/usr/sbin/kamailio -P /var/run/kamailio/kamailio_pid -f /etc/kamailio/kamailio_'.
Program terminated with signal SIGBUS, Bus error.
#0  0xf3237de8 in __bencode_decode_dictionary (end=0xf3294859 <buf+25> "", s=0xf329484a <buf+10> "6:result4:ponge", buf=0xffd06608) at bencode.c:464
464	bencode.c: No such file or directory.
(gdb) bt
#0  0xf3237de8 in __bencode_decode_dictionary (end=0xf3294859 <buf+25> "", s=0xf329484a <buf+10> "6:result4:ponge", buf=0xffd06608) at bencode.c:464
#1  __bencode_decode (buf=buf@entry=0xffd06608, s=0xf3294849 <buf+9> "d6:result4:ponge", end=0xf3294859 <buf+25> "") at bencode.c:613
#2  0xf3238a98 in bencode_decode (buf=buf@entry=0xffd06608, s=<optimized out>, len=<optimized out>) at bencode.c:638
#3  0xf32469dc in bencode_decode_expect (expect=BENCODE_DICTIONARY, len=<optimized out>, s=<optimized out>, buf=0xffd06600) at bencode.h:490
#4  rtpp_test (node=node@entry=0xb31464d0, isdisabled=isdisabled@entry=0, force=force@entry=1) at rtpengine.c:2954
#5  0xf324ef5c in build_rtpp_socks (lmode=lmode@entry=0, rtest=-1968410368, rtest@entry=1) at rtpengine.c:2021
#6  0xf32516d4 in child_init (rank=1) at rtpengine.c:2105
#7  child_init (rank=1) at rtpengine.c:2082
#8  0x0019fe08 in init_mod_child (m=0xf5cbbb54, rank=rank@entry=1) at core/sr_module.c:899
#9  0x0019fdd4 in init_mod_child (m=0xf5cbc2dc, rank=rank@entry=1) at core/sr_module.c:892
#10 0x0019fdd4 in init_mod_child (m=0xf5cbdbe4, rank=rank@entry=1) at core/sr_module.c:892
#11 0x0019fdd4 in init_mod_child (m=0xf5cbe46c, rank=rank@entry=1) at core/sr_module.c:892
#12 0x0019fdd4 in init_mod_child (m=0xf5cbe6fc, rank=rank@entry=1) at core/sr_module.c:892
#13 0x0019fdd4 in init_mod_child (m=0xf5cbea70, rank=rank@entry=1) at core/sr_module.c:892
#14 0x0019fdd4 in init_mod_child (m=0xf5cc4d64, rank=rank@entry=1) at core/sr_module.c:892
#15 0x0019fdd4 in init_mod_child (m=0xf5cc5558, rank=rank@entry=1) at core/sr_module.c:892
#16 0x0019fdd4 in init_mod_child (m=0xf5cc6670, rank=rank@entry=1) at core/sr_module.c:892
#17 0x0019fdd4 in init_mod_child (m=0xf5cc7630, rank=rank@entry=1) at core/sr_module.c:892
#18 0x001a486c in init_child (rank=rank@entry=1) at core/sr_module.c:953
#19 0x001117b0 in fork_process (child_id=1, desc=desc@entry=0xffd06f44 "\230I\244\001\214:\323\365udp receiver child=0 sock=192.168.100.57:50060 (86.123.193.247:50060)", make_sock=make_sock@entry=1)
    at core/pt.c:337
#20 0x000369c8 in main_loop () at main.c:1708
#21 0x0002c21c in main (argc=<optimized out>, argv=0xffd074a4) at main.c:3086

Looks like some bencode issue I'm having but lib bencode version looks ok. So far couldn't figure it out. Note that I also had a kernel bump to 6.1.25-v8+ recently.

Any opinions on this?

Thank you,
Stefan
@rfuchs
Copy link
Member

rfuchs commented May 9, 2023 •
edited

Probably unaligned memory access. Github isn't cooperating right now and it won't let me open a PR or branch, but you can apply this patch directly to the sources and see if that fixes it: sipwise/rtpengine@ade8100

@smititelu
Copy link
Contributor Author

hi, I see that commit is already included in latest sipwise rtpengine upstream master, from:

ade8100d3 (Richard Fuchs    2021-03-29 12:48:13 -0400  17) #define BENCODE_ALLOC_ALIGN          8

@rfuchs
Copy link
Member

rfuchs commented May 9, 2023

hi, I see that commit is already included in latest sipwise rtpengine upstream master, from:

ade8100d3 (Richard Fuchs    2021-03-29 12:48:13 -0400  17) #define BENCODE_ALLOC_ALIGN          8

Yes, I'm saying to apply it to the Kamailio sources (rtpengine module)

@smititelu
Copy link
Contributor Author

ok, will try

linuxmaniac pushed a commit that referenced this issue May 9, 2023
@rfuchs @linuxmaniac
rtpengine: fix unaligned memory access
89ba057 
Make sure the pointers we return from our continuous memory buffer is
always 64-bit aligned as it's used not only for strings, but also for
structs/objects, and such unaligned memory access is undefined on some
archs and flagged as such by ASAN.

From sipwise/rtpengine@ade8100

fixes #3444
@linuxmaniac linuxmaniac mentioned this issue May 9, 2023
Merged
10 tasks
@linuxmaniac
Copy link
Member

Probably unaligned memory access. Github isn't cooperating right now and it won't let me open a PR or branch, but you can apply this patch directly to the sources and see if that fixes it: sipwise/rtpengine@ade8100

created the PR with the code mentioned

@smititelu
Copy link
Contributor Author

@rfuchs @linuxmaniac I confirm this fixes the crash. Thank you.

@smititelu
Copy link
Contributor Author

is ok to backport this to 5.6 and 5.7?

@miconda
Copy link
Member

miconda commented May 10, 2023

Yes, it can be backported. 5.5 is also a maintained branch at this time, so it can go there as well.

smititelu pushed a commit that referenced this issue May 10, 2023
@rfuchs @smititelu
rtpengine: fix unaligned memory access
04b49bb 
Make sure the pointers we return from our continuous memory buffer is
always 64-bit aligned as it's used not only for strings, but also for
structs/objects, and such unaligned memory access is undefined on some
archs and flagged as such by ASAN.

From sipwise/rtpengine@ade8100

fixes #3444

(cherry-picked from commit 43ac6b2)
smititelu pushed a commit that referenced this issue May 10, 2023
@rfuchs @smititelu
rtpengine: fix unaligned memory access
2498924 
Make sure the pointers we return from our continuous memory buffer is
always 64-bit aligned as it's used not only for strings, but also for
structs/objects, and such unaligned memory access is undefined on some
archs and flagged as such by ASAN.

From sipwise/rtpengine@ade8100

fixes #3444

(cherry-picked from commit 43ac6b2)
smititelu pushed a commit that referenced this issue May 10, 2023
@rfuchs @smititelu
rtpengine: fix unaligned memory access
ce08c37 
Make sure the pointers we return from our continuous memory buffer is
always 64-bit aligned as it's used not only for strings, but also for
structs/objects, and such unaligned memory access is undefined on some
archs and flagged as such by ASAN.

From sipwise/rtpengine@ade8100

fixes #3444

(cherry-picked from commit 43ac6b2)
ThomasLobker pushed a commit to ProVoice/kamailio that referenced this issue Apr 18, 2024
@rfuchs
rtpengine: fix unaligned memory access
ec88047 
closes kamailio#3444
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@linuxmaniac @miconda @rfuchs @smititelu