Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SIGABRT after upgrading to 3.11.14 version #1027

Closed
vpalmisano opened this issue Mar 21, 2023 · 4 comments · Fixed by #1028
Closed

SIGABRT after upgrading to 3.11.14 version #1027

vpalmisano opened this issue Mar 21, 2023 · 4 comments · Fixed by #1028
Labels
bug

Comments

@vpalmisano
Copy link
Contributor

vpalmisano commented Mar 21, 2023 •
edited

Bug Report

Your environment

  • Operating system: Linux
  • Node version:
  • npm version:
  • gcc/clang version:
  • mediasoup version: 3.11.14
  • mediasoup-client version: 3.6.82

Issue description

After upgrading to 3.11.14 version (containing #1023), we received a SIGABRT, with this stack trace:

#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x0000ffffabab7aa0 in __GI_abort () at abort.c:79
#2  0x0000ffffabdad238 in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
#3  0x0000ffffabdaad4c in ?? () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
#4  0x0000ffffabdaadb0 in std::terminate() () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
#5  0x0000ffffabdab0a4 in __cxa_throw () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
#6  0x0000ffffabdd3950 in std::__throw_out_of_range_fmt(char const*, ...) () from /usr/lib/aarch64-linux-gnu/libstdc++.so.6
#7  0x0000aaaabc66aa3c in RTC::RetransmissionBuffer::Insert(RTC::RtpPacket*, std::shared_ptr<RTC::RtpPacket>&) ()
#8  0x0000aaaabc68aadc in RTC::RtpStreamSend::ReceivePacket(RTC::RtpPacket*, std::shared_ptr<RTC::RtpPacket>&) ()
#9  0x0000aaaabc6afca4 in RTC::SvcConsumer::SendRtpPacket(RTC::RtpPacket*, std::shared_ptr<RTC::RtpPacket>&) ()
#10 0x0000aaaabc676384 in RTC::Router::OnTransportProducerRtpPacketReceived(RTC::Transport*, RTC::Producer*, RTC::RtpPacket*) ()
#11 0x0000aaaabc663e7c in RTC::Producer::ReceiveRtpPacket(RTC::RtpPacket*) ()
#12 0x0000aaaabc6b4b18 in RTC::Transport::ReceiveRtpPacket(RTC::RtpPacket*) ()
#13 0x0000aaaabc64b0d8 in RTC::PipeTransport::OnUdpSocketPacketReceived(RTC::UdpSocket*, unsigned char const*, unsigned long, sockaddr const*) ()
#14 0x0000aaaabc96cc34 in uv.udp_recvmmsg ()
#15 0x0000aaaabc96d8c4 in uv.udp_io ()
#16 0x0000aaaabc970cc4 in uv.io_poll ()
#17 0x0000aaaabc9646a4 in uv_run ()
#18 0x0000aaaabc5c6074 in DepLibUV::RunLoop() ()
#19 0x0000aaaabc5cf3d0 in Worker::Worker(Channel::ChannelSocket*, PayloadChannel::PayloadChannelSocket*) ()
#20 0x0000aaaabc5c4c14 in mediasoup_worker_run ()
#21 0x0000aaaabc5c3834 in main ()
@vpalmisano vpalmisano added the bug label Mar 21, 2023
@ibc ibc mentioned this issue Mar 21, 2023
Merged
@ibc
Copy link
Member

ibc commented Mar 21, 2023 •
edited by nazar-pc

I'm able to reproduce the issue in ongoing PR #1028:

[fuzzer] RTP fuzzers enabled
==4995==T2: stack [0x7f343f0f9000,0x7f343f888180) size 0x78f180; local=0x7f343f8880a4
INFO: seed corpus: files: 5617 min: 1b max: 261774b total: 1447245b rss: 53Mb
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976892730]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:13157, timestamp:862283031]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:13114, timestamp:859453491]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976892264]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976897098]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:13114, timestamp:859464290]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976889088]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:13056, timestamp:855638184]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976891950]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:14906, timestamp:976891962]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:12846, timestamp:841888049]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976905830]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:15677, timestamp:1027420485]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:17722, timestamp:1161443894]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976892672]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:13102, timestamp:858665774]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:12850, timestamp:842150702]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:14906, timestamp:976891941]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:15677, timestamp:1027423549]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:12346, timestamp:809120580]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:12645, timestamp:828715313]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:12645, timestamp:828702743]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:257, maxSize:2500]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33742, timestamp:2211317269]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 4294962805ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:33998, timestamp:2228092928]
terminate called after throwing an instance of 'std::out_of_range'
  what():  deque::_M_range_check: __n (which is 33741)>= this->size() (which is 1)
==4995== ERROR: libFuzzer: deadly signal
    #0 0x558534677381 in __sanitizer_print_stack_trace (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x350381) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #1 0x5585345e9c18 in fuzzer::PrintStackTrace() (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x2c2c18) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #2 0x5585345cf693 in fuzzer::Fuzzer::CrashCallback() (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x2a8693) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #3 0x7f3443cc351f  (/lib/x86_64-linux-gnu/libc.so.6+0x4251f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #4 0x7f3443d17a7b in pthread_kill (/lib/x86_64-linux-gnu/libc.so.6+0x96a7b) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #5 0x7f3443cc3475 in gsignal (/lib/x86_64-linux-gnu/libc.so.6+0x42475) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #6 0x7f3443ca97f2 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x287f2) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #7 0x7f3444052bbd  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xa2bbd) (BuildId: f57e02bfadacc0c923c82457d5e18e1830b5faea)
    #8 0x7f344405e24b  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xae24b) (BuildId: f57e02bfadacc0c923c82457d5e18e1830b5faea)
    #9 0x7f344405e2b6 in std::terminate() (/lib/x86_64-linux-gnu/libstdc++.so.6+0xae2b6) (BuildId: f57e02bfadacc0c923c82457d5e18e1830b5faea)
    #10 0x7f344405e517 in __cxa_throw (/lib/x86_64-linux-gnu/libstdc++.so.6+0xae517) (BuildId: f57e02bfadacc0c923c82457d5e18e1830b5faea)
    #11 0x7f3444055512  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xa5512) (BuildId: f57e02bfadacc0c923c82457d5e18e1830b5faea)
    #12 0x558534af4dcb in std::deque<RTC::RtpRetransmissionBuffer::Item*, std::allocator<RTC::RtpRetransmissionBuffer::Item*> >::_M_range_check(unsigned long) const /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_deque.h:1356:4
    #13 0x558534af1018 in std::deque<RTC::RtpRetransmissionBuffer::Item*, std::allocator<RTC::RtpRetransmissionBuffer::Item*> >::at(unsigned long) /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/stl_deque.h:1377:2
    #14 0x558534aef94b in RTC::RtpRetransmissionBuffer::Insert(RTC::RtpPacket*, std::shared_ptr<RTC::RtpPacket>&) /mediasoup/worker/out/Release/build/../../../src/RTC/RtpRetransmissionBuffer.cpp:265:36
    #15 0x558534f06c3d in Fuzzer::RTC::RtpRetransmissionBuffer::Fuzz(unsigned char const*, unsigned long) /mediasoup/worker/out/Release/build/../../../fuzzer/src/RTC/FuzzerRtpRetransmissionBuffer.cpp:37:24
    #16 0x558534efe881 in LLVMFuzzerTestOneInput /mediasoup/worker/out/Release/build/../../../fuzzer/src/fuzzer.cpp:47:3
    #17 0x5585345d0c23 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x2a9c23) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #18 0x5585345d1845 in fuzzer::Fuzzer::TryDetectingAMemoryLeak(unsigned char const*, unsigned long, bool) (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x2aa845) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #19 0x5585345d2008 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile> >&) (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x2ab008) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #20 0x5585345d24d2 in fuzzer::Fuzzer::Loop(std::vector<fuzzer::SizedFile, std::allocator<fuzzer::SizedFile> >&) (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x2ab4d2) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #21 0x5585345c0822 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x299822) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #22 0x5585345ea512 in main (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x2c3512) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)
    #23 0x7f3443caad8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #24 0x7f3443caae3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
    #25 0x5585345b5264 in _start (/mediasoup/worker/out/Release/mediasoup-worker-fuzzer+0x28e264) (BuildId: c4e70f11a7aac36986e4945b010003e9aa512c02)

NOTE: libFuzzer has rudimentary signal handlers.
      Combine libFuzzer with AddressSanitizer or similar for better crash reports.
SUMMARY: libFuzzer: deadly signal
MS: 0 ; base unit: 0000000000000000000000000000000000000000
0x84,0xce,0x0,0x0,
\204\316\000\000
artifact_prefix='fuzzer/reports/'; Test unit written to fuzzer/reports/crash-d54bc342b7d2ee881a9ba853289e831b93375e2d
Base64: hM4AAA==

@ibc
Copy link
Member

ibc commented Mar 21, 2023 •
edited

In summary, the crash is reproducible by calling rtpRetransmissionBuffer->Insert() with the following RTP packets:

  1. seq: 12645, timestamp: 828702743
  2. seq: 33998, timestamp: 2228092928
  3. seq: 33998, timestamp: 2228092928
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:12645, timestamp:828702743]
RTC::RtpRetransmissionBuffer::Insert() | buffer empty [seq:12645, timestamp:828702743]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500, startSeq:12645]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:12645, timestamp:828702743]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:12645, timestamp:828702743]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Insert() | packet in order [seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Insert() | packet has too high seq and forces buffer emptying [ssrc:2, seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Dump() | <RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Dump() |   buffer [size:1, maxSize:2500, startSeq:0]
RTC::RtpRetransmissionBuffer::Dump() |   oldest item [seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Dump() |   newest item [seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Dump() |   buffer window: 0ms
RTC::RtpRetransmissionBuffer::Dump() | </RtpRetransmissionBuffer>
RTC::RtpRetransmissionBuffer::Insert() | packet [seq:33998, timestamp:2228092928]
RTC::RtpRetransmissionBuffer::Insert() | packet out of order and in between oldest and newest packets in the buffer [seq:33998, timestamp:2228092928]

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mediasoup-worker-test is a Catch v2.13.7 host application.
Run with -? for options

-------------------------------------------------------------------------------
Scenario: RtpRetransmissionBuffer
  fuzzer generated packets
-------------------------------------------------------------------------------
../../../test/src/RTC/TestRtpRetransmissionBuffer.cpp:233
...............................................................................

../../../test/src/RTC/TestRtpRetransmissionBuffer.cpp:233: FAILED:
due to unexpected exception with message:
  deque::_M_range_check: __n (which is 33997)>= this->size() (which is 1)

@ibc
Copy link
Member

ibc commented Mar 21, 2023

Issue fixed in #1028. Will release version once CI passes.

@ibc ibc closed this as completed in #1028 Mar 21, 2023
@ibc
Copy link
Member

ibc commented Mar 21, 2023

3.11.15 is out.

ibc added a commit that referenced this issue Mar 22, 2023
@ibc
RtpRetransmissionBuffer: get rid of startSeq private member
6db399b 
- `this->startSeq` is just the seq of the first element in `this->buffer`, nothing else.
- It doesn't make sense to have a separate member just for it.
- We can just read `sequenceNumber` in the oldest item in the buffer.
- The recent crash reported in #1027 was caused due to a non properly updated `this->startSeq`. So let's get rid of it completely.
@ibc ibc mentioned this issue Mar 22, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add fuzzer for RtpRetransmissionBuffer and fix crash versatica/mediasoup
2 participants
@ibc @vpalmisano