Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create time #7

Merged
merged 1 commit into from Aug 26, 2013
Merged

Create time #7

merged 1 commit into from Aug 26, 2013

Conversation

ken-tilera
Copy link

Clean up code. I was debugging a performance issue and modified one copy of CreateTimeString() thinking that was the only copy and then didn't see any effect, because I had edited the wrong copy.

There were 8 identical copies of CreateTimeString() in 8 files.
Most used SCLocalTime, to replace localtime_r(), but some did not.
Created one copy in util-time.c.
regit added a commit that referenced this pull request Aug 26, 2013
@regit regit merged commit faf242f into regit:regit-master Aug 26, 2013
@ken-tilera ken-tilera deleted the create-time branch November 14, 2013 19:53
regit added a commit that referenced this pull request Nov 17, 2014
Under serious load, it is possible that a app layer get changed on
a flow when another packet of the Flow is still examined in Detect.
The consequence is that it is possible to app layer to get updated
and the rest of the detection run with the new app layer that don't
have the same property (such as Tx handling function which are called
on TLS session).

This is mainly the case when alproto is fetched from Flow and then
the Flow is unlocked and modifiable by another thread. When alstate
is fetch later, we can have alstate not matching alproto and this
causes crashes.

To fix that this patch is using alindex to access to the original
application layer during the detection. This means some function
prototype have been update to use alindex instead of alproto.
Also the FlowGet*AtIndex function are used with alindex param to
access to the correct application layer.

For reference, here's one the backtrace:

 (gdb) bt
 #0  0x0000000000000000 in ?? ()
 #1  0x00000000004310e3 in AppLayerParserSetTransactionInspectId (pstate=0x151743b10, ipproto=ipproto@entry=6 '\006', alproto=alproto@entry=4, alstate=alstate@entry=0x14d47b790, direction=direction@entry=4 '\004')
     at app-layer-parser.c:536
 #2  0x000000000048e6d7 in DeStateUpdateInspectTransactionId (f=0x7ffefc68ba00, direction=4 '\004') at detect-engine-state.c:785
 #3  0x000000000045c034 in SigMatchSignatures (th_v=0x3420fa50, de_ctx=0x23b6f00, det_ctx=0x13fd68ea0, p=<optimized out>) at detect.c:1589
 #4  0x000000000045c9f3 in Detect (data=<optimized out>, p=<optimized out>, tv=<optimized out>, pq=<optimized out>, postpq=<optimized out>) at detect.c:1744
 #5  Detect (tv=<optimized out>, p=<optimized out>, data=<optimized out>, pq=<optimized out>, postpq=<optimized out>) at detect.c:1716
 #6  0x000000000053d24d in TmThreadsSlotVarRun (tv=0x3420fa50, p=0x13fd56920, slot=0x14d47b790, slot@entry=0x13db1ecc0) at tm-threads.c:575
 #7  0x00000000005186ba in TmThreadsSlotProcessPkt (p=0x13fd56920, s=0x13db1ecc0, tv=0x3420fa50) at tm-threads.h:148
 #8  AFPReadFromRing (ptv=ptv@entry=0x13b029bd0) at source-af-packet.c:875
 #9  0x000000000051b5fd in ReceiveAFPLoop (tv=<optimized out>, data=0x13b029bd0, slot=<optimized out>) at source-af-packet.c:1215
 #10 0x00000000005408db in TmThreadsSlotPktAcqLoop (td=0x3420fa50) at tm-threads.c:722
 #11 0x00007ffff6920b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
 #12 0x00007ffff51e8a7d in clone () from /lib/x86_64-linux-gnu/libc.so.6
 #13 0x0000000000000000 in ?? ()
regit added a commit that referenced this pull request Jun 16, 2015
This patch fixes a partial long duration lock up in Suricata. The problem arises when
max_pending_packet is reached in worker mode. In that condition some capture threads
get blocked in FlowGetFlowFromHash call.

The following backtrace shows an example of the lock up. The first thread is waiting on
the flow bucket mutex and the second one is remaining stuck at PacketPoolWait because
there is almost no signalling in the used worker mode:

 (gdb) bt
 #0  0x00007f5442a4ed5c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0
 #1  0x00007f5442a4a3a9 in _L_lock_926 () from /lib/x86_64-linux-gnu/libpthread.so.0
 #2  0x00007f5442a4a1cb in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
 #3  0x00000000005346c0 in FlowGetFlowFromHash (tv=0x2c9f68c80, dtv=0x2cc63cc30, p=0x2cc62a700) at flow-hash.c:653
 #4  0x000000000053122c in FlowHandlePacket (tv=0x2c9f68c80, dtv=0x2cc63cc30, p=0x2cc62a700) at flow.c:340
 #5  0x0000000000461cb0 in DecodeTCP (tv=0x2c9f68c80, dtv=0x2cc63cc30, p=0x2cc62a700, pkt=0x7f542ec00bd4 "\312?\037L\277h\257p", len=32, pq=0x2c9f68f10) at decode-tcp.c:206
 #6  0x000000000045db38 in DecodeIPV4 (tv=0x2c9f68c80, dtv=0x2cc63cc30, p=0x2cc62a700, pkt=0x7f542ec00bc0 "E", len=61, pq=0x2c9f68f10) at decode-ipv4.c:561
 #7  0x0000000000459887 in DecodeEthernet (tv=0x2c9f68c80, dtv=0x2cc63cc30, p=0x2cc62a700, pkt=0x7f542ec00bb2 "", len=75, pq=0x2c9f68f10) at decode-ethernet.c:60
 #8  0x00000000005a928d in DecodeAFP (tv=0x2c9f68c80, p=0x2cc62a700, data=0x2cc63cc30, pq=0x2c9f68f10, postpq=0x0) at source-af-packet.c:1872
 #9  0x00000000005db191 in TmThreadsSlotVarRun (tv=0x2c9f68c80, p=0x2cc62a700, slot=0x2c9f68ed0) at tm-threads.c:132
 #10 0x00000000005a08a5 in TmThreadsSlotProcessPkt (tv=0x2c9f68c80, s=0x2c9f68ed0, p=0x2cc62a700) at tm-threads.h:147
 #11 0x00000000005a2982 in AFPReadFromRing (ptv=0x2cc62b710) at source-af-packet.c:874
 #12 0x00000000005a40c2 in ReceiveAFPLoop (tv=0x2c9f68c80, data=0x2cc62b710, slot=0x2c9f68d90) at source-af-packet.c:1214
 #13 0x00000000005dbae6 in TmThreadsSlotPktAcqLoop (td=0x2c9f68c80) at tm-threads.c:336
 #14 0x00007f5442a47b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
 #15 0x00007f544130f95d in clone () from /lib/x86_64-linux-gnu/libc.so.6
 #16 0x0000000000000000 in ?? ()
 (gdb) thread  5
 [Switching to thread 5 (Thread 0x7f54325a6700 (LWP 9282))]
 #0  0x00007f5442a4c344 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
 (gdb) bt
 #0  0x00007f5442a4c344 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/x86_64-linux-gnu/libpthread.so.0
 #1  0x00000000005d806e in PacketPoolWait () at tmqh-packetpool.c:152
 #2  0x0000000000539443 in FlowForceReassemblyPseudoPacketGet (direction=1, f=0x83a7880, ssn=0x2dcea3680, dummy=0) at flow-timeout.c:257
 #3  0x000000000053972f in FlowForceReassemblyForFlow (f=0x83a7880, server=2, client=1) at flow-timeout.c:377
 #4  0x0000000000535197 in FlowManagerFlowTimedOut (f=0x83a7880, ts=0x7f54325a52a0) at flow-manager.c:246
 #5  0x0000000000535231 in FlowManagerHashRowTimeout (f=0x83a7880, ts=0x7f54325a52a0, emergency=0, counters=0x7f54325a5280) at flow-manager.c:294
 #6  0x00000000005354f8 in FlowTimeoutHash (ts=0x7f54325a52a0, try_cnt=0, hash_min=0, hash_max=1048576, counters=0x7f54325a5280) at flow-manager.c:389
 #7  0x0000000000535e48 in FlowManager (th_v=0x2dd38e330, thread_data=0x2dd38de80) at flow-manager.c:612
 #8  0x00000000005dc7a0 in TmThreadsManagement (td=0x2dd38e330) at tm-threads.c:600
 #9  0x00007f5442a47b50 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
 #10 0x00007f544130f95d in clone () from /lib/x86_64-linux-gnu/libc.so.6
 #11 0x0000000000000000 in ?? ()

This problem is due to the fact that the return_stack condition is not signaled if a
packet is returned to the thread own PacketPool. So if the FlowManager try to get a
packet and has to wait for some to be available then it can get stuck on the condition
for a long time.
regit added a commit that referenced this pull request Mar 2, 2016
This patch fixes the following leak:

Direct leak of 9982880 byte(s) in 2902 object(s) allocated from:
    #0 0x4c253b in malloc ??:?
    #1 0x10c39ac in MimeDecInitParser /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/util-decode-mime.c:2379
    #2 0x6a0f91 in SMTPProcessRequest /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1085
    #3 0x697658 in SMTPParse /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1185
    #4 0x68fa7a in SMTPParseClientRecord /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1208
    #5 0x6561c5 in AppLayerParserParse /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-parser.c:908
    #6 0x53dc2e in AppLayerHandleTCPData /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer.c:444
    #7 0xf8e0af in DoReassemble /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:2635
    #8 0xf8c3f8 in StreamTcpReassembleAppLayer /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3028
    #9 0xf94267 in StreamTcpReassembleHandleSegmentUpdateACK /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3404
    #10 0xf9643d in StreamTcpReassembleHandleSegment /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3432
    #11 0xf578b4 in HandleEstablishedPacketToClient /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:2245
    #12 0xeea3c7 in StreamTcpPacketStateEstablished /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:2489
    #13 0xec1d38 in StreamTcpPacket /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:4568
    #14 0xeb0e16 in StreamTcp /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:5064
    #15 0xff52a4 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:130
    #16 0xffdad1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474
    #17 0x7f7cd678d181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2)

We come to this case when a SMTP session contains at least 2 mails
and then the ending of the first is not correctly detected. In that
case, switching to a new tx seems a good solution. This way we still
have partial logging.
regit added a commit that referenced this pull request Mar 3, 2016
This patch fixes the following leak:

Direct leak of 9982880 byte(s) in 2902 object(s) allocated from:
    #0 0x4c253b in malloc ??:?
    #1 0x10c39ac in MimeDecInitParser /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/util-decode-mime.c:2379
    #2 0x6a0f91 in SMTPProcessRequest /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1085
    #3 0x697658 in SMTPParse /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1185
    #4 0x68fa7a in SMTPParseClientRecord /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-smtp.c:1208
    #5 0x6561c5 in AppLayerParserParse /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer-parser.c:908
    #6 0x53dc2e in AppLayerHandleTCPData /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/app-layer.c:444
    #7 0xf8e0af in DoReassemble /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:2635
    #8 0xf8c3f8 in StreamTcpReassembleAppLayer /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3028
    #9 0xf94267 in StreamTcpReassembleHandleSegmentUpdateACK /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3404
    #10 0xf9643d in StreamTcpReassembleHandleSegment /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp-reassemble.c:3432
    #11 0xf578b4 in HandleEstablishedPacketToClient /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:2245
    #12 0xeea3c7 in StreamTcpPacketStateEstablished /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:2489
    #13 0xec1d38 in StreamTcpPacket /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:4568
    #14 0xeb0e16 in StreamTcp /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/stream-tcp.c:5064
    #15 0xff52a4 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:130
    #16 0xffdad1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474
    #17 0x7f7cd678d181 in start_thread /build/buildd/eglibc-2.19/nptl/pthread_create.c:312 (discriminator 2)

We come to this case when a SMTP session contains at least 2 mails
and then the ending of the first is not correctly detected. In that
case, switching to a new tx seems a good solution. This way we still
have partial logging.
regit added a commit that referenced this pull request Mar 4, 2016
Code was unused and was leaking memory.

This fixes:

Direct leak of 614240 byte(s) in 3839 object(s) allocated from:
    #0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
    #1 0x11bc12e in LogFileNewCtx /home/pmanev/sandnet-qa/stage/oisf/src/util-logopenfile.c:474:27
    #2 0xcf7ef2 in LogFilestoreLogInitCtx /home/pmanev/sandnet-qa/stage/oisf/src/log-filestore.c:430:31
    #3 0xec3275 in RunModeInitializeOutputs /home/pmanev/sandnet-qa/stage/oisf/src/runmodes.c:763:26
    #4 0xeae17f in UnixSocketPcapFilesCheck /home/pmanev/sandnet-qa/stage/oisf/src/runmode-unix-socket.c:391:9
    #5 0x109bc37 in UnixCommandBackgroundTasks /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:430:20
    #6 0x10a9be2 in UnixManager /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:977:9
    #7 0x1075643 in TmThreadsManagement /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:600:9
    #8 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
regit added a commit that referenced this pull request Mar 6, 2016
Code was unused and was leaking memory.

This fixes:

Direct leak of 614240 byte(s) in 3839 object(s) allocated from:
    #0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
    #1 0x11bc12e in LogFileNewCtx /home/pmanev/sandnet-qa/stage/oisf/src/util-logopenfile.c:474:27
    #2 0xcf7ef2 in LogFilestoreLogInitCtx /home/pmanev/sandnet-qa/stage/oisf/src/log-filestore.c:430:31
    #3 0xec3275 in RunModeInitializeOutputs /home/pmanev/sandnet-qa/stage/oisf/src/runmodes.c:763:26
    #4 0xeae17f in UnixSocketPcapFilesCheck /home/pmanev/sandnet-qa/stage/oisf/src/runmode-unix-socket.c:391:9
    #5 0x109bc37 in UnixCommandBackgroundTasks /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:430:20
    #6 0x10a9be2 in UnixManager /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:977:9
    #7 0x1075643 in TmThreadsManagement /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:600:9
    #8 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
regit added a commit that referenced this pull request Mar 7, 2016
Code was unused and was leaking memory.

This fixes:

Direct leak of 614240 byte(s) in 3839 object(s) allocated from:
    #0 0x4c396b in malloc (/opt/suricata-asan/bin/suricata+0x4c396b)
    #1 0x11bc12e in LogFileNewCtx /home/pmanev/sandnet-qa/stage/oisf/src/util-logopenfile.c:474:27
    #2 0xcf7ef2 in LogFilestoreLogInitCtx /home/pmanev/sandnet-qa/stage/oisf/src/log-filestore.c:430:31
    #3 0xec3275 in RunModeInitializeOutputs /home/pmanev/sandnet-qa/stage/oisf/src/runmodes.c:763:26
    #4 0xeae17f in UnixSocketPcapFilesCheck /home/pmanev/sandnet-qa/stage/oisf/src/runmode-unix-socket.c:391:9
    #5 0x109bc37 in UnixCommandBackgroundTasks /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:430:20
    #6 0x10a9be2 in UnixManager /home/pmanev/sandnet-qa/stage/oisf/src/unix-manager.c:977:9
    #7 0x1075643 in TmThreadsManagement /home/pmanev/sandnet-qa/stage/oisf/src/tm-threads.c:600:9
    #8 0x7fbc9fcb3181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
regit added a commit that referenced this pull request Mar 18, 2016
In JsonEmailLogJsonData function, an invalid state was leading to
early exit without a proper freeing of resources.

This should fix:

Indirect leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
    #1 0x7fb09c1e886a in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x686a)
    #2 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
    #3 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
    #4 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
    #5 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
    #6 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
    #7 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
regit added a commit that referenced this pull request Mar 18, 2016
This possibly fix:

ndirect leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
    #1 0x7fb09c1e8aaa in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x6aaa)
    #2 0xd67553 in JsonEmailLogJsonData /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:290:27
    #3 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
    #4 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
    #5 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
    #6 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
    #7 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
    #8 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
regit added a commit that referenced this pull request Mar 21, 2016
This patch fixes some error handling in code generating JSON output for
email event.

This fixes:

Indirect leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394771c  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x371c)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 96 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bc7b  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7c7b)

Indirect leak of 82 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff923949924  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x5924)

Indirect leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bcda in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7cda)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf5a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f5a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf2a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f2a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)
regit added a commit that referenced this pull request Mar 21, 2016
This patch fixes some error handling in code generating JSON output for
email event.

This fixes:

Indirect leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394771c  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x371c)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 96 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bc7b  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7c7b)

Indirect leak of 82 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff923949924  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x5924)

Indirect leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bcda in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7cda)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf5a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f5a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf2a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f2a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)
regit added a commit that referenced this pull request Mar 24, 2016
In JsonEmailLogJsonData function, an invalid state was leading to
early exit without a proper freeing of resources.

This should fix:

Indirect leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
    #1 0x7fb09c1e886a in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x686a)
    #2 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
    #3 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
    #4 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
    #5 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
    #6 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
    #7 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
regit added a commit that referenced this pull request Mar 24, 2016
This possibly fix:

ndirect leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x4c264b in malloc (/home/victor/qa/buildbot/donkey/z600fuzz/Private/src/.libs/lt-suricata+0x4c264b)
    #1 0x7fb09c1e8aaa in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x6aaa)
    #2 0xd67553 in JsonEmailLogJsonData /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:290:27
    #3 0xd6a272 in JsonEmailLogJson /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-email-common.c:370:19
    #4 0xd956b9 in JsonSmtpLogger /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-json-smtp.c:103:9
    #5 0xdcedac in OutputTxLog /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/output-tx.c:165:17
    #6 0xff6669 in TmThreadsSlotVarRun /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:132:17
    #7 0xffecc1 in TmThreadsSlotVar /home/victor/qa/buildbot/donkey/z600fuzz/Private/src/tm-threads.c:474:17
    #8 0x7fb09bfcc181 in start_thread /build/eglibc-3GlaMS/eglibc-2.19/nptl/pthread_create.c:312
regit added a commit that referenced this pull request Mar 24, 2016
This patch fixes some error handling in code generating JSON output for
email event.

This fixes:

Indirect leak of 128 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394771c  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x371c)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 96 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bc7b  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7c7b)

Indirect leak of 82 byte(s) in 3 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff923949924  (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x5924)

Indirect leak of 72 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bcda in json_object (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7cda)
    #2 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #3 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #4 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #5 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #6 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #7 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #8 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 64 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf5a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f5a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)

Indirect leak of 48 byte(s) in 1 object(s) allocated from:
    #0 0x50c142 in malloc (/home/eric/git/oisf/src/.libs/lt-suricata+0x50c142)
    #1 0x7ff92394bf2a in json_array (/usr/lib/x86_64-linux-gnu/libjansson.so.4+0x7f2a)
    #2 0x1bd04b5 in JsonEmailLogJsonData /home/eric/git/oisf/src/output-json-email-common.c:296:27
    #3 0x1bd3309 in JsonEmailLogJson /home/eric/git/oisf/src/output-json-email-common.c:376:19
    #4 0x1bfe774 in JsonSmtpLogger /home/eric/git/oisf/src/output-json-smtp.c:103:9
    #5 0x1c378ff in OutputTxLog /home/eric/git/oisf/src/output-tx.c:165:17
    #6 0x1f94ef3 in TmThreadsSlotVarRun /home/eric/git/oisf/src/tm-threads.c:134:17
    #7 0x1d33478 in TmThreadsSlotProcessPkt /home/eric/git/oisf/src/./tm-threads.h:150:9
    #8 0x1d32dd4 in PcapFileCallbackLoop /home/eric/git/oisf/src/source-pcap-file.c:184:9
    #9 0x7ff924199013  (/usr/lib/x86_64-linux-gnu/libpcap.so.0.8+0x1f013)
regit pushed a commit that referenced this pull request Mar 31, 2016
Direct leak of 80 byte(s) in 5 object(s) allocated from:
    #0 0x4c673b in __interceptor_malloc (/home/victor/dev/suricata/src/suricata+0x4c673b)
    #1 0xb7a425 in DetectEngineSignatureIsDuplicate /home/victor/dev/suricata/src/detect-parse.c:1715:10
    #2 0xb79390 in DetectEngineAppendSig /home/victor/dev/suricata/src/detect-parse.c:1836:19
    #3 0x86fe56 in DetectLoadSigFile /home/victor/dev/suricata/src/detect.c:357:15
    #4 0x815fee in ProcessSigFiles /home/victor/dev/suricata/src/detect.c:419:13
    #5 0x8139a8 in SigLoadSignatures /home/victor/dev/suricata/src/detect.c:499:15
    #6 0xfe435d in LoadSignatures /home/victor/dev/suricata/src/suricata.c:1979:9
    #7 0xfcd87e in main /home/victor/dev/suricata/src/suricata.c:2345:17
    #8 0x7fb66bf7cec4 in __libc_start_main /build/eglibc-3GlaMS/eglibc-2.19/csu/libc-start.c:287
regit added a commit that referenced this pull request Oct 14, 2016
The root packet was accessed even if it is NULL causing a NULL dereference:

ASAN:SIGSEGV
=================================================================
==24408==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000060 (pc 0x00000076f948 bp 0x7f435c000240 sp 0x7f435c000220 T5)
ASAN:SIGSEGV
==24408==AddressSanitizer: while reporting a bug found another one. Ignoring.
    #0 0x76f947 in NFQBypassCallback /home/victor/dev/suricata/src/source-nfq.c:510
    #1 0x4d0f02 in PacketBypassCallback /home/victor/dev/suricata/src/decode.c:395
    #2 0x7b8a95 in StreamTcpPacket /home/victor/dev/suricata/src/stream-tcp.c:4661
    #3 0x7b9ddd in StreamTcp /home/victor/dev/suricata/src/stream-tcp.c:4913
    #4 0x68fa50 in FlowWorker /home/victor/dev/suricata/src/flow-worker.c:194
    #5 0x7f0abd in TmThreadsSlotVarRun /home/victor/dev/suricata/src/tm-threads.c:128
    #6 0x7f2958 in TmThreadsSlotVar /home/victor/dev/suricata/src/tm-threads.c:585
    #7 0x7f436368e6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
    #8 0x7f4362802b5c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x106b5c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/victor/dev/suricata/src/source-nfq.c:510 NFQBypassCallback
Thread T5 (W#04) created by T0 (Suricata-Main) here:
    #0 0x7f4364ff2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f9c48 in TmThreadSpawn /home/victor/dev/suricata/src/tm-threads.c:1843
    #2 0x8da7c0 in RunModeSetIPSAutoFp /home/victor/dev/suricata/src/util-runmodes.c:519
    #3 0x73e3ff in RunModeIpsNFQAutoFp /home/victor/dev/suricata/src/runmode-nfq.c:74
    #4 0x7503fa in RunModeDispatch /home/victor/dev/suricata/src/runmodes.c:382
    #5 0x7e5cb3 in main /home/victor/dev/suricata/src/suricata.c:2547
    #6 0x7f436271c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
regit added a commit that referenced this pull request Oct 14, 2016
In case of a tunnel packet, adding a mark to the root packet will have
for consequence to bypass all the flows that are hosted in this tunnel.
This is not the attended behavior and as initial fix let's simply warn
suricata that bypass for NFQ is not possible for this kind of packets.

This patch also fixes a segfault. The root packet was accessed even if it is
NULL causing a NULL dereference:

ASAN:SIGSEGV
=================================================================
==24408==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000060 (pc 0x00000076f948 bp 0x7f435c000240 sp 0x7f435c000220 T5)
ASAN:SIGSEGV
==24408==AddressSanitizer: while reporting a bug found another one. Ignoring.
    #0 0x76f947 in NFQBypassCallback /home/victor/dev/suricata/src/source-nfq.c:510
    #1 0x4d0f02 in PacketBypassCallback /home/victor/dev/suricata/src/decode.c:395
    #2 0x7b8a95 in StreamTcpPacket /home/victor/dev/suricata/src/stream-tcp.c:4661
    #3 0x7b9ddd in StreamTcp /home/victor/dev/suricata/src/stream-tcp.c:4913
    #4 0x68fa50 in FlowWorker /home/victor/dev/suricata/src/flow-worker.c:194
    #5 0x7f0abd in TmThreadsSlotVarRun /home/victor/dev/suricata/src/tm-threads.c:128
    #6 0x7f2958 in TmThreadsSlotVar /home/victor/dev/suricata/src/tm-threads.c:585
    #7 0x7f436368e6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
    #8 0x7f4362802b5c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x106b5c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/victor/dev/suricata/src/source-nfq.c:510 NFQBypassCallback
Thread T5 (W#04) created by T0 (Suricata-Main) here:
    #0 0x7f4364ff2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f9c48 in TmThreadSpawn /home/victor/dev/suricata/src/tm-threads.c:1843
    #2 0x8da7c0 in RunModeSetIPSAutoFp /home/victor/dev/suricata/src/util-runmodes.c:519
    #3 0x73e3ff in RunModeIpsNFQAutoFp /home/victor/dev/suricata/src/runmode-nfq.c:74
    #4 0x7503fa in RunModeDispatch /home/victor/dev/suricata/src/runmodes.c:382
    #5 0x7e5cb3 in main /home/victor/dev/suricata/src/suricata.c:2547
    #6 0x7f436271c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
regit added a commit that referenced this pull request Oct 19, 2016
In case of a tunnel packet, adding a mark to the root packet will have
for consequence to bypass all the flows that are hosted in this tunnel.
This is not the attended behavior and as initial fix let's simply warn
suricata that bypass for NFQ is not possible for this kind of packets.

This patch also fixes a segfault. The root packet was accessed even if it is
NULL causing a NULL dereference:

ASAN:SIGSEGV
=================================================================
==24408==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000060 (pc 0x00000076f948 bp 0x7f435c000240 sp 0x7f435c000220 T5)
ASAN:SIGSEGV
==24408==AddressSanitizer: while reporting a bug found another one. Ignoring.
    #0 0x76f947 in NFQBypassCallback /home/victor/dev/suricata/src/source-nfq.c:510
    #1 0x4d0f02 in PacketBypassCallback /home/victor/dev/suricata/src/decode.c:395
    #2 0x7b8a95 in StreamTcpPacket /home/victor/dev/suricata/src/stream-tcp.c:4661
    #3 0x7b9ddd in StreamTcp /home/victor/dev/suricata/src/stream-tcp.c:4913
    #4 0x68fa50 in FlowWorker /home/victor/dev/suricata/src/flow-worker.c:194
    #5 0x7f0abd in TmThreadsSlotVarRun /home/victor/dev/suricata/src/tm-threads.c:128
    #6 0x7f2958 in TmThreadsSlotVar /home/victor/dev/suricata/src/tm-threads.c:585
    #7 0x7f436368e6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
    #8 0x7f4362802b5c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x106b5c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/victor/dev/suricata/src/source-nfq.c:510 NFQBypassCallback
Thread T5 (W#04) created by T0 (Suricata-Main) here:
    #0 0x7f4364ff2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f9c48 in TmThreadSpawn /home/victor/dev/suricata/src/tm-threads.c:1843
    #2 0x8da7c0 in RunModeSetIPSAutoFp /home/victor/dev/suricata/src/util-runmodes.c:519
    #3 0x73e3ff in RunModeIpsNFQAutoFp /home/victor/dev/suricata/src/runmode-nfq.c:74
    #4 0x7503fa in RunModeDispatch /home/victor/dev/suricata/src/runmodes.c:382
    #5 0x7e5cb3 in main /home/victor/dev/suricata/src/suricata.c:2547
    #6 0x7f436271c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
regit added a commit that referenced this pull request Oct 19, 2016
In case of a tunnel packet, adding a mark to the root packet will have
for consequence to bypass all the flows that are hosted in this tunnel.
This is not the attended behavior and as initial fix let's simply warn
suricata that bypass for NFQ is not possible for this kind of packets.

This patch also fixes a segfault. The root packet was accessed even if it is
NULL causing a NULL dereference:

ASAN:SIGSEGV
=================================================================
==24408==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000060 (pc 0x00000076f948 bp 0x7f435c000240 sp 0x7f435c000220 T5)
ASAN:SIGSEGV
==24408==AddressSanitizer: while reporting a bug found another one. Ignoring.
    #0 0x76f947 in NFQBypassCallback /home/victor/dev/suricata/src/source-nfq.c:510
    #1 0x4d0f02 in PacketBypassCallback /home/victor/dev/suricata/src/decode.c:395
    #2 0x7b8a95 in StreamTcpPacket /home/victor/dev/suricata/src/stream-tcp.c:4661
    #3 0x7b9ddd in StreamTcp /home/victor/dev/suricata/src/stream-tcp.c:4913
    #4 0x68fa50 in FlowWorker /home/victor/dev/suricata/src/flow-worker.c:194
    #5 0x7f0abd in TmThreadsSlotVarRun /home/victor/dev/suricata/src/tm-threads.c:128
    #6 0x7f2958 in TmThreadsSlotVar /home/victor/dev/suricata/src/tm-threads.c:585
    #7 0x7f436368e6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
    #8 0x7f4362802b5c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x106b5c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/victor/dev/suricata/src/source-nfq.c:510 NFQBypassCallback
Thread T5 (W#04) created by T0 (Suricata-Main) here:
    #0 0x7f4364ff2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f9c48 in TmThreadSpawn /home/victor/dev/suricata/src/tm-threads.c:1843
    #2 0x8da7c0 in RunModeSetIPSAutoFp /home/victor/dev/suricata/src/util-runmodes.c:519
    #3 0x73e3ff in RunModeIpsNFQAutoFp /home/victor/dev/suricata/src/runmode-nfq.c:74
    #4 0x7503fa in RunModeDispatch /home/victor/dev/suricata/src/runmodes.c:382
    #5 0x7e5cb3 in main /home/victor/dev/suricata/src/suricata.c:2547
    #6 0x7f436271c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
regit added a commit that referenced this pull request Oct 28, 2016
In case of a tunnel packet, adding a mark to the root packet will have
for consequence to bypass all the flows that are hosted in this tunnel.
This is not the attended behavior and as initial fix let's simply warn
suricata that bypass for NFQ is not possible for this kind of packets.

This patch also fixes a segfault. The root packet was accessed even if it is
NULL causing a NULL dereference:

ASAN:SIGSEGV
=================================================================
==24408==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000060 (pc 0x00000076f948 bp 0x7f435c000240 sp 0x7f435c000220 T5)
ASAN:SIGSEGV
==24408==AddressSanitizer: while reporting a bug found another one. Ignoring.
    #0 0x76f947 in NFQBypassCallback /home/victor/dev/suricata/src/source-nfq.c:510
    #1 0x4d0f02 in PacketBypassCallback /home/victor/dev/suricata/src/decode.c:395
    #2 0x7b8a95 in StreamTcpPacket /home/victor/dev/suricata/src/stream-tcp.c:4661
    #3 0x7b9ddd in StreamTcp /home/victor/dev/suricata/src/stream-tcp.c:4913
    #4 0x68fa50 in FlowWorker /home/victor/dev/suricata/src/flow-worker.c:194
    #5 0x7f0abd in TmThreadsSlotVarRun /home/victor/dev/suricata/src/tm-threads.c:128
    #6 0x7f2958 in TmThreadsSlotVar /home/victor/dev/suricata/src/tm-threads.c:585
    #7 0x7f436368e6f9 in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76f9)
    #8 0x7f4362802b5c in clone (/lib/x86_64-linux-gnu/libc.so.6+0x106b5c)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/victor/dev/suricata/src/source-nfq.c:510 NFQBypassCallback
Thread T5 (W#04) created by T0 (Suricata-Main) here:
    #0 0x7f4364ff2253 in pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.2+0x36253)
    #1 0x7f9c48 in TmThreadSpawn /home/victor/dev/suricata/src/tm-threads.c:1843
    #2 0x8da7c0 in RunModeSetIPSAutoFp /home/victor/dev/suricata/src/util-runmodes.c:519
    #3 0x73e3ff in RunModeIpsNFQAutoFp /home/victor/dev/suricata/src/runmode-nfq.c:74
    #4 0x7503fa in RunModeDispatch /home/victor/dev/suricata/src/runmodes.c:382
    #5 0x7e5cb3 in main /home/victor/dev/suricata/src/suricata.c:2547
    #6 0x7f436271c82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
regit pushed a commit that referenced this pull request May 20, 2020
Fixes runs with --enable-debug-validation. The target did not init a
packet pool, so for a tunnel packet would try to get a packet from
an uninitialized pool. In non-debug mode, this silently works by
falling back to a packet from alloc.

    (gdb) bt
    #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
    #1  0x00007ffff35a6801 in __GI_abort () at abort.c:79
    #2  0x00007ffff359639a in __assert_fail_base (fmt=0x7ffff371d7d8 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x555557fe7260 "!(pool->initialized == 0)",
        file=file@entry=0x555557fe7220 "tmqh-packetpool.c", line=line@entry=253, function=function@entry=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:92
    #3  0x00007ffff3596412 in __GI___assert_fail (assertion=0x555557fe7260 "!(pool->initialized == 0)", file=0x555557fe7220 "tmqh-packetpool.c", line=253,
        function=0x555557fe7500 <__PRETTY_FUNCTION__.21181> "PacketPoolGetPacket") at assert.c:101
    #4  0x00005555577e24be in PacketPoolGetPacket () at tmqh-packetpool.c:253
    #5  0x0000555556914ecd in PacketGetFromQueueOrAlloc () at decode.c:183
    #6  0x00005555569161e1 in PacketTunnelPktSetup (tv=0x555559863980 <tv>, dtv=0x614000068e40, parent=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=72, proto=DECODE_TUNNEL_IPV4) at decode.c:286
    #7  0x00005555569de694 in DecodeIPv4inIPv6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", plen=72) at decode-ipv6.c:59
    #8  0x00005555569e60b5 in DecodeIPV6ExtHdrs (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc470 "LL", len=112) at decode-ipv6.c:522
    #9  0x00005555569e846f in DecodeIPV6 (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-ipv6.c:641
    #10 0x0000555556a032f9 in DecodeRaw (tv=0x555559863980 <tv>, dtv=0x614000068e40, p=0x61e0000fc080, pkt=0x61e0000fc420 "cLL", len=255) at decode-raw.c:70
    #11 0x0000555557659ba8 in DecodePcapFile (tv=0x555559863980 <tv>, p=0x61e0000fc080, data=0x614000068e40) at source-pcap-file.c:412
    #12 0x0000555556573401 in LLVMFuzzerTestOneInput (data=0x613000000047 "\241\262\315\064", size=339) at tests/fuzz/fuzz_sigpcap.c:158
    #13 0x0000555557a4dc66 in main (argc=2, argv=0x7fffffffdfa8) at tests/fuzz/onefile.c:51

That line:

    BUG_ON(pool->initialized == 0);
regit pushed a commit that referenced this pull request May 20, 2020
In case of bad IPv4, TCP or UDP, the per packet ip4vars/tcpvars/udpvar
structures would not be cleaned up because the cleanup depends on the
'header' pointer being set, but the error handling would unset that.

This could mean these structures were already filled with values before
the error was detected. As packets were recycled, the next packet decoding
would use this unclean structure.

To make things worse these structures are part of unions. IPv4/IPv6 and
TCP/ICMPv4/ICMPv6 share the same memory location.

LibFuzzer+UBSAN found this both locally and in Oss-Fuzz:

decode-ipv6.c:654:9: runtime error: load of value 6, which is not a valid value for type 'bool'
    #0 0x6146f0 in DecodeIPV6 /src/suricata/src/decode-ipv6.c:654:9
    #1 0x617e96 in DecodeNull /src/suricata/src/decode-null.c:70:13
    #2 0x9dd8a4 in DecodePcapFile /src/suricata/src/source-pcap-file.c:412:9
    #3 0x4c8ed2 in LLVMFuzzerTestOneInput /src/suricata/src/tests/fuzz/fuzz_sigpcap.c:158:25
    #4 0x457e51 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:556:15
    #5 0x457575 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:470:3
    #6 0x459917 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:698:19
    #7 0x45a6a5 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:830:5
    #8 0x448728 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:824:6
    #9 0x472552 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:19:10
    #10 0x7ff0d097b82f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2082f)
    #11 0x41bde8 in _start (/out/fuzz_sigpcap+0x41bde8)

Bug: OISF#3496
regit pushed a commit that referenced this pull request Oct 13, 2023
Make sure to first close all ports before freeing device mempools.

Thread 1 "Suricata-Main" received signal SIGSEGV, Segmentation fault.
0x00007ffff456a3fb in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so
(gdb) bt
 #0  0x00007ffff456a3fb in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so
 #1  0x00007ffff469a948 in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so
 #2  0x00007ffff45606aa in ?? () from /usr/lib/x86_64-linux-gnu/dpdk/pmds-20.0/librte_pmd_mlx5.so
 #3  0x00007ffff6d4ed8d in rte_eth_dev_close () from /usr/lib/x86_64-linux-gnu/librte_ethdev.so.20.0
 #4  0x000000000055fc4c in DPDKCloseDevice (ldev=ldev@entry=0xe3a400) at util-dpdk.c:53
 #5  0x000000000055f4eb in LiveDeviceListClean () at util-device.c:331
 #6  0x00000000005511c8 in GlobalsDestroy (suri=<optimized out>) at suricata.c:381
 #7  0x0000000000550a76 in SuricataMain (argc=<optimized out>, argv=<optimized out>) at suricata.c:3059
 #8  0x00007ffff6a24083 in __libc_start_main (main=0x54cca0 <main>, argc=8, argv=0x7fffffffe4c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffe4b8) at ../csu/libc-start.c:308
 #9  0x000000000054cbde in _start ()

Bug: OISF#5619.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants