Implement multi-protocol detection #4

lucaderi · 2015-05-10T21:00:16Z

Ability to return multi-protocol detection (e.g. IP, UDP, DNS, Twitter) instead of just Twitter for a DNS query for www.twitter.com

jnicholls · 2015-06-25T00:17:58Z

+1

kong156 · 2015-06-25T22:17:14Z

How many revisions are you planning between 1.6 and 2.0? :)) It is mandatory to split nDPI "protocols" into it's basics. This is a must for QoS classifiers. Also it would be nice if you could collapse or expand these basic protocols related to the main services (Youtube, Skype, Facebook, Tweeter, etc.) and maybe splitting at L7 is enough (for example Google may be presented as http/s, smtp, imap, pop3, dns and so on, Youtube as http/s, quic, dns, ...) More than that, It would be nice if you could present http/s into more specific services like google drive, gmail, google+, amazon S3...

lucaderi · 2015-06-26T15:12:10Z

@kong156 I know that but we need time and help. Would you be willing to contribute perhaps?

kong156 · 2015-06-26T15:26:10Z

Sorry, but if I could I think that already would have happened. Unfortunately I'm not a programmer, just a humble user of your wonderful piece of software.

lucaderi · 2015-07-01T16:16:23Z

Implemented in 7f807b8

Reported by oss-fuzz: ``` ==685288==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000687 (pc 0x0000005aba64 bp 0x7ffe3f29f510 sp 0x7ffe3f29f400 T0) ==685288==The signal is caused by a READ memory access. SCARINESS: 20 (wild-addr-read) #0 0x5aba64 in quic_len ndpi/src/lib/protocols/quic.c:203:12 ntop#1 0x5aba64 in decrypt_initial_packet ndpi/src/lib/protocols/quic.c:993:16 ntop#2 0x5aba64 in get_clear_payload ndpi/src/lib/protocols/quic.c:1302:21 ntop#3 0x5aba64 in ndpi_search_quic ndpi/src/lib/protocols/quic.c:1658:19 ntop#4 0x579f00 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4683:6 ntop#5 0x57abe6 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0 ntop#6 0x583b2c in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:5545:15 ntop#7 0x55e75e in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:30:3 [...] ```

Reported by oss-fuzz: ``` ==685288==ERROR: AddressSanitizer: SEGV on unknown address 0x61a100000687 (pc 0x0000005aba64 bp 0x7ffe3f29f510 sp 0x7ffe3f29f400 T0) ==685288==The signal is caused by a READ memory access. SCARINESS: 20 (wild-addr-read) #0 0x5aba64 in quic_len ndpi/src/lib/protocols/quic.c:203:12 #1 0x5aba64 in decrypt_initial_packet ndpi/src/lib/protocols/quic.c:993:16 #2 0x5aba64 in get_clear_payload ndpi/src/lib/protocols/quic.c:1302:21 #3 0x5aba64 in ndpi_search_quic ndpi/src/lib/protocols/quic.c:1658:19 #4 0x579f00 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4683:6 #5 0x57abe6 in ndpi_check_flow_func ndpi/src/lib/ndpi_main.c:0 #6 0x583b2c in ndpi_detection_process_packet ndpi/src/lib/ndpi_main.c:5545:15 #7 0x55e75e in LLVMFuzzerTestOneInput ndpi/fuzz/fuzz_process_packet.c:30:3 [...] ```

Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280 ``` ==263603==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9 ntop#1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7 ntop#2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7 ntop#3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13 ntop#4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2 ntop#5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5 ntop#6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6 ```

Found by oss-fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=44280 ``` ==263603==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x592478 in ndpi_is_printable_string ndpi/src/lib/ndpi_utils.c:2200:9 #1 0x5b047c in processCertificateElements ndpi/src/lib/protocols/tls.c:400:7 #2 0x5ac880 in processCertificate ndpi/src/lib/protocols/tls.c:790:7 #3 0x5c3a32 in processTLSBlock ndpi/src/lib/protocols/tls.c:844:13 #4 0x5c2c61 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:973:2 #5 0x5c117d in ndpi_search_tls_wrapper ndpi/src/lib/protocols/tls.c:2367:5 #6 0x552a50 in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:4792:6 ```

``` ==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48 READ of size 1 at 0x60400008957f thread T0 #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18 ntop#1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 ntop#2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 ntop#3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16 ntop#4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98) 0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f) allocated by thread T0 here: ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036

``` ==38674==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60400008957f at pc 0x5653fcee6434 bp 0x7ffe9b554b50 sp 0x7ffe9b554b48 READ of size 1 at 0x60400008957f thread T0 #0 0x5653fcee6433 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:2050:18 #1 0x5653fce9fc0d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x5653fcea03eb in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7fe71dc3d0b2 in __libc_start_main /build/glibc-YbNSs7/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x5653fcddf67d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x58a67d) (BuildId: 525418a27e8c37d6c492cc3220e0e97809c40f98) 0x60400008957f is located 0 bytes to the right of 47-byte region [0x604000089550,0x60400008957f) allocated by thread T0 here: ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=45036

``` ==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8 READ of size 4 at 0x60300000578c thread T0 #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16 ntop#1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 ntop#2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 ntop#3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16 ntop#4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37) 0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d) ```

``` ==20492==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60300000578c at pc 0x55c47455e3ea bp 0x7ffc62ca1eb0 sp 0x7ffc62ca1ea8 READ of size 4 at 0x60300000578c thread T0 #0 0x55c47455e3e9 in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/example/reader_util.c:1840:16 #1 0x55c47451b9cd in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:107:7 #2 0x55c47451c1ab in main /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:179:17 #3 0x7f661b50e0b2 in __libc_start_main /build/glibc-sMfBJT/glibc-2.31/csu/../csu/libc-start.c:308:16 #4 0x55c47445b54d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader_with_main+0x61054d) (BuildId: eba4bb4cd43b7101e4f0028ec0fb79087bae0e37) 0x60300000578d is located 0 bytes to the right of 29-byte region [0x603000005770,0x60300000578d) ```

``` ==19255==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f515bb3bf80 at pc 0x55796e01394a bp 0x7fff4fb5c050 sp 0x7fff4fb5b7e0 WRITE of size 58 at 0x7f515bb3bf80 thread T0 #0 0x55796e013949 in scanf_common(void*, int, bool, char const*, __va_list_tag*) asan_interceptors.cpp.o ntop#1 0x55796e0147df in __isoc99_sscanf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x77f7df) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) ntop#2 0x55796e0fc74a in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2771:13 ntop#3 0x55796e0fb029 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4411:16 ntop#4 0x55796e103738 in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4901:8 ntop#5 0x55796e0ca96d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3 ntop#6 0x55796dfd78e0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x7428e0) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) ntop#7 0x55796dfc0e93 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x72be93) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) ntop#8 0x55796dfc6d96 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x731d96) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) ntop#9 0x55796dff1672 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x75c672) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) ntop#10 0x7f515df19082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16 ntop#11 0x55796dfbbb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x726b0d) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) Address 0x7f515bb3bf80 is located in stack of thread T0 at offset 128 in frame #0 0x55796e0fb977 in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2703 This frame has 4 object(s): [32, 36) 'pin' (line 2705) [48, 64) 'pin6' (line 2706) [80, 96) 'd' (line 2769) [112, 128) 'tail' (line 2770) <== Memory access at offset 128 overflows this variable HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-buffer-overflow asan_interceptors.cpp.o in scanf_common(void*, int, bool, char const*, __va_list_tag*) Shadow bytes around the buggy address: ```

``` ==19255==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f515bb3bf80 at pc 0x55796e01394a bp 0x7fff4fb5c050 sp 0x7fff4fb5b7e0 WRITE of size 58 at 0x7f515bb3bf80 thread T0 #0 0x55796e013949 in scanf_common(void*, int, bool, char const*, __va_list_tag*) asan_interceptors.cpp.o #1 0x55796e0147df in __isoc99_sscanf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x77f7df) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #2 0x55796e0fc74a in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2771:13 #3 0x55796e0fb029 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4411:16 #4 0x55796e103738 in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4901:8 #5 0x55796e0ca96d in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3 #6 0x55796dfd78e0 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x7428e0) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #7 0x55796dfc0e93 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x72be93) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #8 0x55796dfc6d96 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x731d96) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #9 0x55796dff1672 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x75c672) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) #10 0x7f515df19082 in __libc_start_main /build/glibc-BHL3KM/glibc-2.31/csu/../csu/libc-start.c:308:16 #11 0x55796dfbbb0d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols+0x726b0d) (BuildId: a88601afb2c538ead3968648f39b9aa4da53427c) Address 0x7f515bb3bf80 is located in stack of thread T0 at offset 128 in frame #0 0x55796e0fb977 in ndpi_add_host_ip_subprotocol /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2703 This frame has 4 object(s): [32, 36) 'pin' (line 2705) [48, 64) 'pin6' (line 2706) [80, 96) 'd' (line 2769) [112, 128) 'tail' (line 2770) <== Memory access at offset 128 overflows this variable HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork (longjmp and C++ exceptions *are* supported) SUMMARY: AddressSanitizer: stack-buffer-overflow asan_interceptors.cpp.o in scanf_common(void*, int, bool, char const*, __va_list_tag*) Shadow bytes around the buggy address: ```

Fix an use-of-uninitialized-value error ``` ==28646==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55a9b7a3dea2 in ndpi_patricia_lookup /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ndpi_patricia.c:739:8 ntop#1 0x55a9b7442cac in add_to_ptree /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2588:10 ntop#2 0x55a9b7469290 in ndpi_add_ip_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4161:12 ntop#3 0x55a9b746cf14 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4297:11 ntop#4 0x55a9b74842ae in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4944:8 ntop#5 0x55a9b7424706 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3 [...] ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63754

Fix an use-of-uninitialized-value error ``` ==28646==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x55a9b7a3dea2 in ndpi_patricia_lookup /home/ivan/svnrepos/nDPI/src/lib/third_party/src/ndpi_patricia.c:739:8 #1 0x55a9b7442cac in add_to_ptree /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:2588:10 #2 0x55a9b7469290 in ndpi_add_ip_risk_mask /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4161:12 #3 0x55a9b746cf14 in ndpi_handle_rule /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4297:11 #4 0x55a9b74842ae in ndpi_load_protocols_file_fd /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:4944:8 #5 0x55a9b7424706 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_filecfg_protocols.c:38:3 [...] ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=63754

Fix a memory leak ``` ==97697==ERROR: LeakSanitizer: detected memory leaks Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01) ntop#1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25 ntop#2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13 ntop#3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46 ntop#4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5 ntop#5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11 ntop#6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5 ntop#7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3 ntop#8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4 ntop#9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9 ntop#10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5 ntop#11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22 ntop#12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31 ntop#13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10 ntop#14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7 [...] SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s). ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564

Fix a memory leak ``` ==97697==ERROR: LeakSanitizer: detected memory leaks Direct leak of 16 byte(s) in 1 object(s) allocated from: #0 0x55a6967cfa7e in malloc (/home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader+0x701a7e) (BuildId: c7124999fa1ccc54346fa7bd536d8eab88c3ea01) #1 0x55a696972ab5 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25 #2 0x55a696972da0 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13 #3 0x55a696b7658d in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2394:46 #4 0x55a696b86e81 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:897:5 #5 0x55a696b80649 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1262:11 #6 0x55a696b67a57 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2751:5 #7 0x55a696b67758 in switch_to_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1408:3 #8 0x55a696c47810 in stun_search_again /home/ivan/svnrepos/nDPI/src/lib/protocols/stun.c:422:4 #9 0x55a6968a22af in ndpi_process_extra_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7247:9 #10 0x55a6968acd6f in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7746:5 #11 0x55a6968aba3f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8013:22 #12 0x55a69683d30e in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1723:31 #13 0x55a69683d30e in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2440:10 #14 0x55a69680f08f in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ndpi_reader.c:135:7 [...] SUMMARY: AddressSanitizer: 16 byte(s) leaked in 1 allocation(s). ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=64564

``` ==53992==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511000007e04 at pc 0x555da2165fd0 bp 0x7ffddf7e3990 sp 0x7ffddf7e3988 READ of size 2 at 0x511000007e04 thread T0 #0 0x555da2165fcf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2384:50 ntop#1 0x555da217c31f in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5 ntop#2 0x555da2176720 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1273:11 ntop#3 0x555da215a628 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2883:5 ntop#4 0x555da1e95c30 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6720:6 ntop#5 0x555da1e969f3 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6756:10 ntop#6 0x555da1e96394 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6789:12 ntop#7 0x555da1ea7991 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7929:15 ntop#8 0x555da1ea547f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8104:22 ntop#9 0x555da1de137f in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1721:31 ntop#10 0x555da1de137f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2438:1 ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65362

``` ==53992==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x511000007e04 at pc 0x555da2165fd0 bp 0x7ffddf7e3990 sp 0x7ffddf7e3988 READ of size 2 at 0x511000007e04 thread T0 #0 0x555da2165fcf in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2384:50 #1 0x555da217c31f in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5 #2 0x555da2176720 in ndpi_search_tls_udp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1273:11 #3 0x555da215a628 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2883:5 #4 0x555da1e95c30 in check_ndpi_detection_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6720:6 #5 0x555da1e969f3 in check_ndpi_udp_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6756:10 #6 0x555da1e96394 in ndpi_check_flow_func /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:6789:12 #7 0x555da1ea7991 in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:7929:15 #8 0x555da1ea547f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8104:22 #9 0x555da1de137f in packet_processing /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:1721:31 #10 0x555da1de137f in ndpi_workflow_process_packet /home/ivan/svnrepos/nDPI/fuzz/../example/reader_util.c:2438:1 ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=65362

``` Direct leak of 12 byte(s) in 1 object(s) allocated from: #0 0x55779e1a46ff in malloc (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x8706ff) (BuildId: 14c2fc626744710d49d652ea1c5bbb24a8cbab4f) ntop#1 0x55779e2120c7 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/ndpiReader.c:298:10 ntop#2 0x55779e5fa215 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25 ntop#3 0x55779e5fa500 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13 ntop#4 0x55779e42153c in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2554:46 ntop#5 0x55779e4359a1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5 ntop#6 0x55779e432de7 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1097:2 ntop#7 0x55779e4133f9 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2913:5 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67250

``` Direct leak of 12 byte(s) in 1 object(s) allocated from: #0 0x55779e1a46ff in malloc (/home/ivan/svnrepos/nDPI/example/ndpiReader+0x8706ff) (BuildId: 14c2fc626744710d49d652ea1c5bbb24a8cbab4f) #1 0x55779e2120c7 in ndpi_malloc_wrapper /home/ivan/svnrepos/nDPI/example/ndpiReader.c:298:10 #2 0x55779e5fa215 in ndpi_malloc /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:60:25 #3 0x55779e5fa500 in ndpi_strdup /home/ivan/svnrepos/nDPI/src/lib/ndpi_memory.c:113:13 #4 0x55779e42153c in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2554:46 #5 0x55779e4359a1 in processTLSBlock /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:908:5 #6 0x55779e432de7 in ndpi_search_tls_tcp /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1097:2 #7 0x55779e4133f9 in ndpi_search_tls_wrapper /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2913:5 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67250

``` AddressSanitizer:DEADLYSIGNAL ================================================================= ==29723==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x562910b70ddb bp 0x7ffcb22c5b70 sp 0x7ffcb22c5a80 T0) ==29723==The signal is caused by a READ memory access. ==29723==Hint: address points to the zero page. #0 0x562910b70ddb in binary_fuse16_contain /home/ivan/svnrepos/nDPI/src/lib/./third_party/include/binaryfusefilter.h:492:8 ntop#1 0x562910b70bbe in ndpi_bitmap64_isset /home/ivan/svnrepos/nDPI/src/lib/ndpi_bitmap64.c:178:10 ntop#2 0x562910788fd3 in ndpi_domain_classify_longest_prefix /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:261:5 ntop#3 0x56291078940e in ndpi_domain_classify_contains /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:291:9 ntop#4 0x56291069a392 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ds_domain_classify.cpp:52:5 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67369 See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67372

``` AddressSanitizer:DEADLYSIGNAL ================================================================= ==29723==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x562910b70ddb bp 0x7ffcb22c5b70 sp 0x7ffcb22c5a80 T0) ==29723==The signal is caused by a READ memory access. ==29723==Hint: address points to the zero page. #0 0x562910b70ddb in binary_fuse16_contain /home/ivan/svnrepos/nDPI/src/lib/./third_party/include/binaryfusefilter.h:492:8 #1 0x562910b70bbe in ndpi_bitmap64_isset /home/ivan/svnrepos/nDPI/src/lib/ndpi_bitmap64.c:178:10 #2 0x562910788fd3 in ndpi_domain_classify_longest_prefix /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:261:5 #3 0x56291078940e in ndpi_domain_classify_contains /home/ivan/svnrepos/nDPI/src/lib/ndpi_domain_classify.c:291:9 #4 0x56291069a392 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_ds_domain_classify.cpp:52:5 ``` Found by oss-fuzz See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67369 See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67372

The bug is triggered when `pe_offset == (u_int32_t)-1` ``` AddressSanitizer:DEADLYSIGNAL ================================================================= ==23719==ERROR: AddressSanitizer: SEGV on unknown address 0x5081000002b3 (pc 0x55c69274ac72 bp 0x7ffffffc8e70 sp 0x7ffffffc8cc0 T0) ==23719==The signal is caused by a READ memory access. #0 0x55c69274ac72 in ndpi_search_portable_executable /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8191:7 ntop#1 0x55c69271606b in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8596:5 ntop#2 0x55c69270f58f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8629:22 ntop#3 0x55c6926a07e7 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5 ntop#4 0x55c6925a79b6 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x64e9b6) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) ntop#5 0x55c692590d48 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x637d48) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) ntop#6 0x55c69259685a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x63d85a) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) ntop#7 0x55c6925c0e02 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x667e02) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) ntop#8 0x7f8e99793082 in __libc_start_main /build/glibc-wuryBv/glibc-2.31/csu/../csu/libc-start.c:308:16 ntop#9 0x55c69258baed in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x632aed) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67881

The bug is triggered when `pe_offset == (u_int32_t)-1` ``` AddressSanitizer:DEADLYSIGNAL ================================================================= ==23719==ERROR: AddressSanitizer: SEGV on unknown address 0x5081000002b3 (pc 0x55c69274ac72 bp 0x7ffffffc8e70 sp 0x7ffffffc8cc0 T0) ==23719==The signal is caused by a READ memory access. #0 0x55c69274ac72 in ndpi_search_portable_executable /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8191:7 #1 0x55c69271606b in ndpi_internal_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8596:5 #2 0x55c69270f58f in ndpi_detection_process_packet /home/ivan/svnrepos/nDPI/src/lib/ndpi_main.c:8629:22 #3 0x55c6926a07e7 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet.c:24:5 #4 0x55c6925a79b6 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x64e9b6) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) #5 0x55c692590d48 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x637d48) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) #6 0x55c69259685a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x63d85a) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) #7 0x55c6925c0e02 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x667e02) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) #8 0x7f8e99793082 in __libc_start_main /build/glibc-wuryBv/glibc-2.31/csu/../csu/libc-start.c:308:16 #9 0x55c69258baed in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_process_packet+0x632aed) (BuildId: ec46c60ec7e03ebfb3d825bd6308d0a8d6e9803b) ``` Found by oss-fuzzer See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=67881

``` ==22779==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0900701020 at pc 0x555bcd2a6f02 bp 0x7ffe3ba5e790 sp 0x7ffe3ba5e788 READ of size 1 at 0x7f0900701020 thread T0 #0 0x555bcd2a6f01 in shoco_decompress /home/ivan/svnrepos/nDPI/src/lib/third_party/src/shoco.c:184:26 ntop#1 0x555bcd2a4018 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:18:3 ntop#2 0x555bcd1aa816 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x4f816) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) ntop#3 0x555bcd193be8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x38be8) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) ntop#4 0x555bcd1996fa in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3e6fa) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) ntop#5 0x555bcd1c3c92 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x68c92) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) ntop#6 0x7f090257a082 in __libc_start_main /build/glibc-e2p3jK/glibc-2.31/csu/../csu/libc-start.c:308:16 ntop#7 0x555bcd18e96d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3396d) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) Address 0x7f0900701020 is located in stack of thread T0 at offset 4128 in frame #0 0x555bcd2a3d97 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:5 This frame has 4 object(s): [32, 4128) 'out' (line 9) <== Memory access at offset 4128 overflows this variable [4256, 8352) 'orig' (line 9) ``` Found by oss-fuzzer. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68211

``` ==22779==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f0900701020 at pc 0x555bcd2a6f02 bp 0x7ffe3ba5e790 sp 0x7ffe3ba5e788 READ of size 1 at 0x7f0900701020 thread T0 #0 0x555bcd2a6f01 in shoco_decompress /home/ivan/svnrepos/nDPI/src/lib/third_party/src/shoco.c:184:26 #1 0x555bcd2a4018 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:18:3 #2 0x555bcd1aa816 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x4f816) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) #3 0x555bcd193be8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x38be8) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) #4 0x555bcd1996fa in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3e6fa) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) #5 0x555bcd1c3c92 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x68c92) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) #6 0x7f090257a082 in __libc_start_main /build/glibc-e2p3jK/glibc-2.31/csu/../csu/libc-start.c:308:16 #7 0x555bcd18e96d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco+0x3396d) (BuildId: c54d1c32163c9937e06f62127348ae6bd26d9309) Address 0x7f0900701020 is located in stack of thread T0 at offset 4128 in frame #0 0x555bcd2a3d97 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_alg_shoco.cpp:5 This frame has 4 object(s): [32, 4128) 'out' (line 9) <== Memory access at offset 4128 overflows this variable [4256, 8352) 'orig' (line 9) ``` Found by oss-fuzzer. See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68211

The new values has been checked against the ones reported by Wireshark. Found while fixing a Use-of-uninitialized-value error reported by oss-fuzz ``` ==7582==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x5a6549abc368 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:1762:10 ntop#1 0x5a6549ab88a0 in processClientServerHello ndpi/src/lib/protocols/tls.c:2863:10 ntop#2 0x5a6549ac1452 in processTLSBlock ndpi/src/lib/protocols/tls.c:909:5 ntop#3 0x5a6549abf588 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:1098:2 ntop#4 0x5a65499c53ec in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:7215:6 ``` See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68449&q=ndpi&can=1&sort=-id

The new values has been checked against the ones reported by Wireshark. Found while fixing a Use-of-uninitialized-value error reported by oss-fuzz ``` ==7582==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x5a6549abc368 in ndpi_compute_ja4 ndpi/src/lib/protocols/tls.c:1762:10 #1 0x5a6549ab88a0 in processClientServerHello ndpi/src/lib/protocols/tls.c:2863:10 #2 0x5a6549ac1452 in processTLSBlock ndpi/src/lib/protocols/tls.c:909:5 #3 0x5a6549abf588 in ndpi_search_tls_tcp ndpi/src/lib/protocols/tls.c:1098:2 #4 0x5a65499c53ec in check_ndpi_detection_func ndpi/src/lib/ndpi_main.c:7215:6 ``` See: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=68449&q=ndpi&can=1&sort=-id

@catenacyber

``` SCARINESS: 12 (1-byte-read-heap-buffer-overflow) #0 0x557f3a5b5100 in ndpi_get_host_domain /src/ndpi/src/lib/ndpi_domains.c:158:8 ntop#1 0x557f3a59b561 in ndpi_check_dga_name /src/ndpi/src/lib/ndpi_main.c:10412:17 ntop#2 0x557f3a51163a in process_chlo /src/ndpi/src/lib/protocols/quic.c:1467:7 ntop#3 0x557f3a469f4b in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_quic_get_crypto_data.c:44:7 ntop#4 0x557f3a46abc8 in NaloFuzzerTestOneInput (/out/fuzz_quic_get_crypto_data+0x4cfbc8) ``` Some notes about the leak: if the insertion into the uthash fails (because of an allocation failure), we need to free the just allocated entry. But the only way to check if the `HASH_ADD_*` failed, is to perform a new lookup: a bit costly, but we don't use that code in the fast-path. See also efb261a Credits for finding the issues to Philippe Antoine (@catenacyber) and its `nallocfuzz` fuzzing engine See: https://github.com/catenacyber/nallocfuzz See: google/oss-fuzz#9902

@catenacyber

``` SCARINESS: 12 (1-byte-read-heap-buffer-overflow) #0 0x557f3a5b5100 in ndpi_get_host_domain /src/ndpi/src/lib/ndpi_domains.c:158:8 #1 0x557f3a59b561 in ndpi_check_dga_name /src/ndpi/src/lib/ndpi_main.c:10412:17 #2 0x557f3a51163a in process_chlo /src/ndpi/src/lib/protocols/quic.c:1467:7 #3 0x557f3a469f4b in LLVMFuzzerTestOneInput /src/ndpi/fuzz/fuzz_quic_get_crypto_data.c:44:7 #4 0x557f3a46abc8 in NaloFuzzerTestOneInput (/out/fuzz_quic_get_crypto_data+0x4cfbc8) ``` Some notes about the leak: if the insertion into the uthash fails (because of an allocation failure), we need to free the just allocated entry. But the only way to check if the `HASH_ADD_*` failed, is to perform a new lookup: a bit costly, but we don't use that code in the fast-path. See also efb261a Credits for finding the issues to Philippe Antoine (@catenacyber) and his `nallocfuzz` fuzzing engine See: https://github.com/catenacyber/nallocfuzz See: google/oss-fuzz#9902

For some unclear reasons, fuzzers using pl7m create huge corpus, triggering OOM in oss-fuzz runs (where the memory RSS limit is set to 2560Mb). Example: ``` ==25340== ERROR: libFuzzer: out-of-memory (used: 2564Mb; limit: 2560Mb) To change the out-of-memory limit use -rss_limit_mb=<N> Live Heap Allocations: 2364004039 bytes in 133791 chunks; quarantined: 60662293 bytes in 3664 chunks; 176432 other chunks; total chunks: 313887; showing top 95% (at most 8 unique contexts) 1285841683 byte(s) (54%) in 2956 allocation(s) #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 ntop#1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0 ntop#2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3 ntop#3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9 ntop#4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10 ntop#5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16 ntop#6 0x56f814de7df2 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:829:7 ntop#7 0x56f814de8127 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:867:3 ntop#8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6 ntop#9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 ntop#10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 1031350683 byte(s) (43%) in 2468 allocation(s) #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 ntop#1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0 ntop#2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3 ntop#3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9 ntop#4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10 ntop#5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16 ntop#6 0x56f814de7635 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:760:19 ntop#7 0x56f814de8425 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:905:5 ntop#8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6 ntop#9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 ntop#10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 ``` See: https://oss-fuzz.com/testcase-detail/4717811415449600 See: https://oss-fuzz.com/testcase-detail/6164130982068224 Let's *try* the following workaround: set the parameter `-max-len` to 512K, to force the engine to not genereate inputs (i.e. pcap files...) larger than 512K. Right now the value used is 1MB, i.e the default, because we have file larger than 1MB in the initial seeds (i.e. `/tests/pcaps/*`). Let's hope than having smaller files lead to smaller corpus... Update pl7m code (fix a Use-of-uninitialized-value error)

For some unclear reasons, fuzzers using pl7m create huge corpus, triggering OOM in oss-fuzz runs (where the memory RSS limit is set to 2560Mb). Example: ``` ==25340== ERROR: libFuzzer: out-of-memory (used: 2564Mb; limit: 2560Mb) To change the out-of-memory limit use -rss_limit_mb=<N> Live Heap Allocations: 2364004039 bytes in 133791 chunks; quarantined: 60662293 bytes in 3664 chunks; 176432 other chunks; total chunks: 313887; showing top 95% (at most 8 unique contexts) 1285841683 byte(s) (54%) in 2956 allocation(s) #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0 #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3 #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9 #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10 #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16 #6 0x56f814de7df2 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:829:7 #7 0x56f814de8127 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:867:3 #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6 #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 1031350683 byte(s) (43%) in 2468 allocation(s) #0 0x56f814ef4bde in __interceptor_malloc /src/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3 #1 0x56f814e04416 in operator new(unsigned long) cxa_noexception.cpp:0 #2 0x56f814de6b2d in assign<unsigned char *, 0> /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1443:3 #3 0x56f814de6b2d in operator= /work/llvm-stage2/runtimes/runtimes-bins/compiler-rt/lib/fuzzer/libcxx_fuzzer_x86_64/include/c++/v1/vector:1412:9 #4 0x56f814de6b2d in fuzzer::InputCorpus::AddToCorpus(std::__Fuzzer::vector<unsigned char, std::__Fuzzer::allocator<unsigned char>> const&, unsigned long, bool, bool, bool, std::__Fuzzer::chrono::duration<long long, std::__Fuzzer::ratio<1l, 1000000l>>, std::__Fuzzer::vector<unsigned int, std::__Fuzzer::allocator<unsigned int>> const&, fuzzer::DataFlowTrace const&, fuzzer::InputInfo const*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerCorpus.h:221:10 #5 0x56f814de60e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool, bool*) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:539:16 #6 0x56f814de7635 in fuzzer::Fuzzer::MutateAndTestOne() /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:760:19 #7 0x56f814de8425 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, std::__Fuzzer::allocator<fuzzer::SizedFile>>&) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:905:5 #8 0x56f814dd6736 in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:914:6 #9 0x56f814e02c62 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10 #10 0x7fa11e2c3082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16 ``` See: https://oss-fuzz.com/testcase-detail/4717811415449600 See: https://oss-fuzz.com/testcase-detail/6164130982068224 Let's *try* the following workaround: set the parameter `-max-len` to 512K, to force the engine to not genereate inputs (i.e. pcap files...) larger than 512K. Right now the value used is 1MB, i.e the default, because we have file larger than 1MB in the initial seeds (i.e. `/tests/pcaps/*`). Let's hope than having smaller files lead to smaller corpus... Update pl7m code (fix a Use-of-uninitialized-value error)

``` SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:1812:22 ================================================================= ==97754==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ba835bde8e5 at pc 0x557ebb644241 bp 0x7ffec04b0ea0 sp 0x7ffec04b0648 WRITE of size 7 at 0x7ba835bde8e5 thread T0 #0 0x557ebb644240 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x6bf240) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ntop#1 0x557ebb645b1d in snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x6c0b1d) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ntop#2 0x557ebb749dbc in ndpi_compute_ja4 /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1812:12 ntop#3 0x557ebb7445a7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2946:10 ntop#4 0x557ebb7073c9 in process_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1397:3 ntop#5 0x557ebb6ff815 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data.c:46:7 ntop#6 0x557ebb602dcb in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x67ddcb) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ntop#7 0x557ebb5ecea8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x667ea8) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ntop#8 0x557ebb5f299a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x66d99a) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ntop#9 0x557ebb61c482 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x697482) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ntop#10 0x7fa837e27082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16 ntop#11 0x557ebb5e7b5d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x662b5d) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ```

``` SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior protocols/tls.c:1812:22 ================================================================= ==97754==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7ba835bde8e5 at pc 0x557ebb644241 bp 0x7ffec04b0ea0 sp 0x7ffec04b0648 WRITE of size 7 at 0x7ba835bde8e5 thread T0 #0 0x557ebb644240 in vsnprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x6bf240) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #1 0x557ebb645b1d in snprintf (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x6c0b1d) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #2 0x557ebb749dbc in ndpi_compute_ja4 /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:1812:12 #3 0x557ebb7445a7 in processClientServerHello /home/ivan/svnrepos/nDPI/src/lib/protocols/tls.c:2946:10 #4 0x557ebb7073c9 in process_tls /home/ivan/svnrepos/nDPI/src/lib/protocols/quic.c:1397:3 #5 0x557ebb6ff815 in LLVMFuzzerTestOneInput /home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data.c:46:7 #6 0x557ebb602dcb in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x67ddcb) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #7 0x557ebb5ecea8 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x667ea8) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #8 0x557ebb5f299a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x66d99a) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #9 0x557ebb61c482 in main (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x697482) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) #10 0x7fa837e27082 in __libc_start_main /build/glibc-LcI20x/glibc-2.31/csu/../csu/libc-start.c:308:16 #11 0x557ebb5e7b5d in _start (/home/ivan/svnrepos/nDPI/fuzz/fuzz_quic_get_crypto_data+0x662b5d) (BuildId: ce17f7c48055e1f051360bed543c1e18c05f684f) ```

lucaderi added this to the 1.6 milestone May 10, 2015

lucaderi added the enhancement label May 10, 2015

lucaderi modified the milestones: 1.6, 2.0 Jun 14, 2015

lucaderi closed this as completed Jul 1, 2015

jjdevx mentioned this issue Jun 29, 2016

ndpiReader 1.8.0 or ntopng 2.4 crashes while loading protos.txt #219

Closed

u-Map mentioned this issue Aug 16, 2016

ndpiReader Segmantaion Fault with latest compiled code #249

Closed

JeevanNailwal mentioned this issue Oct 3, 2016

ndpi binary crashing in ftp_control file. #272

Closed

subhedarnikhil mentioned this issue Mar 26, 2017

Dropbox uses TCP or UDP..???? #355

Closed

This was referenced Apr 19, 2017

ndpiReader crashing #363

Closed

ndpiReader crash while analyzing Server Hello #366

Closed

lynn1050 mentioned this issue Jul 19, 2017

ndpiReader crash while processiong the live traffic #423

Closed

This was referenced Feb 8, 2019

Fix whatsapp signature detecting. #665

Merged

whatsapp bug #663

Closed

xlb767923274 mentioned this issue Apr 24, 2019

ERROR: AddressSanitizer: heap-buffer-overflow #697

Closed

rkerur mentioned this issue Oct 1, 2019

ndpiReader crashes with latest git repo code when run with '-c mining_hosts.txt' #789

Closed

aouinizied mentioned this issue May 10, 2020

Segmentation fault using ndpiReader on live capture. #898

Closed

This was referenced Oct 13, 2023

vpp, including npdi lib, complied by asan, generate a SIGEGV signal callbak #2100

Closed

vpp generate callbak stack complied with asan which including ndpi libary. projectcalico/vpp-dataplane#643

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Implement multi-protocol detection #4

Implement multi-protocol detection #4

lucaderi commented May 10, 2015

jnicholls commented Jun 25, 2015

kong156 commented Jun 25, 2015

lucaderi commented Jun 26, 2015

kong156 commented Jun 26, 2015

lucaderi commented Jul 1, 2015

Implement multi-protocol detection #4

Implement multi-protocol detection #4

Comments

lucaderi commented May 10, 2015

jnicholls commented Jun 25, 2015

kong156 commented Jun 25, 2015

lucaderi commented Jun 26, 2015

kong156 commented Jun 26, 2015

lucaderi commented Jul 1, 2015