-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Zoom & LINECALL UDP traffic not detecting in driver but detected in nDPI reader #144
Comments
Don't use "cat /proc/net/xt_ndpi/flows". Use "ndpi_flow_dump -m flows -s" |
Thanks for the quick response.. Will perform the test once more and use ndpi_flow_dump command to print the flows.. We actually enabled the debug and flows in nDPI driver only after we saw that the firewall logs were not getting printed for the UDP flows of Zoom and Line. Is there any other debug that we can enable to get more logs, so that it would helpful in debugging this flow |
So we did 2 tests yesterday
|
I fixed the bug due to missing ndpi_detection_giveup() call ce5cca1. |
@vel21ripn, speaking only of the upstream-nDPI, if you find the word "cache" in the confidence description it means some kind of LRU has been used. |
IMHO Commit d238ff6 must be resolve this bug. |
Fixed cf017cc |
Thankyou for the quick fix. I will get back to you on Monday as I am currently travelling and don't have access to a setup to test the same.. |
Zoom is detecting properly now.. Thankyou Linecall seems to still have some issues.. Also I noticed you added 1 more fix from the initial fix.. Will add that in also in the testing.. Once I gather a proper pcap will keep you updated |
For Line we are seeing the behavior to be not so consistent in the driver, but the ndpiReader is always detecting the same. Attached pcap has a linecall where the UDP flow is from 172.8.0.100 to 147.92.169.21 Following filter can be used in Wireshark Below is the nDPI reader output.. The flow in question is below.. UDP 172.8.0.100:63719 <-> 147.92.169.21:16673
The driver ndpi_flow_dump output is as follows
Please let me know if there is any other output that we can provide to help debug the same |
I will try to fix this problem. An error can be reproduced on a short fragment:
|
We need change parameters for module xt_ndpi "max_unk_udp=32 max_unk_tcp=32 max_unk_other=32"
|
Thanks @vel21ripn . We tested this change and it is detecting Linecall now.. I feel based on our earlier tests also, analyzing 32 packets should be sufficient to detect line. I am marking this issue as fixed
|
@vel21ripn So we were retesting zoom and it stopped getting detected, while nDPIreader shows it properly in all the pcaps. We have narrowed it down to when we add the 'max_unk_udp=32 max_unk_tcp=32 max_unk_other=32" during driver load time to increase the packets which are traversed for the line, is when zoom stops getting detected. Can you please check.. We performed 4 runs.. Below are the results modprobe xt_ndpi max_unk_udp=32 max_unk_tcp=32 max_unk_other=32 ndpi_enable_flow=1 ndpi_flow_limit=500 -> Does not detect zoom. it detect Line fine. Run 2 -> modprobe xt_ndpi max_unk_udp=32 max_unk_tcp=32 max_unk_other=32 -> Does not detect zoom. it detect Line fine. Run 3 -> modprobe xt_ndpi ndpi_enable_flow=1 ndpi_flow_limit=500 -> It detects zoom but does not detect Line Run 4 -> modprobe xt_ndpi -> It detects zoom but does not detect Line We also checked with the latest tree also which had a commit for improved zoom detection. |
Can you give me a sample of the traffic that shows the error? |
Attached pcap. I trimmed the tcpdump based on the following filter ip.src == 144.195.22.81 || ip.dst == 144.195.22.81 and captured the first 10000 packets as the call was more than 10 minutes long.. Hopefully the behavior remains the same when simulated with those parameters (max_unk_udp=32 max_unk_tcp=32 max_unk_other=32).. if not we will capture one more dump. Below is the ndpi reader output.
|
Please check commit 59c0a10 |
Thankyou... Will check with that fix |
Thanks @vel21ripn .. We tested this and it detects both zoom and linecall. Closing the issue |
@vel21ripn
We recently moved from a Feb 2022 nDPI flow_info-4 base code to the latest flow_info-4 base code and looks like zoom UDP traffic detection is not working. We are able to detect TLS.Zoom which is TCP, DNS.Zoom. The same is true for LINECALL traffic
We ran the pcap with the nDPI reader and those detect the zoom traffic properly. The nDPI driver flows still show it as Unknown. I noticed that a similar issue was mentioned for Mining protocol back in May ( #132 ) and it was fixed. Not sure if the UDP path has some issue
iptables command that we are using to track nDPI flows
iptables -t mangle -I FORWARD 1 -m ndpi --proto all -j ACCEPT
Was not sure if we needed any new command with the addition of --inprogress and --clevel. It did not seem necessary but just thought that I would check with you on the same.
Attached is the pcap
zoom_audio_call_new.zip
root@kickseed:~/pcaps/nDPI/example# ./ndpiReader -i /root/pcaps/zoom_audio_call_new.pcap
Using nDPI (4.5.0-4208-dcff3ef8) [1 thread(s)]
Using libgcrypt version 1.8.6internal
Reading packets from pcap file /root/pcaps/zoom_audio_call_new.pcap...
Running thread 0...
nDPI Memory statistics:
nDPI Memory (once): 36.86 KB
Flow Memory (per flow): 912 B
Actual Memory: 38.47 MB
Peak Memory: 38.47 MB
Setup Time: 27 msec
Packet Processing Time: 71 msec
Traffic statistics:
Ethernet bytes: 36331113 (includes ethernet CRC/IFC/trailer)
Discarded bytes: 5056
IP packets: 129901 of 130000 packets total
IP bytes: 33213489 (avg pkt size 255 bytes)
Unique flows: 168
TCP Packets: 16572
UDP Packets: 113299
VLAN Packets: 0
MPLS Packets: 0
PPPoE Packets: 0
Fragmented Packets: 0
Max Packet size: 1480
Packet Len < 64: 11328
Packet Len 64-128: 33606
Packet Len 128-256: 22172
Packet Len 256-1024: 60096
Packet Len 1024-1500: 2699
Packet Len > 1500: 0
nDPI throughput: 1.83 M pps / 3.81 Gb/sec
Analysis begin: 17/Nov/2022 04:34:24
Analysis end: 17/Nov/2022 05:02:29
Traffic throughput: 77.08 pps / 168.41 Kb/sec
Traffic duration: 1685.375 sec
Guessed flow protos: 47
DPI Packets (TCP): 822 (8.30 pkts/flow)
DPI Packets (UDP): 240 (4.00 pkts/flow)
DPI Packets (other): 9 (1.00 pkts/flow)
Confidence: Unknown 42 (flows)
Confidence: Match by port 1 (flows)
Confidence: DPI (partial cache) 3 (flows)
Confidence: DPI 122 (flows)
Detected protocols:
Unknown packets: 604 bytes: 57788 flows: 42
DNS packets: 42 bytes: 5477 flows: 17
HTTP packets: 42 bytes: 5594 flows: 4
NTP packets: 2 bytes: 180 flows: 1
SMBv1 packets: 2 bytes: 486 flows: 1
OCSP packets: 10 bytes: 1703 flows: 2
STUN packets: 40 bytes: 3440 flows: 2
ICMP packets: 30 bytes: 3415 flows: 9
TLS packets: 1093 bytes: 115821 flows: 8
WindowsUpdate packets: 112 bytes: 24938 flows: 8
Zoom packets: 126525 bytes: 32368781 flows: 30
Microsoft packets: 1143 bytes: 500441 flows: 35
Microsoft365 packets: 227 bytes: 121377 flows: 8
Azure packets: 29 bytes: 4048 flows: 1
Protocol statistics:
Safe 642903 bytes
Acceptable 32512312 bytes
Dangerous 486 bytes
Unrated 57788 bytes
Risk stats [found 34 (20.2 %) flows with risks]:
TLS (probably) Not Carrying HTTPS 5 [14.7 %]
Unsafe Protocol 1 [ 2.9 %]
TLS Cert Validity Too Long 4 [11.8 %]
Error Code 12 [35.3 %]
Unidirectional Traffic 12 [35.3 %]
the fklow
root@kickseed:~/pcaps/nDPI/example# ./ndpiReader -i /root/pcaps/zoom_audio_call_new.pcap -v 2 -v 3 | grep -i "Zoom" | grep -v "DNS.Zoom" | grep -v "TLS.Zoom" | grep -v "91/TLS"
Reading packets from pcap file /root/pcaps/zoom_audio_call_new.pcap...
Zoom packets: 126525 bytes: 32368781 flows: 30
Attached screenshot after we enabled the nDPI flows . The zoom traffic IP and port for this pcap and screenshot
206.247.74.82:8801
Please let me know if you require any additional information.
We are going to retry this from the older Feb 2022 code with just an updated Zoom IP database to see if it works. Will get back to you what we find.
The text was updated successfully, but these errors were encountered: