-
Notifications
You must be signed in to change notification settings - Fork 69
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ndpi-netfilter kernel panics #29
Comments
Hi, are you building the ndpi-netfilter with lastest version? And with the embedded ndpi library? |
I cloned the latest repo from github and followed steps from ndpi.install. |
@rightkick can you test something in iptables?
The above would make all traffic passing in the machine to be inspected by the NDPI module. Also, I want to rebuild on the same machine, is it a 64 bit or 32 bit? |
Hi @elico. I was testing on x64 with below firewall (a simple statefull one that allows only HTTP, HTTPS and DNS for browsing). All packets are passed from ndpi filtering before their state is examined.
I enabled ndpi checks on all filter chains for the same purpose of stressing the ndpi module. I will test with your rules and let you know. |
@rightkick your rules are almost the same as what I proposed. |
One question for the installation: @elico, when defining the rules you suggested, I receive the following error: It is the line with the rule: |
@rightkick I'm not the developer but ./configure should be OK. which works for the wheezy backports kernel. |
Thanx @elico. I also had no issues during build on Debian7. It is only through testing that I observed issues. I have transferred now all ndpi filtering rules at mangle table, after loopback ACCEPT. I am suspecting that putting ndpi check rule before looback ACCEPT rules might be causing loops. Below is the revised portion of firewall where ndpi rules reside, with few rules for accounting only:
It is not clear to me what --dpi_check does. When printing with iptables -t mangle -nvL ndpi-filter it is showing as protocol CHECK with zero matching packets. Other traffic (e.g HTTP) is matched and counters are increasing, as below:
any idea? |
@rightkick It's pretty simple.
The concept is that the all will be a catch all while the others will match if possible. |
Hi @elico. Ok amended the firewall accordingly, but -A ndpi-filter -m ndpi --all -j RETURN is not loading. Error given: Applying new ruleset... iptables-restore v1.4.14: unknown option "--all". |
@rightkick This is a dump from my iptables but I do not know why it's not working on your side.
I can try to run a VM and see how it all glued together. |
Hi Elico, |
@betolj The vel21ripn is not working for me... I feel so stupid. |
You can use modinfo: The dpi_check option can be used to map all packets to the ndpi-flow. |
@betolj Can it be that I have used the xtables module of one with the lib_xtndpi of another? |
The crash snapshot: |
Maybe.. I thought in this possibility too, but i dont know what's the compatibility with "project API". And in my fork, DNS queries for facebook and youtube will fail if you filter these protocols. |
So, are you receiving kernel panics in this fork? After rebuild? |
Ok. I will review the code. In my tests it has been stable. root@humberto-XPS-8300:~# cat /etc/issue root@humberto-XPS-8300:~# uname -r *mangle |
leap-router:~ # cat /etc/issue leap-router:~ # uname -r |
Hi, |
I've been testing for more then 24 hours, and seems stable. No kernel panics until now with the new firewall setup. I would be interested to pfring if it can be plugged with ndpi-netfilter, although I have not seen performance issues with libpcap. Additionally, is there a way to define and load custom protocols with ndpi-netfilter? I see an example file included at ndpi-netfilter/nDPI/example/protos.txt |
@betolj I managed to build and install the ndpi-netfilter modules on OpenSUSE leap and it seems to work as expected without any kernel crashes for the last hour. Thanks!! |
@betolj Sorry but it crashed again the machine. |
Testing several days with Debian 7 and kernel 3.18.36, and with the above mentioned firewall, has yielded stable results. |
Hi, i have kernel panics on Debian 7 with 3.18.26 mptcp patched kernel.
admin@docker-host-dev-2 ~/ndpi-netfilter $ uname -r admin@docker-host-dev-2 ~/ndpi-netfilter $ sudo modinfo ./src/xt_ndpi.ko |
I get a very similar panic ndpi_detection_process_packet with a vanilla kernel 4.4.15 on debian jessie. What information can I provide that might help resolve it? |
rightclick, I have some random crashes too, running on arm64, arm, x86 and x86_64, leading me to the conclusion that it has nothing to do with architecture, but with my firewall rules... Interesting your experience, mind to share some more information? How many days the system is running fine? How many machines are running through your firewall? With vel21ripn fork the kernel crashes on restoring iptables rules, this betolj fork randomly crashes like, in 1 hour on up system, runs 2 days straight and then crashes. Not quite getting any logic path to reproduce these errors... betolj, congratulations for the effort! |
Indeed, thanks a lot to betolj! I love this port that doesn't require any kernel patching! To reproduce this issue on-demand, I have a script that loads all ndpi protocols in the filter table like so: Then after a while (0-5 minutes), I guess because of traffic, kernel just panics, usually it mentions ndpi_detection_process_packet, at least once I saw a mention about "ftp_control". Any idea of how could we further hunt this issue? (I'm using kernel 4.4.X btw) |
Hi, following more than 20 days of stress tests I deem my setup stable. My systemdetails and firewall rules have been provided to a previous post. Seems that you need to accept traffic on local interface before you do any filtering with ndpi to avoid panics. On July 28, 2016 7:48:49 AM EEST, graucio notifications@github.com wrote:
Sent from my Android device with K-9 Mail. Please excuse my brevity. |
There is still a panic tho (see @sduponch's post), I'm trying to identify what causes it... @rightkick, from what you say, it seems that you have traffic coming from loopback that was causing the kernel panic (that's why filtering it out resolves it for you). Maybe you could try to identify it to help with the issue! You could capture loopback traffic, and try filtering parts of it until you find out the source of the panic! |
@wirelessmundi for me, catching telegram's packets was causing the most crashes... |
@rightkick I am not sure I understood what you wrote.
|
Correct. I meant traffic on lo interface. In my case I have quite some traffic from zabbix monitoring system. I might be able to capture some and provide a breakdown shortly. On August 4, 2016 9:27:17 AM EEST, Eliezer Croitoru notifications@github.com wrote:
Sent from my Android device with K-9 Mail. Please excuse my brevity. |
Indeed, it seems that on lo interface I have a lot of Zabbix traffic on TCP port 10050 and some traffic on mysql port TCP 3306. I see also some ports I use for redirections from squid3. So my lo port is somewhat busy. |
@rightkick , I've done what you did and seems that I have no lucky as you did hehehehe, it mostly crashes on SSL traffic it seems... Using Skype leads to kernel panics, I'll look further to confirm this. Any news, please share with us! Thanks all! |
@graucio, I have tested Skype filetring with success. Can you paste the firewall rules to look into? On August 5, 2016 7:53:54 PM EEST, graucio notifications@github.com wrote:
Sent from my Android device with K-9 Mail. Please excuse my brevity. |
Not sure if it can help, but latest version from vel21ripn does not panic, see: |
I've been testing https://github.com/vel21ripn/nDPI and weirdly it does panic sometimes when issuing apt-get update. Especially immediately after iptables have been loaded. With other traffic it does not panic. |
@rightkick you need to exclude OUTPUT and INPUT traffic from the "lo" one. |
@elico, I have ACCEPT rules in "lo" in place, as per below and I was seeing kernel panics:
I am putting the DPI filtering on mangel PREROUTING, to avoid altogether processing of locally generated traffic. I will test with the new firewall and will revert with outcome. I was thinking also if it is better to move the DPI filtering at filter table, before state check, instead of mangle. |
@rightkick it's really an issue. |
@elico, putting the DPI filter rules at mangle PREROUTING I don't receive kernel panics. |
@rightkick if something goes into squid don't touch it with iptables and let squid what it does best. A side note: if you need filtering and not caching then RedWood is much more efficient then Squid-Cache in the sense of memory and CPU. |
@elico filtering at mangle PREROUTING will drop any traffic before it is redirected at squid. In case I filter only at mangle FORWARD isn't there possible applications to use tcp port 80 through squid to go out (like teamviewer, skype, etc)? I will try to test this and see. Testing for more then 24 hours with rules at mangle PREROUTING has not given any kernel panics. I have a squid setup and I do not intend to replace it soon as I am interested on the caching option. Thanx though for pointing out RedWood. |
During testing I experienced kernel panic even when filtering at mangle PREROUTING. |
I have been tested for several days with 40Mbit traffic with filtering rules (DROP) at mangle FORWARD. |
With abort 1000 Mbit/s and 150 000 packet per second i get kernel panics exactly the same as mentioned above, this with @betolj's fork. Hope this can be worked out seems to happend more often the more traffic. I was using this to drop torrent packets in FORWARD chain. |
@rightkick The latest commit still have this problem, with higher speeds as 1Gbit/s it crashes after about 2-3 hours. |
Hi. You mention that to compile netfilter-ndpi I need iptables-dev >= version 1.4.21-1ubuntu1.
I have a Debian 7:
kernel 3.18.36
iptables 1.4.14-3.1
iptables-dev 1.4.14-3.1
conntrack 1:1.2.1-1+deb7u1 enabled (defaults from official debian repo).
When compiled I managed to filter traffic etc but I've been getting some random kernel panics (reporting out of memory), especially when I put filtering rules at INCOMING and OUTGOING chains of the filter table. (kernel panics are more frequent when filtering rules are at INPUT, OUTGOING, FORWARD chains, while panics are less frequent when filtering only at FORWARD, indicating that the problem is exacerbated when more traffic is processed ).
Have you tested with Debian 7? Going to iptables-dev >= 1.4.21 is the only option?
Thanx
The text was updated successfully, but these errors were encountered: