Matching not correct. #1

fcastello · 2014-09-23T16:54:50Z

I think I found that protocols don't match the protocol name with the actual protocol being described.
I guess you fixed ewildgoose version by adding to xt_ndpi.h
NDPI_PROTOCOL_LONG_STRING
NDPI_PROTOCOL_SHORT_STRING

which does the matching of the protocol names in iptables -m ndpi --[protocol name]
This was usualy done in older versions of the nDPI library. But those constants are no longer part of the nDPI library.
I will try with an older nDPI to get it to work. Otherwise it should be fixed to work with newer versions of the nDPI library since they don't match protocols with NDPI_PROTOCOL_LONG_STRING
NDPI_PROTOCOL_SHORT_STRING any more.

fcastello · 2014-09-23T16:56:04Z

I got it working on ubuntu 14.04 and 12.04. It work's. compiles and let me use iptables targets but. I don't think it's doing what it is supposed to do.

betolj · 2014-09-23T17:38:29Z

I did some tests yesterday and it has worked for me.
For example:

iptables -I INPUT -m ndpi --http -j DROP
lynx www.google.com
2 613 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol HTTP

iptables -I INPUT -m ndpi --ssh -j DROP
ssh x.y.z.w -p 2222
19 26728 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol SSH

iptables -I OUTPUT -m ndpi --rdp -j DROP
rdesktop x.y.z.w
7 602 DROP all -- * * 0.0.0.0/0 0.0.0.0/0 protocol RDP

Unfortunately, I noticed that the choice of the proper chain depends on the flow through which data was received.

And, unless you recompile the kernel with the patch, the module nf_conntrack_netlink can not be loaded into memory (but you will have problems with conntrack tool).
rmmod nf_conntrack_netlink
modprobe xt_ndpi

betolj · 2014-09-23T17:40:25Z

root@humberto-XPS-8300:# cat /etc/debian_version
jessie/sid
root@humberto-XPS-8300:# uname -a
Linux humberto-XPS-8300.ms 3.13.0-35-generic #62-Ubuntu SMP Fri Aug 15 01:58:42 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

root@humberto-XPS-8300:~# cat /etc/os-release
NAME="Ubuntu"
VERSION="14.04.1 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.1 LTS"

betolj · 2014-09-23T17:51:12Z

For ssh you must have at INPUT (or http).
iptables -I INPUT -m ndpi --ssh -j DROP

The signature is in the response.

For routed traffic you probably will not have any problem because the rule may cover both directions, but be careful to define the flow in the rule.

fcastello · 2014-09-23T18:06:42Z

I have been testing it and you are correct.
It is not matching http services like google facebook whatsapp etc.
I am trying to find out why.
It is not working with any of the services defined in nDPI/src/lib/ndpi_content_match.c.inc

betolj · 2014-09-23T18:19:31Z

Yes... and this is an old problem!

Apparently, it is more safe and guaranteed to work with protocols where there was a specific module build (ftp_data, http, rdp and others).

Most filters in nDPI/src/lib/ndpi_content_match.c.inc or do not work well or take unwanted action. When I filter for twitter, several sites that reference it did not open. This is certainly a problem.

betolj · 2014-09-23T18:30:10Z

It works well for protocols in /usr/src/nDPI/src/lib/protocols/
This is why works with the twitter

fcastello · 2014-09-23T18:34:01Z

I have no idea. I will get some pcaps and see why is matchin twitter and not the others.
The good thing is that I don't want to block twitter. I just want to apply some traffic control policy to it. Which won't be as bad as filterning anything with the word twitter

fcastello · 2014-09-23T18:37:36Z

In ntop everything works like a charm, I guess they do analisis of the entire network flow matching the criteria.

betolj · 2014-09-23T18:46:08Z

It's different.

I think that the filters in ndpi_content_match.c.inc are internal for nDPI and netfilter can not use them because it depends on which modules are compiled and integrated.

The url filters is also indicated in Squid or other proxy.
You can use zph to set a specific DSCP if you are looking for QoS.

syadnom · 2014-11-13T18:22:42Z

I'm building against 8323 and it's not matching anything... This is the most recent nDPI I can get the build module to load.

betolj · 2014-11-13T18:43:13Z

Hi, Unfortunately this module is incompatible with nf_conntrack_netlink.

And, unless you recompile the kernel with the patch, the module
nf_conntrack_netlink can not be loaded into memory (but you will have
problems with conntrack tool).

rmmod nf_conntrack_netlink
modprobe xt_ndpi

2014-11-13 15:22 GMT-03:00 syadnom notifications@github.com:

I'm building against 8323 and it's not matching anything... This is the
most recent nDPI I can get the build module to load.

—
Reply to this email directly or view it on GitHub
#1 (comment).

betolj · 2014-11-20T19:18:18Z

I will make several fixes as soon and protocol id bugfix too.
Wait a little bit.

alrferreira · 2014-12-02T02:55:07Z

Hello,
I'm testing and really many filters do not work. I would like to contribute you with signatures, but first wanted to know what the situation is.

mcitew · 2015-03-13T02:02:27Z

I tested this too. I found it's working for some few protocols such as SSL, http, ftp.

But one of the most important protocol, youtube ( specifically youtube over SSL ) is not working at all.

I just add an iptables rule,

iptables -I FORWARD -m ndpi --youtube

Matches nothing. But when I run ndpiReader -i interface, it picks up youtube traffic

Mile-Lile · 2015-06-11T08:49:41Z

same for me. Did you founded solution?

betolj · 2015-06-12T01:44:59Z

Hi,

I'm working in a patch for the latest nDPI version and i will include support
for youtube too.
And i'm currently testing youtube detection - it seems to be working.

I'll probably update the repository tomorrow.

2015-06-11 4:49 GMT-04:00 Mile-Lile notifications@github.com:

same for me. Did you founded solution?

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

alrferreira · 2015-06-12T02:26:46Z

Working to me when I enable first a SSL Rule. But Skype don't.

André Luiz R. Ferreira - NETDEEP

-----Mensagem Original-----
De: "Humberto Jucá" notifications@github.com
Enviada em: ‎11/‎06/‎2015 22:45
Para: "betolj/ndpi-netfilter" ndpi-netfilter@noreply.github.com
Cc: "alrferreira" alrferreira@netdeep.com.br
Assunto: Re: [ndpi-netfilter] Matching not correct. (#1)

Hi,

I'm working in a patch for the latest nDPI version and i will include support
for youtube too.
And i'm currently testing youtube detection - it seems to be working.

I'll probably update the repository tomorrow.

2015-06-11 4:49 GMT-04:00 Mile-Lile notifications@github.com:

same for me. Did you founded solution?

—
Reply to this email directly or view it on GitHub
#1 (comment)
.

—
Reply to this email directly or view it on GitHub.
*** Secured by Netdeep ClearMail ***

Mile-Lile · 2015-06-12T10:53:13Z

@betolj
Thx. appreciate it!

betolj · 2015-06-16T02:49:04Z

The "ndpi-netfilter" projects with logical structure based on "ewildgoose" model don't work well nowadays.

Only one exclusive "conntrack notify" call: For this reason, it cann't be used in conjunction with nfnetlink. But, when you remove the nfnetlink kernel module, the conntrack application wont works anymore.
The web host detection depends on http or ssl protocols enabled: Youtube or Facebook protocols are not external modules (like /usr/src/nDPI/src/lib/protocols/*).

I finished xt_ndpi fixes today:

Remove "conntrack notifier" from source code and include a new ndpi flow garbage collector for 3.x kernel series. Now, no longer need remove the nfnetlink kernel module to install xt_ndpi.
Enable http and ssl protocol for youtube, facebook and others.

My tests, showed satisfactory results.
https://github.com/betolj/ndpi-netfilter

Mile-Lile · 2015-06-16T03:17:14Z

Thx again!

hemanth4ap · 2016-06-01T05:22:59Z

how to block particular http/https urls by using ndpi

betolj closed this as completed Jun 16, 2015

betolj mentioned this issue Nov 24, 2015

nDPI 1.7 #16

Closed

hariesnurikhwan mentioned this issue Jul 3, 2016

Compile on Centos 6.x? #28

Open

Opendium mentioned this issue Aug 24, 2016

Scheduling while atomic #36

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Matching not correct. #1

Matching not correct. #1

fcastello commented Sep 23, 2014

fcastello commented Sep 23, 2014

betolj commented Sep 23, 2014

betolj commented Sep 23, 2014

betolj commented Sep 23, 2014

fcastello commented Sep 23, 2014

betolj commented Sep 23, 2014

betolj commented Sep 23, 2014

fcastello commented Sep 23, 2014

fcastello commented Sep 23, 2014

betolj commented Sep 23, 2014

syadnom commented Nov 13, 2014

betolj commented Nov 13, 2014

betolj commented Nov 20, 2014

alrferreira commented Dec 2, 2014

mcitew commented Mar 13, 2015

Mile-Lile commented Jun 11, 2015

betolj commented Jun 12, 2015

alrferreira commented Jun 12, 2015

Mile-Lile commented Jun 12, 2015

betolj commented Jun 16, 2015

Mile-Lile commented Jun 16, 2015

hemanth4ap commented Jun 1, 2016

Matching not correct. #1

Matching not correct. #1

Comments

fcastello commented Sep 23, 2014

fcastello commented Sep 23, 2014

betolj commented Sep 23, 2014

betolj commented Sep 23, 2014

betolj commented Sep 23, 2014

fcastello commented Sep 23, 2014

betolj commented Sep 23, 2014

betolj commented Sep 23, 2014

fcastello commented Sep 23, 2014

fcastello commented Sep 23, 2014

betolj commented Sep 23, 2014

syadnom commented Nov 13, 2014

betolj commented Nov 13, 2014

betolj commented Nov 20, 2014

alrferreira commented Dec 2, 2014

mcitew commented Mar 13, 2015

Mile-Lile commented Jun 11, 2015

betolj commented Jun 12, 2015

alrferreira commented Jun 12, 2015

Mile-Lile commented Jun 12, 2015

betolj commented Jun 16, 2015

Mile-Lile commented Jun 16, 2015

hemanth4ap commented Jun 1, 2016