[FR] Add low level L7 DPI via Suricata

[mimugmail]

Hi guys,

what do you think about adding another rules file in suricata rules folder? I talked with @fichtner about adding low level DPI via Suricata rules and put them in an own repo. But for now it seems an additional file in master is fairly fine :)

Since most of application recognition rules are based on http/s they are easy to block.

All I need is a file based like this:
Application Type URL
Youtube l7-media-streaming youtube.com

This woud create 3 rules for blocking connections to Youtube at all:

alert dns $HOME_NET any -> any 53 (msg:"L7_DPI - Youtube - DNS request for youtube.com"; dns_query; content:"youtube.com"; nocase; classtype:l7-media-streaming; sid:51000000;)
alert http $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"L7_DPI - Youtube - Related URL (youtube.com)"; content:"youtube.com"; http_uri; flow:to_server,established; classtype:l7-media-streaming; sid:51000001; rev:1;)
alert tls $HOME_NET any -> $EXTERNAL_NET any (msg:"L7_DPI - Youtube - Related TLS SNI (youtube.com)"; tls_sni; content:"youtube.com";flow:to_server,established; classtype:l7-media-streaming; sid:51000002; rev:1;)

This would block DNS requests to the domain and if this fails or DNS is internally it looks via http request, or in case of SSL in tls_sni. I tested it and it works great.

Via rules tab it's really easy to manage since you can group by classtype (set in the list) and you can search all by filename or "L7_DPI".

For example you create lists of filesharing sites like uploaded or rapidshare, group them in classtype:filesharing. If you want to block all filesharing sites select the classtype, mark all and enable/drop the rules.

The script is also extendable for plain /URIs/ or just tcp/udp ports.

Also an advantage is that community can easily deliver patterns to add (e.g. if there is a new filesharing site), so no need to actively watch the market

[AdSchellevis]

@mimugmail Since the rest of the files are fetched from a different location, do you mind if we host the L7 rule file on rules.opnsense.org? Then we can add it to the metadata file already in place https://github.com/opnsense/core/blob/stable/17.7/src/opnsense/scripts/suricata/metadata/rules/opnsense.xml#L4
The advantage is people can easily enable/disable the sets and we don't have to think of anything new here (local files can't be managed equally at the moment).

We could add another github repo for version management if you like.

[fichtner]

rules.opnsense.org sounds good, but I'd favour a simple "apps" name or similar for the rules, not L7 or DPI because these terms are either archaic or overloaded.

[AdSchellevis]

@fichtner I agree, I wasn't thinking about the naming yet :)

[fichtner]

PS: categories are individual rules files, so that's taken care of as well

@AdSchellevis I was referring to @mimugmail specifically

Then again, if we have a rules repo, we can ship this via github...

[AdSchellevis]

@fichtner if you don't mind, I rather host them on a nice opnsense url :)

[fichtner]

sure, of course

[mimugmail]

@AdSchellevis opnsense.org Domain is fine, just tell me how to transfer the rules file and we're good! 👍
@fichtner Yep, L7_DPI is nonsense. How about just "OPNsense", because all rules will receive this prefix. It like "ET Open" or "ET PRO" in front of every rule?

I'd strongly insist to use classtype for grouping. From the Suricata docs it's just the way to do it this way. Keep a small word in the rule and a expl. and priority in classification.conf. Imagine how easy it would be to just block all media-streaming when you group them in classtype .. shoutcase, ice.fm, youtoube ...
IMHO these are real business needs. I have customers demanding to block e.g. dropbox and wetransfer.
It's some kind of data protection policy .. you'll surely know this from your main job 👍

[AdSchellevis]

shall I create a github repository "rules" on OPNsense ?

[mimugmail]

From a business perspective I'd keep it private, but since this is an open project a new repo fits best :)

[AdSchellevis]

done https://github.com/opnsense/rules

[mimugmail]

I'd propose 7 classtypes:

file-transfer (file sharing in general)
media-streaming (streaming, like youtube or shoutcast)
social-networking (facebook, google+)
messaging (ICQ, whatsapp)
mail (gmail, yahoo mail, mail.ru)
games (Zynga, battle.net)
web-services (general sites, amazon, ebay ...)

In rules/ there could be a files called webapplication.list where everyone can push requests to, just containing:

Application-name Class-type Domain eg:
Youtube media-streaming youtube.com

and a small doc above the file. Then you don't have to search for typos in rules and generate the ruleset with this file, put in /rules/src, e.g. opnsense.webapplication.rules

And then e.g. a second rules file, opnsense.protocol.rules, for rules don't fitting the script for example blocking of untrusted POP commands withing a POP3 stream.

[fabianfrz]

@mimugmail I would not add web-services as that is everything that runs on port tcp/80 or tcp/443 but your list looks more like shopping.

For social networking etc. I would recommend using squid for that because it is more powerful in blocking web stuff.

However blocking stuff that cannot be blocked via the proxy can be here. Also keep in mind that TLS will make huge issues if you cannot MitM.

[mimugmail]

web-services could be just the rest, like news, sports, or shopping.

I dont want to run squid on all Firewalls, e.g. in the Datacenter. And Suricata with only some good selected rules is less error prone and faster.

For tls.sni theres no need for mitm since this is still plain text.

[fabianfrz]

Yes but it can bypass your firewall.
SNI does not need to be correct - it is just a hint for the server which can be used to select a backend/vhost. I don't know if that is still possible but a CDN with a spoofed SNI was used to bypass GFoC (this is why I would not rely on it).

[mimugmail]

@fabianfrz That's why I'm calling it low level DPI. It's a basic "protection". To be honest, most users of OPN aren't specialized, they want a stupid button to do/block something.
In the long term it would be good to detach L7 from IPS and do both, transparent proxying AND Suricata with one click.

If I really care that much about security that someone might fake a SNI I'd choose a completely different product.

And hey, that would mean GFoC is not better that OPN :P

[fichtner]

We should simply refrain from using the rules to do whitelisting. That way faked SNI is a blocked false positive and does not mean any harm. Whitelisting is difficult anyway as it requires to span dependencies across sites like allowing CDNs or ad networks so the parent page loads...

…

On 21. Oct 2017, at 16:00, Michael ***@***.***> wrote: @fabianfrz That's why I'm calling it low level DPI. It's a basic "protection". To be honest, most users of OPN aren't specialized, they want a stupid button to do/block something. In the long term it would be good to detach L7 from IPS and do both, transparent proxying AND Suricata. If I really care that much about security that someone might fake a SNI I'd choose a completely different product. And hey, that would mean GFoC is not better that OPN :P — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[fichtner]

@mimugmail closing, if not correct let me know

[fabianfrz]

@fichtner @mimugmail sorry to disturb but this may get trouble with TLS1.3 when ESNI is used.

[mimugmail]

yep .. same to openappid, but better than nothing :)