Feature request: use of suricata 'ignoring traffic' features

[petererdos]

This would be the starting point:
https://suricata.readthedocs.io/en/latest/performance/ignoring-traffic.html

It would be very nice if these could be implemented in the GUI, at least capture filters, as they look like (fairly) easy to implement (hopefully they are) :)

The reason for this is that sometimes connections and services brake when using IDS + IPS. There are a few hosts which can be "whitelisted", eg. Windows Update servers (which many times fail without any apparent reasons while nothing is logged as blocked - hunting down the rule(s) causing this behavior is difficult).

It would be awesome (well, maybe useless, if not possible, in the case of WU) to use wildcard dns records somehow, as capture filters look like only support IPs and cidr net addresses. Maybe this can be worked around with aliases (?). If not, at least a wsus server (internal) could be whitelisted and clients could get the updates from there (this would be another workaround).

Many thanks!

[AdSchellevis]

Pass rules should be quite simple to integrate into the "user rules" tab which is already there, although this traffic will still flow trough suricata.

BPF filters would fully skip suricata, but I haven't tested it with netmap (IPS mode) yet, although it seems to be supported.
The disadvantage is that according to the examples we need to specify the filter statically at startup, maybe there is a possibility somewhere to feed in a list of some sort (or a pipe we could feed dynamically), but I haven't looked into that.

There are quite some use cases to skip traffic, like skipping high traffic hosts like netflix, it would be good to investigate the options we have before adding knobs for a specific host.

[fichtner]

Netmap is before BPF AFAIK so that wouldn't work.

[AdSchellevis]

that was I was I thought too, but https://github.com/OISF/suricata/blob/master-4.0.x/src/runmode-netmap.c#L113 suggests these filters also apply on netmap. not sure what the gain is then.

[mimugmail]

I'd take this the next weeks.
Proposal: make User rules a subtab and add subtab whitelist. Also space for future use with a new tab supporting user defined rules.

Suggestions?

[AdSchellevis]

@mimugmail I'm not fully convinced we should use the bpf filters at the moment, the pass operation is simple, although wouldn't have the full gain of skipping traffic completely (easy to integrate, only add an option type). bpf filters need a full restart of suricata and might add more complexity then we would like to have. What did you want to build for this item? I think it's good to first discuss options together with measurements if needed before adding complexity here.

[mimugmail]

Yep, BPF would perform much better, but how complex will your syntax be when you have 3 hosts to 3 different locations to whitelist? Perhaps we can mix it up? A rule whitelisting in rules tab and a BPF exclude in General tab?

[fichtner]

Complex bpf filters are very difficult in grammar. Any two conditions bound with „or“ are extremely difficult to write so it would just be a batch exclude for source or destination IP. Best would be to avoid doing this at all.

…

On 1. Sep 2018, at 14:08, Michael ***@***.***> wrote: Yep, BPF would perform much better, but how complex will your syntax be when you have 3 hosts to 3 different locations to whitelist? Perhaps we can mix it up? A rule whitelisting in rules tab and a BPF exclude in General tab? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

[AdSchellevis]

These things tend to grow :) do we have any statistics for the additional overhead the pass operation would cost? Unfortunately we can't use the new features like eBPF and XDP, which would integrate nicely with the pass keyword if I'm not mistaken. I'm just not very fond of static BPF filters (unless it's only one simple ip exclude in the general tab, which only contains validated entries).

[mimugmail]

For me the UI lacks some features if you have a FP. You can only disable the specific rule completely. A pass rule above with src/dst combination would fit perfectly. Surely it wont speed up or exclude netflix streaming, but if I remind correctly there's some planning for 4.1 or 4.2.

[AdSchellevis]

I'm not sure what you mean now, if it's about false positives from a specific host, the pass operation would logically be sufficient. It also has performance advantages, although I haven't measured anything in that direction so I'm not sure what the differences are.

Excluding traffic like Netflix would indeed be very practical, but in the FreeBSD eco system I'm not sure if and when that will be possible, most development in that area (unfortunately for us) happens in Linux.
As far as I know these techniques use the pass operator to steer the bypass operation.

[mimugmail]

What I mean is that I'd try to add this (pass rules) to the UI. I only asked if you are ok with it, so not to make some useless work :)

[AdSchellevis]

yes, pass is fine :) just add it in the list here https://github.com/opnsense/core/blob/master/src/opnsense/mvc/app/models/OPNsense/IDS/IDS.xml#L21-L24 and I think it will work instantly

[mimugmail]

I think we are crosstalking :) If I just add pass, this would be the same as disabling the rule.
What I want is a input field where I can set source and destination as NetworkField so it's not only any any. I'll add a PR the next days ...

[AdSchellevis]

sure, no problem, if you just add the additional fields to the existing tab I'm fine with it.

[fichtner]

https://suricata.readthedocs.io/en/latest/performance/ignoring-traffic.html#pass-rules was added, closing. thanks!