Feature request: Better firewall live log filtering

[Kimotu]

[x] I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md

[x] I have searched the existing issues and I'm convinced that mine is new.

Is your feature request related to a problem? Please describe.
Examining live log is an essential part of troubleshooting when a host cannot access resources like expected, but the filtering possibilities are aweful. It is just string matching on client side (browser), you cannot define and match several filter criterias like e.g. pfSense does.

Describe the solution you'd like
Add more fields for filters, at least interface, src host, dst host, protocol and src/dst port. Or lets define tcpdump filters (src host 192.168.1.1 and dst port 80).

I think the easiest and fastest possibility for filtering without mucht effort would be to keep client side string matching, but for each column seperately.
Just add fields above every column and start matching from left to right.
Then you just have to fill in the fields that you want to filter, e.g. Interface 'WAN' and Destination ':80'. That would show all entries on interface WAN that connect to Port 80.

Describe alternatives you've considered
My only considered alternative would be exporting the firewall log into an elasticsearch stack and use it for filtering. This needs to setup an ELK stack.
For the moment the easiest is to export /var/log/filter.log into csv, open with excel and use auto filters to analyze. But then it is not a live log any more.

[AdSchellevis]

You can already use regular expressions on the fields visible. 192.168.1.1.*80 would do the same as the suggested tcpdump filter (which won't be possible to add).

There definitely is room for improvement, although most matches are already easy to filter, better documentation on the subject would be a very welcome starting point.

[Kimotu]

Hmmm, strange. I tried wildcards (*) several times and it does not work. I just tried it again:

Interface		Time	Source	Destination	Proto	Label
Wohnheim		Apr 4 10:03:32	10.6.0.113:56581	127.0.0.1:3129	tcp	Default deny rule

So when I type Wohnheim*3129 as soon as I type the 3, the rule is gone, the filtering string not matched.

[fichtner]

You're looking for "Wohnheimmmmmmmmmmm3129", try "Wohnheim.*3129"

[fichtner]

( this is also relevant in the scope if this ticket #2195 )

[Kimotu]

Ok, right. So it is real RegEx. Can confirm, that it works when using .*. Not very intuitive, but better than exporting.

[AdSchellevis]

maybe you would like to consider contributing to the docs, it would help the next person looking for something similar.

[Kimotu]

This link? So I have to work with git or is there a more wikistyle solution?

[AdSchellevis]

yep, Git is the way indeed, we're using Sphinx (https://docs.readthedocs.io/en/stable/intro/getting-started-with-sphinx.html) to write documentation, it's not very difficult.

[Kimotu]

Ok, I'll give it a try.

[Kimotu]

Hmmm, from one issue tracker to the next. First I have to resolve this issue. Cloned git, installed sphinx and run make html

Exception occurred:
  File "/usr/lib/python3.6/site-packages/sphinxcontrib/nwdiag.py", line 221, in html_visit_nwdiag
    self.builder.warn('dot code %r: %s' % (node['code'], str(exc)))
AttributeError: 'StandaloneHTMLBuilder' object has no attribute 'warn'
The full traceback has been saved in /tmp/sphinx-err-bovqgozh.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
make: *** [Makefile:56: html] Error 2

[AdSchellevis]

did you execute pip install -r requirements.txt as described?

[Kimotu]

Yes, I did. It installed sphinx 2.0.0. But I saw that my distro provides a sphinx package. Older (1.6.5) but this time it compiled the docs. :-)

[fichtner]

was merged in opnsense/docs@dde4e51 -- thanks!