filterlog flooding the console

[fraenki]

I've noticed that filterlog will occasionally flood the console with block messages:

Is this intended or a defect? This seems to occur shortly after and during system boot and possibly after configuration changes.

[jschellevis]

I have not seen this on my test systems.. anything particular with your setup? Any idea how to trigger it?
As I think these message should normally not show up on the console.

[fraenki]

I've tried to find a way to reliable reproduce this, but I failed.

Though I've seen it again after adding a new DNS server. I've noticed this time that OPNsense required quite some time to reload it's configuration (> 1 minute), CPU load stayed at 100% (20% user, 80% system) for some time. When the reload finished, the flooding started again, and ceased after about 20 seconds.

Well, it's not too important, since the flooding stops after a short while. But it could hint that something in the reload process is prone to fail.

[fichtner]

It looks like a reconfigure is not atomic and applies pfsync traffic rules at a later stage during reload... Are you running pfsync/carp?

[fraenki]

Not running pfsync/carp on my OPNsense test VM (yet).

[fichtner]

Then it is traffic from another machine in your network I reckon?

[fraenki]

Yes, it's me pinging the OPNsense VM... but I guess one would see any blocked traffic.

[jschellevis]

I tried to reproduce.. but only got a message or two when restarting syslogd.. but that is normal since all messages that cannot be delivered to systlog will go to console.

Can you explain exactly what you did the last time when you saw it? Did you add a DNS server to the DHCP server?

[fraenki]

All I did was adding a DNS server. But as I said it was not reproducable in this particular case. Adding/removing more DNS servers did not trigger the issue again. But I've seen it again on another random configuration change.

So if you can't find anything prone to errors in the config-reload logic, I'd suggest to close this issue for now.

[pv2b]

I've seen this same issue also in version 18.7.1_3, not sure why it's doing this.

Sample output:

Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,43133,0,none,17,udp,56,83.233.94.213,192.5.5.241,20027,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,39956,0,none,17,udp,56,83.233.94.213,192.36.148.17,30509,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,63049,0,none,17,udp,56,83.233.94.213,199.7.91.13,25811,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,27139,0,none,17,udp,56,83.233.94.213,192.33.4.12,46742,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,33387,0,none,17,udp,56,83.233.94.213,193.0.14.129,15843,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,58383,0,none,17,udp,56,83.233.94.213,192.36.148.17,59363,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,32329,0,none,17,udp,56,83.233.94.213,192.203.230.10,33846,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,9026,0,none,17,udp,56,83.233.94.213,199.9.14.201,59427,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,25623,0,none,17,udp,56,83.233.94.213,199.7.91.13,30664,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,51005,0,none,17,udp,56,83.233.94.213,199.7.83.42,34485,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,17368,0,none,17,udp,56,83.233.94.213,192.58.128.30,7059,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,34000,0,none,17,udp,56,83.233.94.213,192.36.148.17,15312,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,24338,0,none,17,udp,56,83.233.94.213,193.0.14.129,34852,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,36409,0,none,17,udp,56,83.233.94.213,202.12.27.33,15595,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,59262,0,none,17,udp,56,83.233.94.213,192.5.5.241,39457,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,17549,0,none,17,udp,56,83.233.94.213,198.97.190.53,62702,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,21569,0,none,17,udp,56,83.233.94.213,192.112.36.4,63604,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,25542,0,none,17,udp,56,83.233.94.213,198.41.0.4,64847,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,12841,0,none,17,udp,56,83.233.94.213,192.203.230.10,39279,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,56367,0,none,17,udp,56,83.233.94.213,192.112.36.4,51022,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,40500,0,none,17,udp,56,83.233.94.213,199.7.83.42,12669,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,16782,0,none,17,udp,56,83.233.94.213,199.9.14.201,42962,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,39679,0,none,17,udp,56,83.233.94.213,198.97.190.53,57719,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,48309,0,none,17,udp,56,83.233.94.213,192.36.148.17,35572,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,32521,0,none,17,udp,56,83.233.94.213,192.5.5.241,46620,53,36
Sep 21 00:54:31 filterlog: 84,,,0,igb1,match,pass,out,4,0x0,,64,37821,0,none,17,udp,56,83.233.94.213,192.58.128.30,5839,53,36

It appears all of these remote IP addresses are DNS root servers and traffic to port 53. Also relevant: The WAN interface was not plugged in at the time. igb1 is the WAN interface in this case.

So, my guess is this is spamming on the console when it's not able to route anywhere.

[markuskoehler]

This issue seems to crash my OPNsense regularly (and I had this problem also with pfSense) on a Proxmox VM... It happens randomly between every 24 hours to 4 days, but I can be sure it happens and then the OPNsense is dead, only reset helps... Any idea on this?

[fichtner]

The suspicion is this can happen when init and / or syslog receives SIGHUP so that log output is redirected to the console because it has nowhere else to go.