Suricata - Policy usage creates error: error installing ids rules ()

[f45tb00t]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug

Hi,

I thought this has been fixed with:
ref: #4946

Actually I am not very sure if this is related to the issue from above.
Basically when I create a policy and click apply I get "error: error installing ids rules ()"

I've removed everything from the policies, shutdown suricata, deleted everything from "/usr/local/etc/suricata/rules" and reinstalled suricata and tried again. The issue persists

I also do not have any extra plugins (like the free ones) installed anymore. I had them installed, but removed them some month ago.

To Reproduce

Steps to reproduce the behavior:

1. Just add a policy and click apply and the error will pop up

Expected behavior

When appliying the policy I would not expect the error.

Describe alternatives you considered

Remove everything and start from scratch. Issue persist.

Screenshots

None yet

Relevant log files
Please let me know which log file you need and I'll provide it asap
Additional context

None yet

Environment

Software version used and hardware type if relevant, e.g.:

OPNsense 21.7.6-amd64 (amd64, OpenSSL).
Intel(R) Celeron(R) CPU J3160 @ 1.60GHz
Network I210 Gigabit Network Connection

[f45tb00t]

Note: The interesting part is, I've checked the file status while creation
38596 8389 -rw-r----- 1 root wheel 30M Nov 29 14:53 rules.sqlite
When it reaches the 30M the error pops up.

Could it be, that the timeout is set too low somehow? Like the error is shown when applying the policy takes too long?

[kulikov-a]

hi
any errors if you run /usr/local/opnsense/scripts/suricata/installRules.py manually?

[f45tb00t]

Hello @kulikov-a,

no errors at all shown via cli when executing the script manually.

[kulikov-a]

@f45tb00t
sorry for the delay.
yes, the configdRun() function contains timouts control (2 min by default). but I had to reduce it to 1 minute for a configuration with snort rules enabled and a 54MB cache db in order to reproduce your error. and this is on a VM with one Xeon(R) CPU E5-2620 core configured and 4GB of RAM (and with unbound with all blacklists allowed).
so imho the reason is something else. resources are already depleted. Policies applying task is not so resource intensive.

[f45tb00t]

Hello,

just let me know what I can provide as logs to track down the issue when you have time. Regardless to the error, it seems the policy is applied. But from where the error comes, I have absolutely no clue.

[kulikov-a]

Regardless to the error, it seems the policy is applied.

i think installRules.py worked anyway (although the configdRun() function returned null because of timeout)

just let me know what I can provide as logs to track down the issue when you have time

if I understood the code correctly, an empty "error: error installing ids rules ()" error may appear if the call to the configdRun("ids install rules") function did not return 'OK'. which in its turn can happen due to a timeout, an execution error, or the impossibility of connecting to the configd socket. (in such cases, the corresponding errors should appear in the backend log, but it seems that for now internal configdRun errors are not processed by syslog-ng. I will try to make a pr for this).
since in your case the script starts executing (the size of the cache db changes), I think that the matter is in the execution timeout. and in this case, I would follow the resources and processes that these resources can absorb (SYSTEM: DIAGNOSTICS: ACTIVITY)