netflow log rotation parameters

[fichtner]

Hi Ad,

The current flowd log file is turned over 10 times for each 10 MB. In cases where this is larger traffic volume old data points may be lost when recreating the database, going into POLA violation territory because it's not obvious. What about making these two values configurable?

Or changing the defaults, or documenting them. Or something else entirely? :)

Cheers,
Franco

See: https://forum.opnsense.org/index.php?topic=3581.0

[AdSchellevis]

Hi Franco,

I don't mind adding some configuration parameters for it, but this issue should only happen when you trash the full database and try to recreate it from its source (which may take quite some time if you increase this value).... Personally I haven't seen corruption on the more aggregated datasets only saw the 30 seconds average get corrupted beyond repair, although others can get damaged by a crash as well (as can flowd.log) .

Basically it isn't a very good idea to trash data you actually want to preserve ;) The flowd log files are only meant for staging. You can't keep a lot of history in there, because it would take too much time to parse it back to our structure.

Cheers,

Ad

[fichtner]

Upon further review this seems to work as intended if we can guarantee the log rotation to kick in, which seems to be the case with the database repair now. I will make a note in the GUI and close this.