nfacctd: print plugin -- writer [netflow] multiple processes are created and consume 40% of my system memory

[hemant61201]

I implemented nfacctd version 1.7.1 and encountered an issue with the nfacctd print plugin. Multiple processes were created by the writer [netflow], consuming 40% of my system memory, and I observed a flow rate of around 15-17k EPS.

[paololucente]

Hi @hemant61201 ,

Can you share your configuration? This may depend on the selected (de-)aggregation along with caching time (refresh time). Also can you confirm which version are you using? 1.7.1 or 1.7.10. If 1.7.1, as you mentioned, that is not supported anymore and i'd advice to upgrade to something more recent.

Paolo

[hemant61201]

Hi @paololucente,

Here is my config file :
nfacctd-conf.txt

I implemented nfacctd version 1.7.1 and encountered an issue with the nfacctd print plugin.

Since version 1.7.1 is not supported, could you please advise which version I should update to in order to resolve this issue? Additionally, could you elaborate on the conditions under which multiple print plugin writer processes are created?

[hemant61201]

Hi @paololucente,

I upgraded pmacct version to 1.7.9 but still i faced the same issue.

Here i Attached the screen shots :

[paololucente]

Hi @hemant61201 ,

The urgent writers are triggered by the fact the cache space of the plugin is finished. This is related to the (de-)aggregation method you configured: the more flows you have, the more you de-aggregate, the more cache space you need in order to keep in memory all the entries for the wanted refresh time (by default 60 seconds). Also, because you write to a fixed file NetFlow (argument to print_output_file[NetFlow]), the writers will tend to lock out with each other.

My recommendations:

1. add at least a %s to the file name, ie.: print_output_file[NetFlow]: NetFlow_%s;
2. increase the cache space (of course this is limited by the memory available, print_cache_entries, read more here:

   pmacct/CONFIG-KEYS

   Line 1422 in c7a3d06

   KEY: [ sql_cache_entries | print_cache_entries | amqp_cache_entries | kafka_cache_entries ]

   ) or reduce the time interval to write to files (print_refresh_time, read more here:

   pmacct/CONFIG-KEYS

   Line 1338 in c7a3d06

   KEY: [ sql_refresh_time | print_refresh_time | amqp_refresh_time | kafka_refresh_time ] (-r)

   )