pmacct 1.7.8 - Failed during write: Resource temporarily unavailable

[JonMurphy]

• Description
  I see LOTS of these messages from the memory plugin after updating from pmacct version 1.7.6 to 1.7.8.

Jun 28 13:58:42 ipfire pmacctd[4228]: WARN ( plugin1/memory ): Failed during write: Resource temporarily unavailable 
. . .
Jun 28 13:58:53 ipfire pmacctd[4232]: WARN ( plugin1/memory ): Missing data detected (plugin_buffer_size=102400 plugin_pipe_size=102400000). 
Jun 28 13:58:53 ipfire pmacctd[4232]: WARN ( plugin1/memory ): Increase values or look for plugin_buffer_size, plugin_pipe_size in CONFIG-KEYS document.  

If I enable debug in the config file then I see these messages:

Jun 28 14:02:09 ipfire pmacctd[19511]: INFO ( plugin1/memory ): plugin_pipe_size=10240000 bytes plugin_buffer_size=10240 bytes
Jun 28 14:02:09 ipfire pmacctd[19511]: INFO ( plugin1/memory ): ctrl channel: obtained=212992 bytes target=8000 bytes 

My config is this. Please see:
#696 (comment)

I did find this:
https://github.com/pmacct/pmacct/blob/master/CONFIG-KEYS#L324-L354

But it did not help me... And I've tried many "guesses" for plugin_pipe_size and plugin_buffer_size.

One thing to point out - It took about 12 days after the update for the pmacct messages to appear. So this seems to be an issue that takes a few days to make itself known. And it doesn't make it easy to debug! But when it does happen it seems to explode and I get 100's of messages.

pmacct message count in log starting June 13:

-rw-rw-r-- 1 root syslogd 977084243 Jun 29 14:56 /var/log/messages
Failed during write (count)                      = 66346
Missing data detected  (count)                  = 12222
Increase values or look . . .  (count)          = 12222

Version
Provide the version in use. An output of -V is typically good, ie. nfacctd -V

[**root@ipfire** ~] # pmacct -V
pmacct IMT plugin client, pmacct 1.7.8-git (RELEASE)

[paololucente]

Hi Jon ( @JonMurphy ),

As indicated in the quickstart guide, i'd totally discourage you to use the native version of internal queueing (what you are using right now) in favor of ZMQ. Please see the X. Internal buffering and queueing section of the QUICKSTART document at this propo: https://github.com/pmacct/pmacct/blob/master/QUICKSTART

Paolo

[JonMurphy]

Will look into ZMQ (I dont know anything about it).

For what it is worth - all of the above messages (Failed during write, Missing data detected, Increase values or look) show up with debug: true. They are very rare with debug disabled. I expect lots of DEBUG messages (and I get those) but not lots of WARNs.

---

EDIT: FYI ...
This issue seems to be more related to enabling debug. As mentioned above I get lots of WARN messages after enabling debug. I looked through all of the logs over the past year for Failed during write, Missing data detected, Increase values or look and below is a count of each since June 4th.

-rw-rw-r-- 1 root syslogd 6138713 Jun 11 00:01 /var/log/messages.4.gz
pmacct . . .DEBUG . . .                   = 0
Failed during write                      = 0
Missing data detected                    = 0
Increase values or look . . .            = 0

-rw-rw-r-- 1 root syslogd 4192844 Jun 18 00:01 /var/log/messages.3.gz
Jun 13 22:21:09 ipfire pakfire: CORE UPGR: Upgrading from release 174 to 175
pmacct . . .DEBUG . . .                   = 0
Failed during write                      = 0
Missing data detected                    = 0
Increase values or look . . .            = 0

-rw-rw-r-- 1 root syslogd 2445920 Jun 25 00:01 /var/log/messages.2.gz
pmacct . . .DEBUG . . .                   = 0
Failed during write                      = 0
Missing data detected                    = 0
Increase values or look . . .            = 0

-rw-rw-r-- 1 root syslogd 18082299 Jul  2 00:01 /var/log/messages.1.gz
pmacct . . .DEBUG . . .                   = 9139773
Failed during write                      = 66353
Missing data detected                    = 12286
Increase values or look . . .            = 12286

-rw-rw-r-- 1 root syslogd 542344900 Jul  3 10:21 /var/log/messages
pmacct . . .DEBUG . . .                   = 5059351
Failed during write                      = 29100
Missing data detected                    = 3769
Increase values or look . . .            = 3769

[JonMurphy]

Paulo,
Again another time I haven't followed up...

We looked at the the ZeroMQ project and we found the package hasn't been updated with a new version since the beginning of 2021. See https://github.com/zeromq/libzmq/releases. And there are many bug issues (and a few pull requests) open since that time. As far as you know, is ZMQ still active?

[paololucente]

Hi Jon ( @JonMurphy ),

As far as i can tell the project itself is active on GitHub, with mainly bug fixes. I can't comment on why there has not been a release in such a long time -- maybe you can open an issue on their GitHub to ask them directly. I myself see no signs of ZeroMQ / libzmq being abandoned.

Paolo