Long log lines cause logging to stop for ipdevpoll processes in multiprocess mode

[jmbredal]

The current mode of operation for ipdevpoll's multiprocess mode is that a master process starts, spawning a set of subprocesses for each of the configured jobs. The stderr/stdout output from these subprocesses is logged to ipdevpoll.conf by the master process.

If one of the subprocesses issue an output line that happens to be larger than 16384 characters long (could happen in some debug cases, I guess), the master process will disconnect its pipe to the child process, and logging from the subprocess stops completely.

This is a limitation of the process monitor implementation in the Twisted library. It could be remedied by simply increasing the MAX_LENGTH value configured in the twisted.runner.procmon.LineLogger class.

A traceback from ipdevpoll.log when this happens may look like this:

Unhandled Error
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 88, in callWithLogger
return callWithContext({"system": lp}, func, *args, **kw)
File "/usr/lib/python2.7/dist-packages/twisted/python/log.py", line 73, in callWithContext
return context.call({ILogContext: newCtx}, func, *args, **kw)
File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 118, in callWithContext
return self.currentContext().callWithContext(ctx, func, *args, **kw)
File "/usr/lib/python2.7/dist-packages/twisted/python/context.py", line 81, in callWithContext
return func(*args,**kw)
--- ---
File "/usr/lib/python2.7/dist-packages/twisted/internet/posixbase.py", line 614, in _doReadOrWrite
why = selectable.doRead()
File "/usr/lib/python2.7/dist-packages/twisted/internet/process.py", line 257, in doRead
return fdesc.readFromFD(self.fd, self.dataReceived)
File "/usr/lib/python2.7/dist-packages/twisted/internet/fdesc.py", line 94, in readFromFD
callback(output)
File "/usr/lib/python2.7/dist-packages/twisted/internet/process.py", line 260, in dataReceived
self.proc.childDataReceived(self.name, data)
File "/usr/lib/python2.7/dist-packages/twisted/internet/process.py", line 887, in childDataReceived
self.proto.childDataReceived(name, data)
File "/usr/lib/python2.7/dist-packages/twisted/internet/protocol.py", line 593, in childDataReceived
self.errReceived(data)
File "/usr/lib/python2.7/dist-packages/twisted/runner/procmon.py", line 41, in outReceived
self.output.dataReceived(data)
File "/usr/lib/python2.7/dist-packages/twisted/protocols/basic.py", line 563, in dataReceived
return self.lineLengthExceeded(line)
File "/usr/lib/python2.7/dist-packages/twisted/protocols/basic.py", line 648, in lineLengthExceeded
return self.transport.loseConnection()
exceptions.AttributeError: DummyTransport instance has no attribute 'loseConnection'

---

Imported from Launchpad using lp2gh.

• date created: 2017-01-12T08:10:42Z
• owner: mbrekkevold
• assignee: mbrekkevold
• the launchpad url was https://bugs.launchpad.net/bugs/1655855

[jmbredal]

(by mbrekkevold)
fix here: https://nav.uninett.no/hg/nav/rev/55b22df20275

[lunkwill42]

git version of fix: 680c7ff

[jmbredal]

Translated changeset references:
https://nav.uninett.no/hg/nav/rev/55b22df20275: 680c7ff