Buildbot step hangs if log contains special characters

[krzk]

One of my build steps, while following log file, produces some unprintable character. In such case the step seems to hang on closing the connection (after the command was executed). When step timeout happens (or after manually hitting cancel):

  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 501, in errback
    self._startRunCallbacks(fail)
  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks
    self._runCallbacks()
  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1475, in gotResult
    _inlineCallbacks(r, g, status)
--- <exception caught here> ---
  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 577, in startStep
    self.results = yield self.run()
  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 707, in run
    results = yield self._start_deferred
  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/home/buildbot/sandbox/local/lib/python2.7/site-packages/buildbot/process/buildstep.py", line 707, in run
    results = yield self._start_deferred
exceptions.TypeError: 'unicode' does not have the buffer interface

Example is one of last steps here ("Power off: odroidxu3_1"):
https://krzk.eu/#/builders/1/builds/2573

The command which causes it is a Python code running pexpect which spawns the picocom (serial/terminal emulation program). The special character goes to picocom which stores it under log file. The log file is read by Buildbot worker. The pexpect and picocom work fine, they finish work and step should finish itself as well... but it does not. It waits and waits and finally prints error as above.

Everything works fine if the log is not created.

Configuration:
Master: Python 2.7.6, Buildbot 1.3.0 (from pip under sandbox, Ubuntu, x86)
Worker: Python 3.7.0, Buildbot 1.3.0 (from pip under sandbox, Arch ARM Linux, ARMv7)

The step which causes it is here:
https://github.com/krzk/tools/blob/1a299defb4d8bbef8a63ddeaef7d578585b28684/buildbot/master_boot.py#L382

Best regards,
Krzysztof

[aclight]

@krzk Did you ever figure out what the problem was? I'm seeing the same problem with one of my workers.

[krzk]

@aclight
I just use workaround, see variable "no_special_chars" in: https://github.com/krzk/tools/blob/master/buildbot/master_boot.py#L108

[aclight]

@krzk Thanks. I eventually tried doing effectively the same thing but that didn't completely eliminate the problem, which I don't understand. I ended up downgrading buildbot-worker to 0.9.11, which is the latest version in which things work correctly.