New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python 3 master <==> python 2 worker, does not work #3416

Closed
rodrigc opened this Issue Jul 5, 2017 · 4 comments

Comments

Projects
None yet
2 participants
@rodrigc
Collaborator

rodrigc commented Jul 5, 2017

From the buildbot master branch, I experimented with setting up
a Python 3.6 master and a Python 2.7 worker. I was able to start the master and worker,
and saw that they connected to each other.

However, when I triggered a build, I got:

Traceback (most recent call last):
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1442, in gotResult
    _inlineCallbacks(r, g, deferred)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
--- <exception caught here> ---
  File "/home/vagrant/buildbot/master/buildbot/process/buildstep.py", line 574, in startStep
    self.results = yield self.run()
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1386, in _inlineCallbacks
    result = g.send(result)
  File "/home/vagrant/buildbot/master/buildbot/process/buildstep.py", line 704, in run
    results = yield self._start_deferred
  File "/home/vagrant/buildbot/master/buildbot/steps/source/git.py", line 171, in startVC
    gitInstalled = yield self.checkBranchSupport()
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/vagrant/buildbot/master/buildbot/steps/source/git.py", line 583, in checkBranchSupport
    stdout = yield self._dovccmd(['--version'], collectStdout=True)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/vagrant/buildbot/master/buildbot/steps/source/git.py", line 381, in _dovccmd
    yield self.runCommand(cmd)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/internet/defer.py", line 1384, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/home/vagrant/buildbot-venv/lib64/python3.6/site-packages/twisted/python/failure.py", line 393, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/home/vagrant/buildbot/master/buildbot/process/buildstep.py", line 900, in runCommand
    res = yield command.run(self, self.remote, self.build.builder.name)
buildbot.process.remotecommand.RemoteException: b'exceptions.AssertionError': b"Unexpected usePTY argument value: u'slave-config'. Expected boolean."
b'Traceback (most recent call last):\n  File "/home/vagrant/buildbot-venv-2.7/lib/python2.7/site-packages/twisted/spread/pb.py", line 1054, in _recvMessage\n    netResult = object.remoteMessageReceived(self, message, netArgs, netKw)\n  File "/home/vagrant/buildbot-venv-2.7/lib/python2.7/site-packages/twisted/spread/flavors.py", line 125, in remoteMessageReceived\n    state = method(*args, **kw)\n  File "/home/vagrant/buildbot/worker/buildbot_worker/base.py", line 150, in remote_startCommand\n    d = self.command.doStart()\n  File "/home/vagrant/buildbot/worker/buildbot_worker/commands/base.py", line 163, in doStart\n    d = defer.maybeDeferred(self.start)\n--- <exception caught here> ---\n  File "/home/vagrant/buildbot-venv-2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 150, in maybeDeferred\n    result = f(*args, **kw)\n  File "/home/vagrant/buildbot/worker/buildbot_worker/commands/shell.py", line 47, in start\n    logEnviron=args.get(\'logEnviron\', True),\n  File "/home/vagrant/buildbot/worker/buildbot_worker/runprocess.py", line 388, in __init__\n    usePTY)\nexceptions.AssertionError: Unexpected usePTY argument value: u\'slave-config\'. Expected boolean.\n'
@rodrigc

This comment has been minimized.

Show comment
Hide comment
@rodrigc

rodrigc Jul 5, 2017

Collaborator

According to @tardyp:

'slave-config' is an old marker which is only valid for eight workers.
https://github.com/buildbot/buildbot/blob/master/master/buildbot/process/remotecommand.py#L404

it is set here:
if self.step.workerVersionIsOlderThan("shell", "3.0"):
# Old worker default of usePTY is to use worker-configuration.
self.args['usePTY'] = "slave-config"

but it shouldn't as you are using a nine worker. I think maybe workerVersionIsOlderThan may not work because of nativeString use.

Pierre

Collaborator

rodrigc commented Jul 5, 2017

According to @tardyp:

'slave-config' is an old marker which is only valid for eight workers.
https://github.com/buildbot/buildbot/blob/master/master/buildbot/process/remotecommand.py#L404

it is set here:
if self.step.workerVersionIsOlderThan("shell", "3.0"):
# Old worker default of usePTY is to use worker-configuration.
self.args['usePTY'] = "slave-config"

but it shouldn't as you are using a nine worker. I think maybe workerVersionIsOlderThan may not work because of nativeString use.

Pierre

@rodrigc

This comment has been minimized.

Show comment
Hide comment
@rodrigc

rodrigc Jul 7, 2017

Collaborator

I traced the problem to this line: https://github.com/buildbot/buildbot/blob/master/master/buildbot/worker/base.py#L359

        self.worker_commands = conn.info.get("worker_commands", {})
        self.worker_environ = conn.info.get("environ", {})
        self.worker_basedir = conn.info.get("basedir", None)
        self.worker_system = conn.info.get("system", None)

If the master is py3, and the worker is py2, conn.info has a key b"worker_commands" on py3, which is not the same as "worker_commands". So self.worker_commands gets assigned to {}.

Collaborator

rodrigc commented Jul 7, 2017

I traced the problem to this line: https://github.com/buildbot/buildbot/blob/master/master/buildbot/worker/base.py#L359

        self.worker_commands = conn.info.get("worker_commands", {})
        self.worker_environ = conn.info.get("environ", {})
        self.worker_basedir = conn.info.get("basedir", None)
        self.worker_system = conn.info.get("system", None)

If the master is py3, and the worker is py2, conn.info has a key b"worker_commands" on py3, which is not the same as "worker_commands". So self.worker_commands gets assigned to {}.

@seankelly

This comment has been minimized.

Show comment
Hide comment
@seankelly

seankelly Jul 11, 2017

Member

Does a Python 3 worker send a string "worker_commands" then?

Member

seankelly commented Jul 11, 2017

Does a Python 3 worker send a string "worker_commands" then?

@rodrigc

This comment has been minimized.

Show comment
Hide comment
@rodrigc

rodrigc Jul 11, 2017

Collaborator

Yes

Collaborator

rodrigc commented Jul 11, 2017

Yes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment