Immediate failure when running Ansible with Mitogen

[knightsg]

I am trying Mitogen with Ansible out for the first time but when I run it I get the following error dump immediately after it starts to run the first task:

ERROR! [pid 3690] 11:58:25.628887 E mitogen: mitogen.service.Pool(0x7ff44ac33490, size=16, th='mitogen.service.Pool.7ff44ac33490.worker-0'): while invoking u'get' of u'ansible_mitogen.services.ContextService'
Traceback (most recent call last):
  File "/root/mitogen/mitogen/service.py", line 516, in _on_service_call
    return invoker.invoke(method_name, kwargs, msg)
  File "/root/mitogen/mitogen/service.py", line 253, in invoke
    response = self._invoke(method_name, kwargs, msg)
  File "/root/mitogen/mitogen/service.py", line 239, in _invoke
    ret = method(**kwargs)
  File "/root/mitogen/ansible_mitogen/services.py", line 370, in get
    reraise(*result)
  File "/root/mitogen/ansible_mitogen/services.py", line 334, in _wait_or_start
    response = self._connect(key, spec, via=via)
  File "/root/mitogen/ansible_mitogen/services.py", line 288, in _connect
    context = method(via=via, unidirectional=True, **spec['kwargs'])
  File "/root/mitogen/mitogen/parent.py", line 1277, in ssh
    return self.connect(u'ssh', **kwargs)
  File "/root/mitogen/mitogen/parent.py", line 1232, in connect
    return self._connect(klass, name=name, **kwargs)
  File "/root/mitogen/mitogen/parent.py", line 1215, in _connect
    stream.connect()
  File "/root/mitogen/mitogen/ssh.py", line 212, in connect
    super(Stream, self).connect()
  File "/root/mitogen/mitogen/parent.py", line 934, in connect
    self.pid, fd, extra_fd = self.start_child()
  File "/root/mitogen/mitogen/parent.py", line 926, in start_child
    return self.create_child(args, **self.create_child_args)
  File "/root/mitogen/mitogen/parent.py", line 264, in hybrid_tty_create_child
    disable_echo(master_fd)
  File "/root/mitogen/mitogen/parent.py", line 135, in disable_echo
    termios.tcsetattr(fd, flags, new)
error: (22, 'Invalid argument')
ERROR! [pid 3690] 11:58:25.638162 E mitogen: mitogen.service.Pool(0x7ff44ac33490, size=16, th='mitogen.service.Pool.7ff44ac33490.worker-1'): while invoking u'get' of u'ansible_mitogen.services.ContextService'
Traceback (most recent call last):
  File "/root/mitogen/mitogen/service.py", line 516, in _on_service_call
    return invoker.invoke(method_name, kwargs, msg)
  File "/root/mitogen/mitogen/service.py", line 253, in invoke
    response = self._invoke(method_name, kwargs, msg)
  File "/root/mitogen/mitogen/service.py", line 239, in _invoke
    ret = method(**kwargs)
  File "/root/mitogen/ansible_mitogen/services.py", line 370, in get
    reraise(*result)
  File "/root/mitogen/ansible_mitogen/services.py", line 334, in _wait_or_start
    response = self._connect(key, spec, via=via)
  File "/root/mitogen/ansible_mitogen/services.py", line 288, in _connect
    context = method(via=via, unidirectional=True, **spec['kwargs'])
  File "/root/mitogen/mitogen/parent.py", line 1277, in ssh
    return self.connect(u'ssh', **kwargs)
  File "/root/mitogen/mitogen/parent.py", line 1232, in connect
    return self._connect(klass, name=name, **kwargs)
  File "/root/mitogen/mitogen/parent.py", line 1215, in _connect
    stream.connect()
  File "/root/mitogen/mitogen/ssh.py", line 212, in connect
    super(Stream, self).connect()
  File "/root/mitogen/mitogen/parent.py", line 934, in connect
    self.pid, fd, extra_fd = self.start_child()
  File "/root/mitogen/mitogen/parent.py", line 926, in start_child
    return self.create_child(args, **self.create_child_args)
  File "/root/mitogen/mitogen/parent.py", line 264, in hybrid_tty_create_child
    disable_echo(master_fd)
  File "/root/mitogen/mitogen/parent.py", line 135, in disable_echo
    termios.tcsetattr(fd, flags, new)
error: (22, 'Invalid argument')
An exception occurred during task execution. To see the full traceback, use -vvv. The error was:     termios.tcsetattr(fd, flags, new)
fatal: [172.30.0.134]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""}
An exception occurred during task execution. To see the full traceback, use -vvv. The error was:     termios.tcsetattr(fd, flags, new)
fatal: [172.30.0.100]: FAILED! => {"failed": true, "msg": "Unexpected failure during module execution.", "stdout": ""

Setup
Controller
Uname: Linux XXXXXXXXXX 4.4.0-43-Microsoft #1-Microsoft Wed Dec 31 14:42:53 PST 2014 x86_64 x86_64 x86_64 GNU/Linux

Ansible:
ansible 2.4.6.0 (stable-2.4 9d08bc8242) last updated 2018/07/16 11:49:56 (GMT -700)
config file = /root/.ansible.cfg
configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/ansible/lib/ansible
executable location = /usr/local/ansible/bin/ansible
python version = 2.7.12 (default, Dec 4 2017, 14:50:18) [GCC 5.4.0 20160609]

Notes: Added 'strategy_plugins = /path/to/mitogen-0.2.1/ansible_mitogen/plugins/strategy' to defaults and ran ansible-playbook with 'ANSIBLE_STRATEGY=mitogen_linear' set.

Host(s)
Uname: Linux XXXXXXXXXXXX 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Python 2.7.6

[dw]

Very curious :/ What version of Ubuntu are you running? Is it a fairly default install, or perhaps you have some security enhancements enabled?

Thanks for the report!

[dw]

This looks like Ubuntu 14.04 going by the kernel version.

[knightsg]

Yep, it's 14.04. I's an AWS image: ami-9707c7f5 which as far as I'm aware is a standard 14.04 install.

[dw]

Thanks for the AMI hint.. will investigate this evening

…

On 24 July 2018 at 13:08, Guy Knights ***@***.***> wrote: Yep, it's 14.04. I's an AWS image: ami-9707c7f5 which as far as I'm aware is a standard 14.04 install. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#319 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAJCyoGO42yqB8lZG2ABD7j_jVctAGrks5uJ37PgaJpZM4VdS5T> .

[dw]

Oh very exciting! This isn't a target problem, it's a Windows Subsystem for Linux problem. Okay, makes a ton more sense. Give me overnight to set up a new test environment.

Thanks for the report!

[dw]

This will shortly be on the master branch and will make it into the next release. To be updated when a new release is made, subscribe to https://networkgenomics.com/mail/mitogen-announce/

Thanks for reporting this!

[guybluebat]

No worries, and thanks for the fast troubleshooting and fix!

[knightsg]

Just an update, I pulled the latest changes in the master branch and this is working for me now. Thanks again!

[zyphermonkey]

I think this problem might be back. I'm trying to run this on WSL using mitogen 2.5 with ansible 2.7.7 and getting the following during the Gather Facts step.

ERROR! [pid 17342] 16:05:31.543223 E mitogen: mitogen.service.Pool(0x7f089dd07c10, size=32, th='mitogen.service.Pool.7f089dd07c10.worker-0'): while invoking u'get' of u'ansible_mitogen.services.ContextService'
Traceback (most recent call last):
  File "/etc/ansible/mitogen-0.2.5/mitogen/service.py", line 544, in _on_service_call
    return invoker.invoke(method_name, kwargs, msg)
  File "/etc/ansible/mitogen-0.2.5/mitogen/service.py", line 274, in invoke
    response = self._invoke(method_name, kwargs, msg)
  File "/etc/ansible/mitogen-0.2.5/mitogen/service.py", line 260, in _invoke
    ret = method(**kwargs)
  File "/etc/ansible/mitogen-0.2.5/ansible_mitogen/services.py", line 467, in get
    reraise(*result)
  File "/etc/ansible/mitogen-0.2.5/ansible_mitogen/services.py", line 425, in _wait_or_start
    response = self._connect(key, spec, via=via)
  File "/etc/ansible/mitogen-0.2.5/ansible_mitogen/services.py", line 377, in _connect
    context = method(via=via, unidirectional=True, **spec['kwargs'])
  File "/etc/ansible/mitogen-0.2.5/mitogen/parent.py", line 2152, in ssh
    return self.connect(u'ssh', **kwargs)
  File "/etc/ansible/mitogen-0.2.5/mitogen/parent.py", line 2097, in connect
    **mitogen.core.Kwargs(kwargs))
  File "/etc/ansible/mitogen-0.2.5/mitogen/parent.py", line 2078, in _connect
    stream.connect()
  File "/etc/ansible/mitogen-0.2.5/mitogen/parent.py", line 1380, in connect
    self.pid, fd, diag_fd = self.start_child()
  File "/etc/ansible/mitogen-0.2.5/mitogen/parent.py", line 1354, in start_child
    return self.create_child(args, **self.create_child_args)
  File "/etc/ansible/mitogen-0.2.5/mitogen/parent.py", line 498, in hybrid_tty_create_child
    disable_echo(master_fd)
  File "/etc/ansible/mitogen-0.2.5/mitogen/parent.py", line 245, in disable_echo
    termios.tcsetattr(fd, flags, new)
error: (22, 'Invalid argument')
An exception occurred during task execution. To see the full traceback, use -vvv. The error was:     termios.tcsetattr(fd, flags, new)