Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mitogen_via + transport=smart throws KeyError: u'smart' #548

Closed
elbunda opened this Issue Feb 19, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@elbunda
Copy link

commented Feb 19, 2019

Hello,

I'm getting following error while executing Ansible with mitogen:

$ ansible -m ping 'lindevtestv02.lindev.test' --ask-vault-pass -vvv
ansible 2.7.5
  config file = /home/burdalib/ansible-mitogen/ansible.cfg
  configured module search path = [u'/home/burdalib/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible
  executable location = /home/burdalib/ansible-mitogen/bin/ansible
  python version = 2.7.5 (default, Sep 12 2018, 05:31:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Using /home/burdalib/ansible-mitogen/ansible.cfg as config file
Vault password:
/home/burdalib/ansible-mitogen/hosts did not meet host_list requirements, check plugin documentation if this is unexpected
/home/burdalib/ansible-mitogen/hosts did not meet script requirements, check plugin documentation if this is unexpected
Parsed /home/burdalib/ansible-mitogen/hosts inventory source with yaml plugin
[pid 23865] 10:38:53.910936 D mitogen: mitogen.service.Pool(0x7fbc373ea450, size=32, th='MainThread'): initialized
[pid 23865] 10:38:53.912781 D ansible_mitogen.process: Service pool configured: size=32
META: ran handlers
Tuesday 19 February 2019  10:38:53 +0200 (0:00:00.258)       0:00:00.258 ******
[pid 23900] 10:38:53.997113 D mitogen: unix.connect(path='/tmp/mitogen_unix_suc5XS.sock')
[pid 23900] 10:38:53.998268 D mitogen: unix.connect(): local ID is 1, remote is 0
[pid 23865] 10:38:53.998221 D mitogen: mitogen.unix.Listener('/tmp/mitogen_unix_suc5XS.sock'): accepted mitogen.core.Stream('unix_client.23900')
[pid 23900] 10:38:54.004739 D mitogen: mitogen.core.Stream('unix_listener.23865').on_disconnect()
[pid 23900] 10:38:54.005015 D mitogen: Waker(Broker(0x7fbc36b9b150) rfd=9, wfd=10).on_disconnect()
[pid 23865] 10:38:54.005028 D mitogen: mitogen.core.Stream('unix_client.23900').on_disconnect()
[pid 23900] 10:38:54.005362 D mitogen: Router(Broker(0x7fbc36b9b150)): stats: 0 module requests in 0 ms, 0 sent (0 ms minify time), 0 negative responses. Sent 0.0 kb total, 0.0 kb avg.
The full traceback is:
Traceback (most recent call last):
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 140, in run
    res = self._execute()
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible/executor/task_executor.py", line 612, in _execute
    result = self._handler.run(task_vars=variables)
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/mixins.py", line 115, in run
    return super(ActionModuleMixin, self).run(tmp, task_vars)
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible/plugins/action/normal.py", line 46, in run
    result = merge_hash(result, self._execute_module(task_vars=task_vars, wrap_async=wrap_async))
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/mixins.py", line 331, in _execute_module
    self._temp_file_gibberish(module_args, wrap_async)
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/mixins.py", line 310, in _temp_file_gibberish
    self._connection.get_good_temp_dir()
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/connection.py", line 702, in get_good_temp_dir
    self._connect()
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/connection.py", line 749, in _connect
    stack = self._build_stack()
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/connection.py", line 657, in _build_stack
    inventory_name=self.inventory_hostname,
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/connection.py", line 623, in _stack_from_spec
    seen_names=seen_names + (spec.inventory_name(),),
  File "/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/connection.py", line 626, in _stack_from_spec
    stack += (CONNECTION_METHOD[spec.transport()](spec),)
KeyError: u'smart'

lindevtestv02.lindev.test | FAILED! => {
    "msg": "Unexpected failure during module execution.",
    "stdout": ""
}
Tuesday 19 February 2019  10:38:54 +0200 (0:00:00.045)       0:00:00.304 ******
===============================================================================
ping ------------------------------------------------------------------------------------------------------------------------------------------------ 0.05s
 ----------------------------------------------------------------------------------------------------------------------------------------------------------
[pid 23865] 10:38:54.011405 D mitogen: Waker(Broker(0x7fbc37428cd0) rfd=6, wfd=8).on_disconnect()
[pid 23865] 10:38:54.011757 D mitogen: Router(Broker(0x7fbc37428cd0)): stats: 0 module requests in 0 ms, 0 sent (0 ms minify time), 0 negative responses. Sent 0.0 kb total, 0.0 kb avg.
$ ansible-config dump --only-changed
ANSIBLE_NOCOWS(/home/burdalib/ansible-mitogen/ansible.cfg) = True
ANSIBLE_SSH_ARGS(/home/burdalib/ansible-mitogen/ansible.cfg) = -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o UserKnownHostsF
DEFAULT_CALLBACK_WHITELIST(/home/burdalib/ansible-mitogen/ansible.cfg) = [u'profile_tasks']
DEFAULT_HOST_LIST(/home/burdalib/ansible-mitogen/ansible.cfg) = [u'/home/burdalib/ansible-mitogen/hosts']
DEFAULT_LOAD_CALLBACK_PLUGINS(/home/burdalib/ansible-mitogen/ansible.cfg) = True
DEFAULT_NO_TARGET_SYSLOG(/home/burdalib/ansible-mitogen/ansible.cfg) = True
DEFAULT_ROLES_PATH(/home/burdalib/ansible-mitogen/ansible.cfg) = [u'/home/burdalib/ansible-mitogen/roles', u'/etc/ansible/roles']
DEFAULT_STRATEGY(/home/burdalib/ansible-mitogen/ansible.cfg) = mitogen_linear
DEFAULT_STRATEGY_PLUGIN_PATH(/home/burdalib/ansible-mitogen/ansible.cfg) = [u'/home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible_mitogen/plu
DEFAULT_TIMEOUT(/home/burdalib/ansible-mitogen/ansible.cfg) = 60
HOST_KEY_CHECKING(/home/burdalib/ansible-mitogen/ansible.cfg) = False
PERSISTENT_CONNECT_TIMEOUT(/home/burdalib/ansible-mitogen/ansible.cfg) = 30
$ ansible --version
ansible 2.7.5
  config file = /home/burdalib/ansible-mitogen/ansible.cfg
  configured module search path = [u'/home/burdalib/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/burdalib/ansible-mitogen/lib/python2.7/site-packages/ansible
  executable location = /home/burdalib/ansible-mitogen/bin/ansible
  python version = 2.7.5 (default, Sep 12 2018, 05:31:16) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
$ pip show mitogen
Name: mitogen
Version: 0.2.5
Summary: Library for writing distributed self-replicating programs.
Home-page: https://github.com/dw/mitogen/
Author: David Wilson
Author-email: UNKNOWN
License: New BSD
Location: /home/burdalib/ansible-mitogen/lib/python2.7/site-packages

After some tests I found out that this error happens only when:

  • ansible_transport is set to smart (default)
  • mitogen_via is set in inventory

When I tried to set ansible_transport = ssh, ping worked.
Also, when I set ansible_ssh_common_args = ' -o ProxyJump ...' instead of mitogen_via, it also worked.

Please let me know if you need more information.

dw added a commit that referenced this issue Feb 19, 2019

issue #548: always treat transport=smart as 'ssh' for mitogen_via=.
The idea behind transport=smart is to select between paramiko and
OpenSSH given the availability of connection multiplexing and/or OSX
kernel bugs. We need to make no such choice.

@dw dw closed this in 79af8ef Feb 19, 2019

dw added a commit that referenced this issue Feb 19, 2019

Merge remote-tracking branch 'origin/dmw'
* origin/dmw:
  docs: update Changelog; closes #548.
  issue #548: always treat transport=smart as 'ssh' for mitogen_via=.
@dw

This comment has been minimized.

Copy link
Owner

commented Feb 19, 2019

This is now on the master branch and will make it into the next release. To be updated when a new release is made, subscribe to https://www.freelists.org/list/mitogen-announce

Thanks for reporting this!

@fdutheil

This comment has been minimized.

Copy link

commented Feb 19, 2019

I can confirm v2.5 -> master fixed this.

@dw

This comment has been minimized.

Copy link
Owner

commented Feb 19, 2019

Hi Florient! IIRC you were having troubles with mitogen_via= / connection delegation. How are things on recent releases? Did >0.2.4 solve everything?

dw added a commit that referenced this issue Mar 6, 2019

Merge remote-tracking branch 'origin/026' into stable
* origin/026:
  docs: update Changelog for release.
  Bump version for release.
  issue #555: ansible: workaround ancient reload(sys) hack.
  issue #554: mitogen_action_script fix
  issue #554: fix Ansible 2.4 compatibility
  issue #554: don't rely on tmp_path autoremoval in test.
  issue #554: track and remove multiple make_tmp_path() calls.
  docs: update Changelog.
  docs: drastically simplify install/changelog.
  issue #552: include process identity in log messages.
  issue #550: update Changelog.
  issue #550: parent: add explanatory comment.
  issue #550: fix up TTY ioctls on WSL 2016 Anniversary Update
  docs: update Changelog.
  service: make service list optional.
  docs: update Changelog; closes #548.
  issue #548: always treat transport=smart as 'ssh' for mitogen_via=.
  docs: better intro paragraph.
  .ci: copy private key file to tempdir.
  os_fork: more doc tweaks
  os_fork: more doc tweaks
  os_fork: yet more doc tidyup
  os_fork: more doc tweaks
  os_fork: clean up docs
  .ci: import soak scripts.
  .ci: allow containers for different jobs to run simultaneously
  os_fork: python 3 fixes and tests.
  issue #535: activate Corker on 2.4 in master too.
  issue #535: update Changelog.
  issue #535: wire mitogen.os_fork into Broker and Pool.
  issue #535: parent: add create_socketpair(size=..) parameter.
  issue #535: introduce mitogen.os_fork module and Corker class.
  issue #535: docs: update Changelog
  issue #535: service: support Pool.defer() like Broker.defer()
  issue #535: core: unicode.encode() may take importer lock on 2.x
  issue #535: docs: fix up Select doc
  issue #535: docs: update Changelog.
  issue #535: core/select: support selecting from Latches.
  core: increase cookie field lengths to 64-bit; closes #545.
  tests: ensure serialization restrictions are in effect
  tests/bench: set process affinity in throughput.py.
  docs: update copyright year.
  docs: update Changelog.
  core: Make Latch.put(obj=) optional.
  docs: change 'unreleased' Changelog format and add a hint.
  docs: update Changelog; closes #542.
  issue #542: return of select poller, new selection logic
  issue #542: .ci: move some tests to Azure and enable Mac job.
  ansible: create stub __init__.py for sdist.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.