Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ansible] Imports from module_utils fail #324

ti-mo opened this issue Jul 26, 2018 · 5 comments

[ansible] Imports from module_utils fail #324

ti-mo opened this issue Jul 26, 2018 · 5 comments


Copy link

@ti-mo ti-mo commented Jul 26, 2018

Hi, thanks for the awesome project, looks very promising so far! We're running into some import issues with some custom modules we have. As a way of sharing libraries between modules (and setting up their remote dependencies if required), we 'package' some module_utils in a separate role. This plays more or less nicely with Ansible's upstream module importer.

In the example below, the file layout is as follows:

  |- calico-modules/
    |- meta/main.yml -> depends on 'libraries' role
    |- action_plugins/ (works and imports fine)
    |- library/ (called by the action plugin)
      > from ansible.module_utils.exec_utils import find_executable
  |- libraries/module_utils/
    > 1: import os

However, any imports in those module_utils files trip up Mitogen. I've been able to get rid of most of them, but replacing os is going to be difficult. Does this scenario fall under the '“Module Replacer” style modules are not supported.' clause, and if so, what would be the recommended way of structuring this?

The full traceback is:
Traceback (most recent call last):
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible/executor/", line 138, in run
    res = self._execute()
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible/executor/", line 554, in _execute
    result =
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 115, in run
    return super(ActionModuleMixin, self).run(tmp, task_vars)
  File "/home/ansible/ansible-common/roles/calico-modules/action_plugins/", line 56, in run
    task_vars=task_vars, tmp=tmp)
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 337, in _execute_module
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 477, in invoke
    response = _invoke_forked_task(invocation, planner)
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 442, in _invoke_forked_task
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/mitogen/", line 997, in call
    return receiver.get().unpickle(throw_dead=False)
  File "/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/mitogen/", line 487, in unpickle
    raise obj
CallError: exceptions.ValueError: __package__ set to non-string
  File "<stdin>", line 2036, in _dispatch_calls
  File "<stdin>", line 2028, in _dispatch_one
  File "master:/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 263, in run_module
  File "master:/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 225, in run
  File "master:/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 535, in setup
  File "master:/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 519, in _setup_imports
  File "<stdin>", line 367, in import_module
  File "master:/home/ansible/.virtualenvs/ansible-27/lib/python2.7/site-packages/ansible_mitogen/", line 270, in load_module
    exec(code, mod.__dict__)
  File "/home/ansible/ansible-common/roles/libraries/module_utils/", line 1, in <module>

The first line of is import os.

I've tried both py2 and py3 on the controller, and the results are the same. (nice!) The remotes are CentOS7 machines running py2.

Is this a scenario Mitogen could support?

Copy link

@dw dw commented Jul 26, 2018

80% chance this is some easy fallout from the Python 3 work. Leave this with me a few days, there are some related tests checked in that don't run yet, so I'll fix that at the same time. Thanks for the report!

@dw dw added bug ansible labels Jul 28, 2018
dw added a commit that referenced this issue Jul 28, 2018
Also enable at last one of its tests.
Copy link

@dw dw commented Jul 28, 2018

Hi there,

Can you please retry with and let me know if the problem is gone? I need to rework the module_utils tests more significantly, because Ansible's semantics when overlapping module names exists is pretty unfortunate

@dw dw added the NeedsTest label Jul 28, 2018
Copy link

@ti-mo ti-mo commented Jul 30, 2018

@dw works perfectly, thank you! (tested with py2 and py3 runners)

Copy link

@dw dw commented Jul 30, 2018

This will make it into the next release. To be updated when a new release is made, subscribe to

Thanks for reporting this!

@dw dw closed this Jul 30, 2018
Copy link

@dw dw commented Jul 30, 2018

Oh heck, sorry, I shouldn't have pointed you at that branch. The tip commit had the per-CPU multiplexer patch on top. If you re-pull (assuming you're using it temporarily) that patch is gone. It may be fine (it might even be great!), but best not use it yet.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants