New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ansible_mitogen strategy not installed by pip #85

Closed
MartinNowak opened this Issue Feb 17, 2018 · 12 comments

Comments

Projects
None yet
3 participants
@MartinNowak
Copy link

MartinNowak commented Feb 17, 2018

Using pip 9.0.1, only the following ansible_mitogen modules are installed.

site-packages/ansible_mitogen/__init__.py
site-packages/ansible_mitogen/helpers.py
site-packages/ansible_mitogen/mixins.py
@arbabnazar

This comment has been minimized.

Copy link

arbabnazar commented Feb 17, 2018

@dw I second @MartinNowak. It didn't install properly

@dw dw closed this in ca573c1 Feb 18, 2018

@dw

This comment has been minimized.

Copy link
Owner

dw commented Feb 18, 2018

I've put it on PyPI as a one-off for testing. The problem with it being on PyPI is that it tends to generate tons of bug reports from people who think the software is ready when it is not (I've seen this before :) ).

@dw

This comment has been minimized.

Copy link
Owner

dw commented Feb 18, 2018

Aah, I see I didn't read the bug report correctly -- this isn't about PyPI, it was about setup.py :) Well that is fixed now too. Sorry!

@dw dw added bug ansible labels Feb 18, 2018

@arbabnazar

This comment has been minimized.

Copy link

arbabnazar commented Feb 18, 2018

@dw Thanks it's working fine and here are some results that might be of your interest:

With Mitogen

TASK [Gathering Facts] ***************************************************************************************************************************************
12:51:45 I mitogen.ctx.ssh.common.example.com.sudo.root: stdout: exc: 'DistributionFiles' object has no attribute 'parse_distribution_file_Archlinux'

ansible-playbook -i common.example.com, common.yml   8.59s user 2.84s system 13% cpu 1:22.70 total

It stuck at gathering facts for few seconds and then shows message and after that work fine, can you please work on this to improve a bit

Without mitogen

ansible-playbook -i common.example.com, common.yml   28.44s user 7.70s system 10% cpu 5:48.91 total```
@dw

This comment has been minimized.

Copy link
Owner

dw commented Feb 18, 2018

Thanks for trying it out, and these results already look good! Can you tell me please,

  • how much latency exists between the host machine and the target? (Just ping it to find out)
  • what is the OS of the target machine
  • what version of Ansible are you using (I want to fix that ugly exception text!)
  • how many steps were executed/skipped/etc? (Final line of output from Ansible)

Most likely the reason for stalling during gather_facts is because the link is quite high latency, and that is a bug with the current importer, however it could also be some other issue.

@dw

This comment has been minimized.

Copy link
Owner

dw commented Feb 18, 2018

Please also re-install the extension, I fixed a nasty bug where it leaves temporary directories everywhere in /tmp on the target machine :/

@arbabnazar

This comment has been minimized.

Copy link

arbabnazar commented Feb 18, 2018

@dw Thanks again!

  • how much latency exists between the host machine and the target? (Just ping it to find out)
PING common.example.com (52.10.90.15): 56 data bytes
64 bytes from 52.20.90.15: icmp_seq=0 ttl=41 time=362.409 ms
64 bytes from 52.10.90.15: icmp_seq=1 ttl=41 time=384.681 ms
64 bytes from 52.10.90.15: icmp_seq=2 ttl=41 time=302.482 ms
64 bytes from 52.10.90.15: icmp_seq=3 ttl=41 time=322.624 ms
64 bytes from 52.10.90.15: icmp_seq=4 ttl=41 time=340.901 ms
64 bytes from 52.10.90.15: icmp_seq=5 ttl=41 time=259.983 ms
  • what is the OS of the target machine

sh Ubuntu 14.04.5 LTS

  • what version of Ansible are you using (I want to fix that ugly exception text!)
ansible 2.4.3.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/Users/arbab/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /Library/Python/2.7/site-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.10 (default, Jul 15 2017, 17:16:57) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]
  • how many steps were executed/skipped/etc? (Final line of output from Ansible)
common.example.com         : ok=37   changed=1    unreachable=0    failed=0

I have re-install the extension using:

pip install -U git+https://github.com/dw/mitogen.git

Is it OK or I need to uninstall and install again?

@dw

This comment has been minimized.

Copy link
Owner

dw commented Feb 18, 2018

Nope, the command you used is fine. pip install -U forces 'upgrade', which means de/reinstall

@arbabnazar

This comment has been minimized.

Copy link

arbabnazar commented Feb 18, 2018

@dw can you please explain bit about this message:

12:51:45 I mitogen.ctx.ssh.common.example.com.sudo.root: stdout: exc: 'DistributionFiles' object has no attribute 'parse_distribution_file_Archlinux'

Thanks

@dw

This comment has been minimized.

Copy link
Owner

dw commented Feb 18, 2018

It is a bug in the setup module that previously went unreported by Ansible, because the setup module continued to exit successfully, and stdout output was otherwise discarded by Ansible.

Mitogen logs stdout output by default, so there are probably many more cases of buggy modules logging junk to the stdout. Therefore I will need to handle this somehow, probably by hiding stdio output except when "ansible-playbook -v" is used, or similar

@arbabnazar

This comment has been minimized.

Copy link

arbabnazar commented Feb 18, 2018

@dw mitogen is production ready? I have fleet of Ubuntu servers ~100 that we are managing through ansible. Can I use it in production. Please recommend. Thanks

@dw

This comment has been minimized.

Copy link
Owner

dw commented Feb 18, 2018

@arbabnazar I would be extremely careful about deploying it for the time being, without thoroughly testing every playbook you have in a staging environment beforehand, with the same variables as would be used in production. Ansible simply has too many moving parts to be sure I haven't missed anything. My big worries are around options like delegate_to, and some internals that are overridden that accept a username that does not appear to be needed any more.

The result is that it's possible that some commands may run as the wrong user or on the wrong host, which could be disastrous This code is less than a week old. It's worth remembering that just 3 hours ago, it was spamming temp directories all over the target machine!

Note that I am available for consulting work, and if this would be of significant benefit to you in production, feel free to drop me an e-mail on dw@botanicus.net to discuss. Otherwise, please just wait a few months, the 'experimental' label will eventually be dropped :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment