New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Importer should warn for __main__ imports lacking execution guard #366

Closed
RedheatWei opened this Issue Sep 12, 2018 · 3 comments

Comments

Projects
None yet
2 participants
@RedheatWei

RedheatWei commented Sep 12, 2018

Background:
I want sync files with mitogen, I think I can read file content as a variable send to other machine, this is my test code. But it has error,I don't know why .
And have some other programme to sync files?
Thanks!

Environment:Python 3.6.5rc1, macOS High Sierra 10.13.6, mitogen 0.2.2

#!/usr/bin/env python
#
'''
file t1.py
'''
import mitogen.master
import mitogen.select
import subprocess
import logging
import mitogen.utils
import time
import os

def read_local():
    with open('/Users/Redheat/Documents/code/easou/ops/manage.py','rb') as f:
        return f.read()


def main(router):
    local = router.local(debug=True)
    file_content = local.call(read_local)
    hostnames = ['192.168.101.25', '192.168.101.27', '192.168.101.29']
    remote_hosts = [router.ssh(hostname=hostname, username='root', check_host_keys='accept') for hostname in hostnames]
    rc = [remote_host.call(write_local,file_content) for remote_host in remote_hosts]
    print('Command return code was:', rc)

def write_local(file_content):
    with open('/tmp/ops.sql', 'wb') as f:
        f.write(file_content)


logging.basicConfig(level=logging.INFO)
mitogen.utils.run_with_router(main)

[Redheat@Redheat-Mac/ops]$python3 t1.py 
Traceback (most recent call last):
  File "t1.py", line 30, in <module>
    mitogen.utils.run_with_router(main)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/utils.py", line 100, in run_with_router
    return func(router, *args, **kwargs)
  File "t1.py", line 18, in main
    file_content = local.call(read_local)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 1027, in call
    return receiver.get().unpickle(throw_dead=False)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/core.py", line 487, in unpickle
    raise obj
mitogen.core.CallError: builtins.AttributeError: 'Router' object has no attribute 'importer'
  File "<stdin>", line 2049, in _dispatch_calls
  File "<stdin>", line 2033, in _dispatch_one
  File "<stdin>", line 367, in import_module
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 656, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 626, in _load_backward_compatible
  File "<stdin>", line 811, in load_module
  File "master:t1.py", line 30, in <module>
    mitogen.utils.run_with_router(main)
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/utils.py", line 100, in run_with_router
    return func(router, *args, **kwargs)
  File "master:t1.py", line 17, in main
    local = router.local(debug=True)
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 1286, in local
    return self.connect(u'local', **kwargs)
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 1256, in connect
    return self._connect(klass, name=name, **kwargs)
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 1239, in _connect
    stream.connect()
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 958, in connect
    self.pid, fd, extra_fd = self.start_child()
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 948, in start_child
    args = self.get_boot_command()
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 903, in get_boot_command
    preamble_compressed = self.get_preamble()
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 939, in get_preamble
    self.get_econtext_config(),
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 930, in get_econtext_config
    'whitelist': self._router.get_module_whitelist(),
  File "master:/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/mitogen/parent.py", line 1214, in get_module_whitelist
    return self.importer.whitelist

@dw

This comment has been minimized.

Owner

dw commented Sep 12, 2018

Wrap the last 2 lines of your script in if __name__ == "__main__" :) The problem is your script starts re-executing on the child the moment it is imported, and the child only has the minimum code loaded at that point.

This is still a bug - the library should have warned you about an import of __main__ where it could not detect an execution guard.

Thanks for reporting!

@dw dw added bug importer labels Sep 12, 2018

@dw dw changed the title from running user function error! builtins.AttributeError: 'Router' object has no attribute 'importer' to Importer should warn for __main__ imports lacking execution guard Sep 12, 2018

@RedheatWei

This comment has been minimized.

RedheatWei commented Sep 12, 2018

@dw
Thank you, dw.

dw added a commit that referenced this issue Oct 1, 2018

@dw dw closed this in 4146648 Oct 2, 2018

@dw

This comment has been minimized.

Owner

dw commented Oct 2, 2018

This is now on the master branch and will make it into the next release. To be updated when a new release is made, subscribe to https://www.freelists.org/list/mitogen-announce

Thanks for reporting this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment