Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
High level API #455
Current Mitogen use cases aren't particularly complicated, but the API to achieve those simple things is amazingly complicated, and requires reimplementing some things the library should probably handle out of the box -- like references to existent connections, parallelized connection setup, and handling unexpected disconnection.
Use cases (WIP):
Completely new desires:
Stretch goal: make it possible for all of the Ansible extension to work in terms of the high level API
The initial idea is to provide a kind of 'Session' object that does pretty much everything internally, that can be stashed away by the consumer code, and each time some activity is desired, some child object is created that specifies the connection configurations to use, activities to execute, retry/failure/timeout policies, and finally uses the Session for long-term caching/persistence of the connections it creates.
Very early/nasty API sketch:
# Thread-safe process global object, wraps up service Pool, FileService, # Broker, Router, etc. session = mitogen.session.Session() # ?? blah = session.connection(method='ssh', hostname='blah') sudo = session.connection(method='sudo', via=blah) # Thread-safe(?) per-task object, wraps CallChain, connection error management, # and the subset of connections that will actually be used. # The objects returned by add_connect() could be Context object in future, once # async connect is done, but today might be placeholder objects used only by # the high-level API. Calling connection() with same args as previously returns # the same object, or maybe arranges for it to be recreated if it got # disconnected. txn = session.start_transaction() txn.set_hosts([blah, sudo]) txn.set_retry_policy(retries=3) txn.set_call_timeout(60) txn.upload_file('/path/to/file', dest='..', overwrite=True, error_if_exists=False, preserve_mtime=True, ..) txn.start_upload_file(..) # async version, works like async funcs below txn.push_directory('/path/to/dir', dest='..') # rsync-alike txn.start_push_directory(..) # async txn.download_file(..) txn.start_download_file(..) # async txn.pull_directory(..) txn.start_pull_directory(..) # async # Array of CallResult objects? With .connection, .retries, .finished, .result, .. ? results = txn.call(func, ...) # Simpler async interface? txn.start_call(func, ...) for status_update in txn.iter_status_update(): # use some funky neat status update abstraction print(status_update.hostname, status_update.value) txn.wait_for_completion() for failure in txn.get_failures(): print('Failed: ', failure.hostname, failure.reason)
Prehistoric Mitogen had an 'ultra simple' API, where the session object was implicit. Maybe it's going too far?