QueueServer is a tool which utilises python's multiprocessing BaseManager and SyncManager to initialize and run queueservers on both local and remote machines.
QueueServer demonstrates a working example (but not limited to) of factorizing a list of supplied numbers and collecting results in an infinite loop. QueueServer can run anywhere in a remote machine and can connect to multiple clients and distribute the job.
- Runner Base implementation for runners that run any callable in single or batch
- ProcRunner Runners that run a single process or a batch of repeted process on ProcessPoolExecutor
- ThreadRunner Runners that run multiple threads on ThreadPoolExecuto
- process augmentation using pipes on runner objects
For further reference visit SyncManager
-
open a terminal and clone the repository from github
git clone && cd queue_server -
create a virtual environment in the folder
python3 -m venv venvand activate virtual environment,source venv/bin/activate -
intall requirements file 'requirement.txt'
pip3 install -r requirements -
edit configurations YAML at 'config.yaml'
-
start the queue server,
python3 src/serverman.py -
open another terminal and run,
source venv/bin/activateto run a single client:python3 src/clientman.py -
open as many terminals as needed (for each new client) and run the clientman.py to visualize the queue job distribution
-
run serverman.py in any server machine and execute
python3 serverman.pyand connect it from local machinepython3 clientman.py.
- Eli Bendersky (for simple factorization)
- Awesome README