How to use proxies for distributed SimBricks simulation running on multiple physical hosts?

[2024slby]

Hi everyone,

Simbricks is powerful!Thanks for your contribution.

I want to know how to use proxies for distributed SimBricks simulation running on multiple physical hosts? I tried to use scripts named "dist_netperf.py" and "dist_multinet.py",it seems that communication between simulators still depends on "net_switch" but not "net_sockets".

And I try to write different scripts to make different simulators running on different physical hosts,such as listener runs on physical host A and connecter runs on physical host B. However, I found that if I want to use socket,connecter has to add a parameter of listener,for example, cp = proxy.SocketsNetProxyConnecter(lp).It seems that it is impossible to write the script separately. So how to use proxies on different physical hosts?

My understanding of simbricks may be not deep enough, please give me some advice, thank you!

BR,
Maria

[FreakyPenguin]

Hi Maria,

First off, thanks for reaching out! I am not sure I fully understand your question, but below some information that hopefully helps. Happy to elaborate more if you have questions.

Unfortunately we have not yet gotten around to properly document distributed simulations yet. The best starting point is probably the corresponding experiment in the Sigcomm Artifact here: https://github.com/simbricks/sigcomm22-artifact/blob/main/dist_memcache/README.md
Long story short, you only write one python script for the distributed simulation. In that script you can then manually annotate which simulators should run on which host (https://github.com/simbricks/simbricks/blob/main/experiments/pyexps/dist_netperf.py#L112C1-L116 there is also some initial functionality for doing this automatically but that is not well tested at all. The SimBricks orchestration framework then takes care of starting and stopping the individual processes on all hosts. The Prerequisite is that the machines are set up so you can ssh into them without passwords (ssh key auth etc.).

You then set up a json file with the host info (see the readme above, number of hosts needs to be at least as many as your experiment needs). You then pass that to the runtime script along with the --dist parameter. The runtime should then take care of running everything.

Note that the proxies are not meant to replace the switch at all. They just replace a shared memory queue between two components with a network connection.

Does that answer your question. If not please just follow up and I'll be happy to help!

Cheers,
Antoine