DPL: improve resource management

[ktf]

Rework the way DPL assign resources to devices. This is a first
step in moving away from the hardcoded 127.0.0.1 ip address and
allows DPL workflows to run distributed without a third party
configuring the channels.

A further optimization will also allow DPL to configure ipc / shared memory when two devices happen to be assigned to the same machine.

• Allow setting the available resources from the command line.
• Only spawn devices which have a resource assigned which match the local hostname.
• If two devices are known to be on the same host, use ipc rather than tcp.

[ktf]

@aalkin

[ktf]

@aalkin this now seems to compile and work.
@knopers8 @matthiasrichter @sawenzel @shahor02 could you verify this does not break anything in your workflows?

This is meant to pave the way for shared memory usage (via DPL driver) and static multiple-host support.

[shahor02]

@ktf I've tested in heaviest reco flows, they are not affected.

[aalkin]

@ktf it seems the test_DeviceSpec and test_Graphviz both fail due to random int in place of a port number, most likely because we are not setting port and range in DefaultOffer and ComputingOffer struct itself has no default initialization.

[knopers8]

QC seems to be doing fine.

[ktf]

Ok. I am merging this and I open a new one which optimises local connections to "ipc" (i.e. unix sockets). We can then try with "inproc" (i.e. managed shared memory) but we cannot get it to work on my mac, yet.

[ktf]

Sorry, I meant ipc+shmem. This is now done in #2517.