Restrict resource usage more strictly to the devices actually used

[illuhad]

The queue pool of multi_stream_executor currently covers all devices of the same backend. This can cause resources to be allocated on devices even if those are not used by the program.
We should think about making this more restrictive so that queues and other resources are only allocated on the devices that are actually used.

See also #1246 #1318 #1318 (review)