How to configure multi-homed servers?

[NielsHoogeveen]

At work, we are trying to set up an Orleans cluster on multi-homed servers, Each server is attached to two networks. while all of our client machine run in either one or the other network. The goal is to make it possible to connect as a client from any client machine in the network to the silo-cluster.

Right now, we create two silos per server where each server registers an IP-address for a corresponding network. The question arises if it is possible to register a silo to listen to multiple IP-addresses at the same time.

Each client machine connects to the cluster by explicitly listing every silo reachable within the network. When using the system store, all silos (even the ones that can't be reached) are returned to the client. Leaving the question whether it is possible to add a filter to the client such that it only tries to connect to silos with a specific sub-net mask.

[veikkoeeva]

Cross-referencing #1812 and #1135 (and #1136 (comment)). Maybe @amccool has pointers? There is a subnet setting, but I'm not familiar with it.

<edit: @NielsHoogeveen, can you share more details on your setup? I aknwoledge there are security, performance and resiliety issues one would like to use multiple NICs in server configuration. It maybe something should be really looked into for robust on-premises (or IaaS) implementations.

[NielsHoogeveen]

The setup we're talking about is actually pretty simple. There are a number of servers (at the moment 7), all of which have at least two NICs and all of which are connected to two networks.

One network is for computers connected to manufacturing machinery and one network is for all other computers. Only the latter has internet access.

The two networks are only connected to each other through the servers, otherwise, they are galvanically isolated.

Two servers are configured as domain controllers, two servers are configured as web servers with network load balancing for HA purposes, and two servers are configured as database server. The seventh server is mostly used for test purposes and software development.

All of those servers need to be accessible from both networks. The computers connected to the manufacturing machinery need access to the databases to log machine status. They need access to the web servers to retrieve work order information, and they need access to the domain controllers for naming purposes and access control.

The computers, attached to the other network, need to access the databases for ad-hoc queries, excel import etc. They need to access the web servers to create work orders, to maintain bills-of-material, to maintain recipes etc. They also need access to the domain controllers for the obvious reasons.

This setup works decent, but we do want to add a notification system. Now users have to pull data from the database or from our web application, but we want to notify them as well when certain events take place (eg. a new work order has been created, triggering a refresh of the list of work orders to process on the machines that control our manufacturing machines). Another application are real time dashboards with manufacturing information. These notifications need to become available in both networks.

To achieve this, we have installed Zookeeper on each of the seven server, creating a pretty robust store for the silo configuration.

As a test, we made a silo setup for a stream of messages informing clients that a new work order has been released. These messages are created by a grain client that runs in our web application and that sends the message right before the commit of the database transaction that releases the work order.

For this test, the silo runs on two servers, where each of the servers run two instances of the silo (one for each network). The silos are configured to work with zookeeper, using defined IP-addresses,

The clients (which receive the messages of released work orders), connect to the silos by means of the Gateway settings (fixed ip addresses), because Zookeeper returns the entire system store (including unreacheable IP-addresses to the client.

Having to duplicate the number of silos for the purpose of serving a different network is not ideal, and neither is having a static client configuration;

Hopefully this gives a bit more detail about the setup.

[sergeybykov]

2.0 supports multi-home scenarios. Feel free to reopen if that's not enough.

[veikkoeeva]

@sergeybykov Is there documentation on it, by the way? I couldn't find any (maybe I've just failed), so thinking if a ticket ought to be opened for that.

[benjaminpetit]

I don't think we have proper doc for that yet. But you can use EndpointOptions to listen to any interface and publish whatever address you want in the membership table. But it's true that, for now, the same IP address is published for both gateway and silo-to-silo communications.

In the case of @NielsHoogeveen, here are the possible configs:

1. Run a HTTP service that is accessible from both network, that will play the role of orleans client
2. If he really, really want to use orleans client from both networks:
   • I suppose that all silos are able to talk to each other, even if they are not on the same network
   • Make the silo listen to any IP
   • Inject it's own IGatewayProvider to filter/translate the gateway IP addresses read from the membership table