Initial work on Orleans-Docker docs #3004

galvesribeiro · 2017-05-09T22:35:19Z

This PR add some guidance on how to deploy Orleans to Docker.

Initially, it explain how to create a project, add Docker support and debugger capabilities.

Swarm is described in this document. Kubernetes will be added later on.

stephenlautier · 2017-05-09T22:48:19Z

src/Documentation/Advanced-Concepts/Docker-Deployment.md

+
+Docker containers and networking model were designed to run mostly stateless and immutable containers. So, spin up a cluster running node.js or nginx applications, is pretty easy. However, if you try to use something more elaborated, like a real clustered or distributed application (like Orleans-based ones) you will eventually have troubles setting it up. It is possible, but not as simples as web-based applications.
+
+Docker clustering consist of putting together multiple hosts to work as a single pull of resources managed using a *Container Orchestrator*. *Docker Inc.* provide **Swarm** as their option for Container Orchestration while *Google* has **Kubernetes** (aka **K8s**). There are other Orchestrators like **DC/OS**, **Mesos**, etc., but in this document we will talk about Swarm and K8s as they are more widely used. 


"single pull" - i think you mean single pool*

stephenlautier · 2017-05-09T23:01:29Z

src/Documentation/Advanced-Concepts/Docker-Deployment.md

+
+# Google Kubernetes (K8s)
+
+Comming soon...


typo Coming*

Done. Thanks

stephenlautier · 2017-05-09T23:04:27Z

src/Documentation/Advanced-Concepts/Docker-Deployment.md

+services:
+  orleans-client:
+    image: orleans-client
+    depends_on: 


not entirely sure, but do you think this is really needed as it doesnt seem to be supported within swarm mode; or perhaps only for docker-compose up?

The depends_on option is ignored when deploying a stack in swarm mode with a version 3 Compose file.
https://docs.docker.com/compose/compose-file/#dependson

Ha! That is new. Didn't checked that update. It used to work... Will check with the Docker folks what should be used now. Thanks for point out.

Ok, just checked... It is only for stack. You still be able to use it with compose. Nevertheless, you are not required to use depends_on. It was just to illustrate the dependency so I think I'll stick with it there. I probably can create another section later (like the Kubernetes one) that show how to leverage stack.

To give an idea, most of the hosted Docker services like in Azure or AWS, still use the older versions of the engine, which mean, we still don't have stack available there.

But again, thanks for point that out.

welcome, and thanks for clarifying 👍

stephenlautier · 2017-05-09T23:13:19Z

src/Documentation/Advanced-Concepts/Docker-Deployment.md

+
+Docker clustering stack is called **Swarm** and you can find more by reading its [documentation here](https://docs.docker.com/engine/swarm/).
+
+To run this article in a `Swarm` cluster, you don't have any extra work. When you run `docker-compose up -d` in a `Swarm` node, it will schedule containers based on the configured rules. The same applies to other Swarm-based services like [Docker Datacenter](https://www.docker.com/enterprise-edition), [Azure ACS](https://azure.microsoft.com/en-us/services/container-service/) (in Swarm mode), [AWS ECS Container Service](https://aws.amazon.com/ecs/) and so on. All you need to do is to deploy your `Swarm` cluster before deploy your **dockerized** Orleans application.


in order to deploy on swarms, it seems to be a better approach to use

docker stack deploy -c docker-compose.yml <name>

when using v3 compose files, perhaps just to mention it - what do you think?

Yeah, like I mentioned in your previously comment. It must have stack support on the engine. I'll add a comment there for it now. Thanks

sorry posted this before saw the reply

Np, thanks :)

gabikliot · 2017-05-10T06:31:41Z

src/Documentation/Advanced-Concepts/Docker-Deployment.md

+```
+
+The string `orleans-silo` here, is essentially the **service name** used in `docker-compose`, which we will dig into more details later on (keep reading!). It will resolve to one of the containers' ips inside that same service (like a Load Balancer). In a docker-compose project, a service is a group of multiple (exactly equal) instances of a give container. For example, we will have a service for our `OrleansSilo` project. Within that service, we will be able to scale the service up and down horizontally and adding each of those containers to the Orleans cluster automatically respecting Orleans Membership Protocol. More on that later.
+


I was looking, but did not fin anything for "later".
I am puzzled how swarm clustering works along side Orleans's Membership.
If Orleans still creates a membership table in Azure, why GW is configured with IP and not pointed to the table?

Can you please explain?

@gabikliot because you are inside a Compose/Swarm project, the sane way to refer to another service, is by using the service name. Also, if the client is outside the same compose project of the silo, the IPs from the GW are not registered on Azure Table. We discussed before changes on the way membership record IPs on the tables, and everyone was against changes in MBR so the only safe way is to get the service IP, which works as a NLB and span across the cluster. It is a docker internal IP, not public, and works for the purpose of finding any of the GWs.

Let me see if I understand.
So if MBR registered the gw ip in the table, that ip is not routable from outside, so it is useless.
Instead, from outside, one has to go via load balancer end point ( which is what nlb stands for).
Correct?

So u just need to associate that end point with all gateway silos, make sure they are part of this load balanced group, and then it is a static end point for the clients to use. The Client will send to the end point, and the load balancer will route to one of the gateways. There is no way in this architecture to let client pick a gateway, be sticky to a particular gw for particular grains, etc...

Did I understand it correctly?

Yes. The point is that you don't need to register the GW IP into the load balancer. The container's IP will be always behind the service IP, which is the load balancer.

So yes, you are correct.

MBR for this case, is only useful for the cluster, not the client.

I'm still investigating Kubernetes, which is way more complicated. Will see how its pod network works.

gabikliot · 2017-05-11T13:53:08Z

Makes sense.

I still think one Could register internal gw ip in the table, as long as it is routable from outside. But there is no need to, as you said, since we can use the service end point.

galvesribeiro · 2017-05-11T15:20:18Z

Yeah, I had a big debate with others from core team on whether we should register an internal IP + router IP in MBR or not. This very same case happens when you have firewalls/routers between OrleansClient and the cluster and this firewall/router only expose 1 single IP or a different IP address for each silo. Like you already know, we tried make a MBR provider for Docker but we fail because among other things, the way we register those IPs is different from what the majority of scenarios expect to be.

In the end, I decided stop fighting and with some updates on Docker I ended up with this solution which work perfectly fine. Even in case of gateway container failure, the service endpoint is notified and the next retry will in fact redirect to another container.

gabikliot · 2017-05-11T16:10:10Z

I was not aware of this discussion.
If it was on GH, I would have had a chance to chime in.

we tried make a MBR provider for Docker but we fail

I do remember you tried. I remember that you tried to make it similar to Service Fabric, but what was not clear to me to then is if Swarm indeed provides the membership service (the list of active containers) in a way that we could feed in Orleans.

Conceptually, I don't see a problem with exposing the actual IP and not the service end point, IF they are indeed routable and secure. Orleans does its own load balancing between gateways, so it does not strickly need the load balanced end point.

I think its similar in spirit to the same high level discussion of why not use storage always and write stateless services and never do in memory state (and have to route to this state, etc)... Load balanced end point is great, but not a panacea. One should not force everyone always to use it. If we have our won solution that works better for us, why not.

galvesribeiro · 2017-05-11T16:16:18Z

@gabikliot #2571

And yes, I agree that would be awesome if we have our own membership. Maybe we can have it Kubernetes, I don't know, still need to check it.

For now, this document and the sample will unblock a lot of people trying to use Orleans and Docker with Swarm. They are eagerly asking for it and since I was the one who got it to work and is actively working with Docker daily, I decided to write it and shed some light for people. We can always revisit it later if/when we have a membership solution for it.

gabikliot · 2017-05-11T16:21:42Z

+1 on unblocking first.

sergeybykov · 2017-05-11T20:44:38Z

Thank you very very much, @galvesribeiro! This is just the beginning.

galvesribeiro · 2017-05-11T20:45:49Z

Yup! Will come up with more later. Just want to make sure people got unblocked and I stop to repeat same thing on Gitter 😄

beckjin · 2017-11-12T09:39:44Z

src/Documentation/Advanced-Concepts/Docker-Deployment.md

+            var config = new ClientConfiguration();
+            config.DeploymentId = "Orleans-Docker";
+            config.PropagateActivityId = true;
+            var hostEntry = await Dns.GetHostEntryAsync("orleans-silo");


When I use service name "orleans-silo", have a error.

Unhandled Exception: System.AggregateException: One or more errors occurred. (No such device or address)

It's my docker-compose file:

version: '3' services: orleans-silo: image: hub.xxx.com/beck/orleans/silo-test:0.0.1 volumes: - /usr/local/beck/docker/orleans/test/siloservice/config:/app/Config ~

RainingNight · 2018-11-09T07:57:21Z

Any 2.x docker sample?

Initial work on Orleans-Docker docs

56be1c2

dnfclas added the cla-already-signed label May 9, 2017

stephenlautier reviewed May 9, 2017

View reviewed changes

Feedback

629bceb

stephenlautier reviewed May 9, 2017

View reviewed changes

galvesribeiro added 2 commits May 9, 2017 20:01

Sync with sample

3fcb70e

More feedback

39a62f5

stephenlautier reviewed May 9, 2017

View reviewed changes

More feedback

f89d566

sergeybykov added this to the 1.5.0 milestone May 10, 2017

gabikliot reviewed May 10, 2017

View reviewed changes

sergeybykov merged commit 2cd9f75 into dotnet:gh-pages May 11, 2017

galvesribeiro deleted the docker-docs branch May 11, 2017 20:45

beckjin reviewed Nov 12, 2017

View reviewed changes

github-actions bot locked and limited conversation to collaborators Dec 9, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial work on Orleans-Docker docs #3004

Initial work on Orleans-Docker docs #3004

galvesribeiro commented May 9, 2017

stephenlautier May 9, 2017

galvesribeiro May 9, 2017

stephenlautier May 9, 2017

galvesribeiro May 9, 2017

stephenlautier May 9, 2017

galvesribeiro May 9, 2017

galvesribeiro May 9, 2017

stephenlautier May 9, 2017

stephenlautier May 9, 2017

galvesribeiro May 9, 2017

stephenlautier May 9, 2017

galvesribeiro May 9, 2017

gabikliot May 10, 2017

galvesribeiro May 10, 2017

gabikliot May 11, 2017

galvesribeiro May 11, 2017

gabikliot commented May 11, 2017

galvesribeiro commented May 11, 2017

gabikliot commented May 11, 2017

galvesribeiro commented May 11, 2017

gabikliot commented May 11, 2017

sergeybykov commented May 11, 2017

galvesribeiro commented May 11, 2017

beckjin Nov 12, 2017

RainingNight commented Nov 9, 2018


		Docker containers and networking model were designed to run mostly stateless and immutable containers. So, spin up a cluster running node.js or nginx applications, is pretty easy. However, if you try to use something more elaborated, like a real clustered or distributed application (like Orleans-based ones) you will eventually have troubles setting it up. It is possible, but not as simples as web-based applications.

		Docker clustering consist of putting together multiple hosts to work as a single pull of resources managed using a Container Orchestrator. Docker Inc. provide Swarm as their option for Container Orchestration while Google has Kubernetes (aka K8s). There are other Orchestrators like DC/OS, Mesos, etc., but in this document we will talk about Swarm and K8s as they are more widely used.


		Docker clustering stack is called Swarm and you can find more by reading its [documentation here](https://docs.docker.com/engine/swarm/).

		To run this article in a `Swarm` cluster, you don't have any extra work. When you run `docker-compose up -d` in a `Swarm` node, it will schedule containers based on the configured rules. The same applies to other Swarm-based services like [Docker Datacenter](https://www.docker.com/enterprise-edition), [Azure ACS](https://azure.microsoft.com/en-us/services/container-service/) (in Swarm mode), [AWS ECS Container Service](https://aws.amazon.com/ecs/) and so on. All you need to do is to deploy your `Swarm` cluster before deploy your dockerized Orleans application.

		```

		The string `orleans-silo` here, is essentially the service name used in `docker-compose`, which we will dig into more details later on (keep reading!). It will resolve to one of the containers' ips inside that same service (like a Load Balancer). In a docker-compose project, a service is a group of multiple (exactly equal) instances of a give container. For example, we will have a service for our `OrleansSilo` project. Within that service, we will be able to scale the service up and down horizontally and adding each of those containers to the Orleans cluster automatically respecting Orleans Membership Protocol. More on that later.

Initial work on Orleans-Docker docs #3004

Initial work on Orleans-Docker docs #3004

Conversation

galvesribeiro commented May 9, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gabikliot commented May 11, 2017

galvesribeiro commented May 11, 2017

gabikliot commented May 11, 2017

galvesribeiro commented May 11, 2017

gabikliot commented May 11, 2017

sergeybykov commented May 11, 2017

galvesribeiro commented May 11, 2017

Choose a reason for hiding this comment

RainingNight commented Nov 9, 2018