This is not an issue rather a question

[suravarapusuryakiran]

Hi Team,

i'm Surya a master thesis student doing my research on which orchestration tool is better for container run time application in an air gapped env for robotic industrial plant .

Since the robots stay at fixed place and are meant to do only certain tasks, we don't expose any services to external world for few security reasons, that means no internet access, all applications run on legacy systems with images pre built in local repo ! number of containers and load on the servers are also predictable .

Our applications are meant to do fixed amount of tasks on a containers with less latency and should not cross deadlines(time critical application).

i created docker swarm cluster and ran rt-app container (https://github.com/scheduler-tools/rt-app) on worker node and pulled the logs. did the same thing via kubernetes.

i'm completely shocked to see the results that, Docker swarm container has anAVG latency of 55.8 and kubernetes has 261.5
The over all run time of application / efficiency of job created inside the container via docker swarm is better than kubernetes.!

is it true what i found ? and also if i'm not wrong docker swarm containers consume less resources compare to kubernetes.! where else docker swarm is better.! It just seems for my requirement docker swarm is better.! just wanted to know if my understanding is in right direction.

[marcosnils]

👋 maybe @BretFisher can help out here since he has vast experience with both orchestrators 🙏

[suravarapusuryakiran]

Above stats i extracted from docker swarm container when kubernetes worker pods are running on same node then i saw both the logs stats are almost same.! what i kind of realised is by default kubernetes adds some load on to server.! if we are Ok with it then both seems taking same time.! correct me if i'm wrong

[marcosnils]

maybe also @mikesir87 could know someone that adds information here.

[mikesir87]

I don't have any hard numbers, but my experience has been similar. It also depends on the version/installation of Kubernetes. In my experience, using k3s has reduced that footprint quite a bit, as it reduces the number of individual processes needed to manage the cluster. But, again... I don't have hard numbers on it.

Sounds like a fun research project! If you ever publish, I'd love to read your paper!

[BretFisher]

Same as @mikesir87, Since all Swarm is just the Docker Engine (with containerd and runc) I'd expect there to be less resource usage, and we've all seen a "typical" K8s node with control plane to have ~10% CPU usage when idle. Like he said, k3s is leaner than other K8s options.

One thing, if you're testing networking, is to understand the networking abstraction. Swarm uses an overlay network by default, which is a VXLAN packet encapsulation that will add overhead to any container in Swarm talking to another container on another node.

K8s however, has many networking plugins and some, like Weave, also suffer from VXLAN encapsulation delays... but there are some that are native IP w/o that additional packet wrapping.

If you were going for max speed of resources and networking, then I'd do a K8s cluster (including k3s) that splits out your control plane nodes from your worker nodes, and only put your workload on the worker nodes, to avoid the resource competition by the control plane. Then I'd choose a wire-speed networking plugin. Calico is a popular choice and has some speed boosts lately: https://thenewstack.io/lightning-fast-kubernetes-networking-with-calico-and-vpp/

[suravarapusuryakiran]

Hi BretFisher,

i'm glad for your response, have been following your udemy courses, Great job.
Coming to my Question, it just seems that for Air gapped application that runs on legacy servers in a plant isolated from rest of the world, Docker swarm is good enough to fit the bill i believe.

I don't need autoscaling features, or API features & functionalities that expose services to external world.
i care more about speed, less resource consumption, rapid development , meets deadline / fault tolerant.

Please have a looks at the table and let me know your view points.

[marcosnils]

cc @gerhard since he recently did an podcast episode on ship it about Swarm, K8s and container orchestrators in general (https://changelog.com/shipit/47). Thought you might find this thread interesting.

Just inviting friends to this party 🎉

[suravarapusuryakiran]

Hi Marcosnils,

Thanks a lot for your suggestions , i have gone through the pod cast of [ANDREA LUZZARDI] and as he mentioned in the podcast , yes it would make sense to look at my use case and go with more suitable orchestration tool. He also mentioned that he probably consider Swarm if he was running an on-prem cluster. Even in my use case it makes sense for me to go with the swarm rather than kubernetes.!