How to add a new worker in kubernenes cluster?

[arm2arm]

I have deployed single node example with https://docs.reana.io/administration/deployment/deploying-locally/ instructions,
Is it possible to extend the nodes?

[tiborsimko]

Hi @arm2arm, yes you can amend the number of nodes dynamically. The concrete instructions depend on how you set up your Kubernetes cluster. For example, with OpenStack Magnum, you would do something like:

$ openstack coe cluster resize mycluster 10

to add 9 new nodes to the cluster.

After extending the cluster, you would typically label the new nodes to designate their purpose, some to serve infrastructure pods (node 1 is enough), some batch pods (say nodes 2-4), the other job pods (say nodes 5-9). This may require to amend your Helm values, see https://docs.reana.io/administration/deployment/deploying-at-scale/#advanced-deployment-scenarios

If you run DB and MQ services inside the cluster, then we recommend to have dedicated nodes for them for best performance.
For example, when we deploy "extensible" REANA clusters, we typically start as follows: (using 8 CPU, 16 GB RAM node flavours):

• 1 node labelled reana.io/system=infrastructure that will run the REANA frontend and backend infrastructure services;
• 1 node labelled reana.io/system=infrastructuredb that will run the PostgreSQL DB service (unless you have some already-existing DB service running outside of the cluster that could be reused without hosting DB yourself);
• 1 node labelled reana.io/system=infrastructuremq that will run the Rabbit MQ service;
• 1 node labelled reana.io/system=runtimebatch that will run the user runtime batch workflow orchestration pods (CWL/Serial/Snakemake/Yadage);
• 1 node labelled reana.io/system=runtimejobs that will run the user runtime job pods (generated by those workflow orchestration pods).

Then you can simply add or remove new runtime batch and job nodes as the user base grows (the last two labels), all the while keeping the infrastructure nodes (and the Helm chart values) the same (the first three labels).

[arm2arm]

Thanks for the detail instructions.
Currently we don't have cluster rather a single node like this:

kubectl get nodes --show-labels
NAME                 STATUS   ROLES                  AGE   VERSION    LABELS
kind-control-plane   Ready    control-plane,master   18d   v1.21.12   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress-ready=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=kind-control-plane,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node-role.kubernetes.io/master=,node.kubernetes.io/exclude-from-external-load-balancers=

Planning to add more VMs as a workers for the reana.
So if I simply keep the every infra in this current singe node and by add VMs with reana.io/system=runtimebatch and reana.io/system=runtimejobs labels will be safe to proceed.
Probably things will be much easy if we could move to Rancher. Are there any successful stories on using reana with rancher?

[tiborsimko]

Ah, I see. Nope, if you use Kind, this is good for local developments only. You could set up more nodes with Kind, but they will run inside the same Docker host, so to speak, so it's not really good for scaling up. For a real multi-node deployment, some "real" Kubernetes cluster orchestration should be set up, using e.g. OpenStack Magnum, or OpenShift, or Rancher, or GKE, etc.

Regarding Rancher, we haven't tried lately to deploy REANA-on-Rancher, but there is some prior success in #357 I assume it should mostly work, e.g. there were no changes necessary to deploy REANA on GKE, besides configuring the shared storage properly and exposing the ports properly. Hopefully it should be smooth with latest Rancher as well...

[arm2arm]

Good to know that some one was trying Rancher!
I think we can close this issue, solution is:
To deploy reana on the real cluster not the Kind isolated or so...