Helm Chart / Kubernetes Configs

[jakehamilton]

To support deployment under other cloud services, we can provide a Helm chart (or alternative) for installation into Kubernetes clusters.

I plan to look into the existing infrastructure here and pull out what I can, but someone more familiar with the project could make progress more quickly.

[rasos]

Anybody working on kubernetes deployment files?

[jakehamilton]

Hey @rasos,

Last year was.. rough. I wasn't able to give this the attention that I wanted to. After a few months my priorities changed and I moved my attention to other projects.

[Utopiah]

@jakehamilton without actually doing it and in order to give a potential path for mozilla/hubs-cloud#135 could you please describe what would be necessary to support other cloud services using any tool?

[jakehamilton]

Hey @Utopiah,

I think the best way to manage this would be with Helm. We can create a Chart that would allow others to simply run helm install mozilla-hubs (and customizing values as necessary).

Each service in the stack would need to be built in a Docker (OCI) image in order for us to deploy them onto a Kubernetes cluster. Once the individual images are built, it is a matter of creating the standard Kubernetes configs to deploy (Deployment), persist storage (PersistentVolumeClaim), expose internally (Service), and expose externally (Ingress).

Initially, it should be fine to just worry about one instance of each service when deploying. Though, it might be worth considering how to scale the application using something like a StatefulSet.

One other thing to note is how to handle database migrations. For a single instance, using an initContainer in the Deployment may suffice.

---

I still have some trouble understanding the architecture of the Hubs stack, so I'm not entirely sure which services there are and what needs to be deployed. If someone wants to start with a very simple example using docker-compose, that'd be a good way to test these pieces running in containers and also give a rough blueprint that we can translate to Kubernetes resources.

[Utopiah]

Here is a rough view of what is running on a single Digital Ocean server including the database :

root@hubs-experiment:~# pstree
systemd─┬─accounts-daemon───2*[{accounts-daemon}]
        ├─2*[agetty]
        ├─atd
        ├─bio-launch─┬─bash─┬─logger
        │            │      └─turnserver───6*[{turnserver}]
        │            ├─beam.smp─┬─erl_child_setup───inet_gethost───2*[inet_gethost]
        │            │          └─17*[{beam.smp}]
        │            ├─bio-sup───59*[{bio-sup}]
        │            ├─dialog─┬─bash───sleep
        │            │        ├─mediasoup-worke───{mediasoup-worke}
        │            │        └─10*[{dialog}]
        │            ├─imgproxy───7*[{imgproxy}]
        │            ├─node───10*[{node}]
        │            ├─postgrest───4*[{postgrest}]
        │            ├─run───sleep
        │            ├─run───gunicorn───4*[gunicorn]
        │            ├─2*[sleep]
        │            └─11*[{bio-launch}]
        ├─cron
        ├─dbus-daemon
        ├─epmd
        ├─fail2ban-server───2*[{fail2ban-server}]
        ├─lvmetad
        ├─lxcfs───10*[{lxcfs}]
        ├─networkd-dispat───{networkd-dispat}
        ├─polkitd───2*[{polkitd}]
        ├─postgres───22*[postgres]
        ├─rsyslogd───3*[{rsyslogd}]
        ├─sshd───sshd───bash───pstree
        ├─systemd───(sd-pam)
        ├─systemd-journal
        ├─systemd-logind
        ├─systemd-network
        ├─systemd-resolve
        ├─systemd-timesyn───{systemd-timesyn}
        ├─systemd-udevd
        └─unattended-upgr───{unattended-upgr}

Unfortunately I know nothing about Helm or k8s except that I should know about them ;)

I'm starting to get the hand of docker-compose though but like you, and that's why I started with a naive pstree I'm still unsure about what services Hubs need, in what order, etc.

[pedroalvesbatista]

Any updates on this folks ?

I'm starting to design some Hubs implementations, and being able to run it in k8s would be awesome.

[Doginal]

Going to leave this here for future googlers

https://github.com/hubs-community/mozilla-hubs-ce-chart