This project contains the code necessary to run a NextCloud instance on Kubernetes.
Derivations from Guiding principles
@TODO(mattjmcnaughton) We have not yet implemented the guiding principles for this application, so we can't discuss how it violates them.
There were a couple of architecture decisions that we debated which are worth documenting:
Deployment vs StatefulSet
As best I can tell, NextCloud unfortunately is a stateful application.
Specifically, it creates a
config.php with a username/password that it creates
during the installation process. Any additional NextCloud container must have
config.php file, or else NextCloud thinks it needs to redo the
installation, which will fail. However, since NextCloud itself creates this
file, we can't easily bind it as a read only volume on multiple pods.
We considered a couple different options. First, we could create a persistent
volume claim in the ReadWriteMany access mode,
meaning we'd attach the volume containing
config.php to many pods. We could then use a StatefulSet resource to ensure
only one pod was initializing the
config.php, and then all remaining pods
would see it already exists and they don't need to create it. Unfortunately,
this does not work because EBS volumes cannot be bound in the ReadWriteMany
Alternatively, we could generate the
config.php file as a one-off job, store
it as a Kubernetes secret, and the attach it in read only mode to each pod.
This solution felt slightly too complex to be worth it.
Finally, we decided to limit our NextCloud replicas to 1, and ensure
config.php was written to a persistent volume in the ReadWriteOnce access
mode. With this setup, we retain
config.php if our NextCloud pod is
terminated/killed, and don't have to do any complicated initialization steps.
We can revisit this decision if we discover there's a real need for multiple NextCloud pods.
Manually Executing Commands on Boot
Specifying that we need to run
kubectl exec ... some-script whenever we first
create a new NextCloud deployment feels like a bit of an anti-pattern,
as we would like deploying NextCloud to be one-touch. However, we settled on it
because the other options had significant challenges.
We considered two other options. First, we thought about creating a
would be responsible for executing this script. We could then execute this
when necessary (and could gate its execution via a flag in
However, we ran into the same problem as before with the multiple replica pods
for NextCloud. Our script needs
config.php to execute, yet currently only one
pod can own
We also considered configuring our NextCloud pod to run this script during its init process. However, the commands in this script will only succeed once. It felt like a bit of an anti-pattern to be continuously running a script we expect to fail. If we swallowed failures, there'd be the risk that we'd miss an actual legitimate failure.
For simplicities sake, we decided to just execute the script in the currently
running pod (with the
config.php) and trust the user to remember to run it.
Considering the user cannot log in if they do not exec the script, there seems
to be a fairly small chance of them forgetting.
Running the backing database
We also had to decide how to run the database NextCloud needs to store state.
The simplest option would have been using Sqlite, which is what NextCloud does by default. We could have mounted a persistent volume to wherever Sqlite writes data, meaning loosing pod would not loose our data. However, we were fairly considered about how using Sqlite would impact performance. We felt we needed to use Postgres to get acceptable performance.
After deciding to use Postgres, we had two options: pay for a hosted Postgres instance (i.e. RDS) or self-host the database on our Kubernetes cluster. We ultimately decided against RDS for multiple reasons. First, it was non-trivially expensive, even if we utilized the smallest database. Additionally, the support for creating a RDS database using Kubernetes primitives wasn't as defined as we would like. Finally, we weren't sure if introducing RDS would cause us to sacrifice dev/prod parity.
With these concerns in mind, we decided to at least attempt self hosting our Postgres database on Kubernetes. We chose to deploy via the Postgresql Helm Chart because we are already utilizing Helm as the basis for our deployments. Another option we considered was kubedb, but we had difficultly getting it to work with a couple hours of experimentation, although we'd be interested to examine it again.
Ultimately, while we went with self-hosted Postgresql deployed via a Helm chart, we imagine we'll revisit this decision in the future as both the technologies we decided against mature.
After first deploying this application (and more specifically the database backing the application), we must manually execute a series of commands to enable the necessary apps, create the non-admin users, etc. We can do so with the following:
kubectl exec -it POD -- /bin/manual-first-boot.sh
We should only need to run this script once.
You can transform
secret-values.yaml with the following:
export NEXTCLOUD_ADMIN_USERNAME=... export NEXTCLOUD_ADMIN_PASSWORD=... ... (see full list of environment variables in secret-values.yaml.sample) cat secret-values.yaml.sample | envsubst > secret-values.yaml
Our next-cloud chart has a dependency on the
chart. We specify the dependency via the
If you update the
requirements.txt, you must run
helm dep update.
In the interest of releasing MVPs, we deployed our NextCloud application before it had all of the functionality we initially intended. The relevant tickets are linked below.