Minio Fails to deploy on 1.9.4 due to ConfigMaps now being mounted ReadOnly

[dominik-bln]

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

BUG REPORT

Version of Helm and Kubernetes:

Helm 2.8.1
Kubernetes 1.9.4

Which chart:
stable/minio

What happened:
Since upgrading our cluster from 1.9.x to 1.9.4, Minio fails to start with the message time="2018-03-19T09:38:54Z" level=fatal msg="Unable to create configuration directories." cause="mkdir /root/.minio/certs: read-only file system" source="[server-main.go:143:serverMain()]"

What you expected to happen:

Minio starting up correctly

How to reproduce it (as minimally and precisely as possible):

Install the Minio Helm chart in an 1.9.4 cluster

Anything else we need to know:

Likely caused by this security fix:
kubernetes/kubernetes#58720

Also related:

#4166

[svmaris]

I submitted PR’s for rabbitmq-ha (#4169) and grafana (#4271) to work around this issue. I dont’t use minio myself, but if we all agree this is an acceptable fix, someone could do the same for this Chart.

[svmaris]

Related: #4267

[dominik-bln]

Thanks. We've found a workaround that was ok for us until this is fixed more permanently was to set the K8s feature gate "ReadOnlyAPIDataVolumes" to false.

Be careful though, this opens you up to this issue: http://cve.circl.lu/cve/CVE-2017-1002102

[svmaris]

@dominik-bln: That should work for now, but please note that the feature gate is marked as deprecated in k8s 1.10 and will be removed in 1.11

[seboudry]

Another quick fix is to specify (using a subPath) each files present in the configmap and mounted on minio container.
This way only specified mounted files are mounted as read-only. The whole config dir is now writeable by container.
Of course, an error will occurs if these files will be modified by minio.

Fix :

        volumeMounts:
        - mountPath: /root/.minio/config.json
          name: minio-server-config
          subPath: config.json
        - mountPath: /root/.minio/initialize
          name: minio-server-config
          subPath: initialize

[svmaris]

@Secathor: I think mounting subPaths from secrets and configmaps was broken in 1.9.4 (kubernetes/kubernetes#61080) and fixed in 1.9.5.

[seboudry]

Using this fix on 1.7.14-gke.1 for now.