Ensure that the runtime mounts RO volumes read-only #58720

joelsmith · 2018-01-23T23:24:39Z

What this PR does / why we need it:

This change is part of the fix to address CVE-2017-1002102 (#60814).

This change makes it so that containers cannot write to secret, configMap, downwardAPI and projected volumes since the runtime will now mount them read-only. This change makes things less confusing for a user since any attempt to update a secret volume will result in an error rather than a successful change followed by a revert by the kubelet when the volume next syncs.

It also adds a feature gate ReadOnlyAPIDataVolumes to a provide a way to disable the new behavior in 1.10, but for 1.11, the new behavior will become non-optional.

Also, E2E tests for downwardAPI and projected volumes are updated to mount the volumes somewhere other than /etc.

Which issue(s) this PR fixes
Fixes #58719

Fixes #60814 for master / 1.10

Release note:

Changes secret, configMap, downwardAPI and projected volumes to mount read-only, instead of allowing applications to write data and then reverting it automatically. Until version 1.11, setting the feature gate ReadOnlyAPIDataVolumes=false will preserve the old behavior.

joelsmith · 2018-01-23T23:27:22Z

/unassign @dims
/unassign @yujuhong
/assign @jsafrane
/assign @derekwaynecarr
/kind bug
/sig storage
/sig node

derekwaynecarr · 2018-01-24T00:24:46Z

/approve

joelsmith · 2018-01-25T03:53:59Z

/retest

jsafrane · 2018-01-25T13:28:03Z

I am afraid this is behavior change. Volumes that were not read-only are read-only now. As you can see in e2e test updates, it can break existing applications.

IMO it is the right way to go, however I am not sure a small release note is enough.

@saad-ali, what do you think?

joelsmith · 2018-01-25T17:35:31Z

/retest

joelsmith · 2018-01-25T18:09:12Z

Due to the issues raised by @jsafrane we should probably hold this PR until we get a 👍 from @smarterclayton and/or @saad-ali .

/hold

cluster/addons/dashboard/dashboard-controller.yaml

liggitt · 2018-01-25T19:39:03Z

I am afraid this is behavior change. Volumes that were not read-only are read-only now. As you can see in e2e test updates, it can break existing applications.

IMO it is the right way to go, however I am not sure a small release note is enough.

Applications that relied on persisting data in these locations were already broken, since the volume sync would remove that data at a later point in time. A release note seems sufficient to me (updated the release note with additional context)

liggitt · 2018-01-25T19:41:41Z

/retest

joelsmith · 2018-01-25T22:31:53Z

The pull-kubernetes-e2e-gke is not marked "Required" and seems to be failing for everyone right now.

* Update statefulset.yaml ConfigMaps are read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The rabbitmq-ha Chart uses a ConfigMap to provision /etc/rabbitmq. The official RabbitMQ docker image modifies these files in its docker-entrypoint.sh. This PR adds an initContainer to the StatefulSet to copy the configs from the ConfigMap to a new emptyDir volume. * Update Chart.yaml Bump version * Update statefulset.yaml

…refactor (helm#4281) * Rename manifests to align with best practices * Refactor minio chart - add ingress resource - consolidate svc resource to support all deployment modes - update labels and selectors to align with helm best practices - general cleanup to align with helm best practices/patterns observed in `helm create` - update values, README and _helpers accordingly - bump image tag - bump chart version * Fix issue caused by ConfigMaps now being mounted ReadOnly Tested on: k8s 1.8.10 and 1.9.6 Related: kubernetes/kubernetes#58720 Fixes: helm#4272 * Bump chart version to 1.0.0

…figMap to an EmptyDir (helm#4271) * Update deployment.yaml ConfigMaps are mounted read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The Grafana Chart uses a ConfigMap to provision the config- and dashboard directories. Grafana tries to create/modify files in these directories, which is not allowed anymore. This PR adds an busybox initContainer to the Deployment that copies the files from the ConfigMap to a new emptyDir, similar to helm#4169. Fixes helm#4267. * bump Chart version

See kubernetes/kubernetes#58720 The chown breaks the gitlab startup in k8s 1.9 and is not necessary.

ludwikbukowski · 2018-09-05T12:13:47Z

How can I create executable files with configMap now?

* Update statefulset.yaml ConfigMaps are read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The rabbitmq-ha Chart uses a ConfigMap to provision /etc/rabbitmq. The official RabbitMQ docker image modifies these files in its docker-entrypoint.sh. This PR adds an initContainer to the StatefulSet to copy the configs from the ConfigMap to a new emptyDir volume. * Update Chart.yaml Bump version * Update statefulset.yaml Signed-off-by: voron <av@arilot.com>

…refactor (helm#4281) * Rename manifests to align with best practices * Refactor minio chart - add ingress resource - consolidate svc resource to support all deployment modes - update labels and selectors to align with helm best practices - general cleanup to align with helm best practices/patterns observed in `helm create` - update values, README and _helpers accordingly - bump image tag - bump chart version * Fix issue caused by ConfigMaps now being mounted ReadOnly Tested on: k8s 1.8.10 and 1.9.6 Related: kubernetes/kubernetes#58720 Fixes: helm#4272 * Bump chart version to 1.0.0 Signed-off-by: voron <av@arilot.com>

…figMap to an EmptyDir (helm#4271) * Update deployment.yaml ConfigMaps are mounted read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The Grafana Chart uses a ConfigMap to provision the config- and dashboard directories. Grafana tries to create/modify files in these directories, which is not allowed anymore. This PR adds an busybox initContainer to the Deployment that copies the files from the ConfigMap to a new emptyDir, similar to helm#4169. Fixes helm#4267. * bump Chart version Signed-off-by: voron <av@arilot.com>

…refactor (#4281) * Rename manifests to align with best practices * Refactor minio chart - add ingress resource - consolidate svc resource to support all deployment modes - update labels and selectors to align with helm best practices - general cleanup to align with helm best practices/patterns observed in `helm create` - update values, README and _helpers accordingly - bump image tag - bump chart version * Fix issue caused by ConfigMaps now being mounted ReadOnly Tested on: k8s 1.8.10 and 1.9.6 Related: kubernetes/kubernetes#58720 Fixes: helm/charts#4272 * Bump chart version to 1.0.0

* Update statefulset.yaml ConfigMaps are read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The rabbitmq-ha Chart uses a ConfigMap to provision /etc/rabbitmq. The official RabbitMQ docker image modifies these files in its docker-entrypoint.sh. This PR adds an initContainer to the StatefulSet to copy the configs from the ConfigMap to a new emptyDir volume. * Update Chart.yaml Bump version * Update statefulset.yaml

…figMap to an EmptyDir (#4271) * Update deployment.yaml ConfigMaps are mounted read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The Grafana Chart uses a ConfigMap to provision the config- and dashboard directories. Grafana tries to create/modify files in these directories, which is not allowed anymore. This PR adds an busybox initContainer to the Deployment that copies the files from the ConfigMap to a new emptyDir, similar to #4169. Fixes #4267. * bump Chart version

* Update statefulset.yaml ConfigMaps are read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The rabbitmq-ha Chart uses a ConfigMap to provision /etc/rabbitmq. The official RabbitMQ docker image modifies these files in its docker-entrypoint.sh. This PR adds an initContainer to the StatefulSet to copy the configs from the ConfigMap to a new emptyDir volume. * Update Chart.yaml Bump version * Update statefulset.yaml

…figMap to an EmptyDir (#4271) * Update deployment.yaml ConfigMaps are mounted read-only since Kubernetes 1.9.4 (kubernetes/kubernetes#58720). The Grafana Chart uses a ConfigMap to provision the config- and dashboard directories. Grafana tries to create/modify files in these directories, which is not allowed anymore. This PR adds an busybox initContainer to the Deployment that copies the files from the ConfigMap to a new emptyDir, similar to #4169. Fixes #4267. * bump Chart version

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 23, 2018

k8s-ci-robot requested review from dims and yujuhong January 23, 2018 23:24

k8s-ci-robot assigned derekwaynecarr and jsafrane Jan 23, 2018

k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. sig/storage Categorizes an issue or PR as relevant to SIG Storage. sig/node Categorizes an issue or PR as relevant to SIG Node. labels Jan 23, 2018

joelsmith mentioned this pull request Jan 23, 2018

UPSTREAM: 58720: Ensure that the runtime mounts RO volumes read-only openshift/origin#18255

Merged

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 24, 2018

joelsmith force-pushed the ro-vol branch from a89d4e6 to 0184d60 Compare January 24, 2018 21:25

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 24, 2018

joelsmith force-pushed the ro-vol branch from 0184d60 to 4fd49c9 Compare January 24, 2018 23:27

joelsmith force-pushed the ro-vol branch from 4fd49c9 to 450e0ee Compare January 25, 2018 18:06

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 25, 2018

liggitt reviewed Jan 25, 2018

View reviewed changes

cluster/addons/dashboard/dashboard-controller.yaml Show resolved Hide resolved

liggitt assigned saad-ali Jan 25, 2018

rawipfel mentioned this pull request May 9, 2018

Add support to openshift v3.6 minio/minio#5903

Closed

alphashuro mentioned this pull request May 10, 2018

Logs pointer volume mounted as readonly DataDog/dd-agent#3734

Open

kocolosk mentioned this pull request May 18, 2018

Chown in docker-entrypoint.sh is clashing with Kube ConfigMap RO behavior apache/couchdb-docker#80

Closed

theinrichs pushed a commit to theinrichs/gitlab that referenced this pull request May 25, 2018

Remove chown of secrets and configmaps

39426ca

See kubernetes/kubernetes#58720 The chown breaks the gitlab startup in k8s 1.9 and is not necessary.

theinrichs mentioned this pull request May 25, 2018

Remove chown of secrets and configmaps inovex/gitlab#4

Closed

theinrichs pushed a commit to inovex/gitlab that referenced this pull request May 25, 2018

Remove chown of secrets and configmaps

657549d

See kubernetes/kubernetes#58720 The chown breaks the gitlab startup in k8s 1.9 and is not necessary.

theinrichs mentioned this pull request May 25, 2018

Remove chown of secrets and configmaps inovex/gitlab#5

Merged

tombh mentioned this pull request Jun 21, 2018

Secrets with specific permissions (defaultMode or mode) not being applied in Kubernetes 1.4.0 #34982

Closed

tsu710785 mentioned this pull request Jul 9, 2018

Error creating the cluster sobotklp/kubernetes-redis-cluster#7

Closed

jstriebel mentioned this pull request Jul 19, 2018

Update Changelog 1.9 #66374

Closed

This was referenced Jul 22, 2018

preInstallConsul job fails with Read-only file system PremiereGlobal/vault-helm-chart#42

Closed

preInstallConfig.job ReadOnly fix PremiereGlobal/vault-helm-chart#43

Merged

muratkars mentioned this pull request Dec 3, 2018

Update EFK stack demo with the latest changes openebs/openebs#2313

Closed

oneonestar mentioned this pull request Apr 9, 2019

Add Presto Dockerfile. trinodb/trino#590

Merged

hzxuzhonghu mentioned this pull request Jul 30, 2020

Fix ssh plugin volcano-sh/volcano#969

Merged

coryasilva mentioned this pull request Feb 1, 2021

PingFederate: does not seem to respect LICENSE_DIR pingidentity/pingidentity-devops-getting-started#180

Closed

tdsacilowski mentioned this pull request Jun 30, 2021

Vault Agent: make auto-delete of JWT file a configurable option hashicorp/vault#11968

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure that the runtime mounts RO volumes read-only #58720

Ensure that the runtime mounts RO volumes read-only #58720

joelsmith commented Jan 23, 2018 •

edited

Loading

joelsmith commented Jan 23, 2018

derekwaynecarr commented Jan 24, 2018

joelsmith commented Jan 25, 2018

jsafrane commented Jan 25, 2018

joelsmith commented Jan 25, 2018

joelsmith commented Jan 25, 2018

liggitt commented Jan 25, 2018 •

edited

Loading

liggitt commented Jan 25, 2018

joelsmith commented Jan 25, 2018

ludwikbukowski commented Sep 5, 2018

Ensure that the runtime mounts RO volumes read-only #58720

Ensure that the runtime mounts RO volumes read-only #58720

Conversation

joelsmith commented Jan 23, 2018 • edited Loading

joelsmith commented Jan 23, 2018

derekwaynecarr commented Jan 24, 2018

joelsmith commented Jan 25, 2018

jsafrane commented Jan 25, 2018

joelsmith commented Jan 25, 2018

joelsmith commented Jan 25, 2018

liggitt commented Jan 25, 2018 • edited Loading

liggitt commented Jan 25, 2018

joelsmith commented Jan 25, 2018

ludwikbukowski commented Sep 5, 2018

joelsmith commented Jan 23, 2018 •

edited

Loading

liggitt commented Jan 25, 2018 •

edited

Loading