Fine-grained control of secret data in volume

[pmorie]

Currently the SecretVolumeSource does not allow the user to control:

1. Which file names a piece of secret data should be presented with
2. What permissions a piece of secret data should have

It should be possible to control both of these things.

[pmorie]

@erictune @thockin @csrwng

[pmorie]

@liggitt fyi

[derekwaynecarr]

I have a use case where a component requires 'secret' information across multiple sub-directories.

/secret-data/secret1/somefile
/secret-data/secret2/somefile
...

Both secret1 and secret2 are related, and it would be convenient to manage them as part of the same Kubernetes Secret resource.

While I am not sure if my reaction is knee-jerk, my initial bias is that I should be able to control what sub-directories a secret key can present itself under relate to relative to the mounted directory rather than all keys being files that are immediate children of the parent directory.

[mfojtik]

I have use case where MongoDB 'keyFile' requires to have 0600 permissions, so having a mechanism to set the permissions for the file in secret will be extra useful. For now, the workaround is to copy the 'keyFile' to other location and adjust the permissions manually.

[derekwaynecarr]

For my use case requiring multiple directories, we have a work around, but I am not sure that would be possible for all other situations. Sometimes it would be useful to have a tar as a secret value and I can ask it to be expanded prior to giving it to my container?

[soltysh]

My use case is to pass a SSH key through secret, with that I'm hitting too broad access permissions. SSH requires its key to be accessible only by the user owning those keys. Simple fix is possible, if I change in pkg/volume/secret/secret.go SetUpAt to do ioutil.WriteFile(hostFilePath, data, 0400) it'll work, but just because I'm using root user. What if I'll try to user different user? There should also be possibility to define the user who will be the owner of that file, so he can safely use it, without additional hacks I'm using currently to modify those permissions.

[bgrant0607]

0400 seems like a reasonable default for secrets.

In general, enabling permissions to be set on generated volumes (as opposed to attached or exposed volumes) would be useful.

[derekwaynecarr]

SSH keys is one example of where this is useful and almost required.

[pmorie]

Related: #7554

[pmorie]

As discussed in #6477, we have now solved all of the problems of validating paths. We should refactor to an API similar to the one provided for the downward API volume.

[pmorie]

Applying the pattern from the downward API and the proposed ConfigData API yeilds:

// SecretVolumeSource represents a volume containing Secret keys
// projected into files
type SecretVolumeSource struct {
  // A list of files containing secret data in the volume.
  Items []SecretVolumeFile `json:"items,omitempty"`
}

// SecretVolumeFile represents a single file containing a Secret key
type SecretVolumeFile struct {
  // A reference to the secret key to project
  SecretKeySelector `json:",inline"`
  // Path is the relative path name of the file to be created.
  // Must not be absolute or contain the '..' path.
  // Must be utf-8 encoded.
  // The first item of the relative path must not start with '..'
  Path string `json:"path"`
  // Ownership and permission information for the file.
  FileInfo FileInfo `json:"fileInfo"`
}

// SecretKeySelector selects a key of a Secret.
//
// This mirrors the ConfigDataSelector type in the proposed ConfigData API
type SecretKeySelector struct {
  // The name of the secret in the pod's namespace to select from.
  SecretName string `json:"secretName"`
  // The key of the secret to select from.  Must be a valid secret key.
  Key string `json:"key"`
}

// FileInfo contains ownership and permission information for a file.
// If FileInfo is specified for a file, that file will not be relabeled
// by ownership management.
type FileInfo struct {
  // The UID to own the file
  UID int32 `json:"uid"`
  // The GID that should own the file
  GID  int32 `json:"gid,omitempty"`
  // The unix rwxrwxrwx permissions of the file.  Only the nine least 
  // significant bits are the allowed to be set.
  Mode int32 `json:"mode"`
}

EDIT:

FileInfo is meant to implement @soltysh's use-case.

@derekwaynecarr the downward API does support your multiple-directory use-case.
cc @deads2k @liggitt

[soltysh]

👍 that should perfectly satisfy what I was looking for, at the same time being a better solution to what I proposed in #7908.

[pmorie]

@soltysh Yeah, the devil's in the details, though. We will need policy to determine what users can specify for UID and GID at least, as well as probably for the mode field.

[pmorie]

@pweil- new policy touchpoint

[deads2k]

SecretKeySelector.SecretName feels a lot like a LocalObjectReference

[deads2k]

FileInfo seems like something that should have some reasonable default behavior. What do you as a reasonable default?

[deads2k]

@pweil- new policy touchpoint

Inspecting the actual data inside of an API object would really have to be done with an admission controller that runs after defaulting. See #17549 and #17637 for details

[pmorie]

@deads2k What aspect of file info did you have in mind?

Being more explicit about the semantics, I think the presence of a FileInfo is an override for the placement of the data for a single secret key. So, the default behavior of a secret volume with no such objects would be the existing behavior.

[rata]

Hi, is someone working on this? I opened another issue (#28317) that is quite similar, although I was planning to extend it for configmaps too (as was requested there).

If someone is working on this, can we coordinate to not step over each other?

[rata]

This can be closed, as the first one is implemented via "items" in the volume and the second via the "defaultMode" on the volume or "mode" in items. And both present in a released k8s version (1.4 is the first version with both).

Thanks!

[wernight]

The defaultMode (and items.mode) seem good but they are OR'ed with something (possibly somehow fsGroup) and as a result 0400 gets transformed into 420.

Editing (kubectl edit ...) does fix it, but doing an apply (kubectl apply ...) changes 0400 into 420, so -r----- gets changed into -rw-r--r--. I don't see clearly from the documentation how to force 0400 as I've not set any fsGroup.

[rata]

@wernight using a kubectl from 1.4 release (previous releases don't have this feature) fixes that, as reported in #34445 ?

[wernight]

Yes, upgrading the client fixes that.

[zamai]

spec:
  volumes:
  - name: config-volume
    secret:
      defaultMode: 0400

Adding 0400 mode worked for me!

[rdxmb]

There should also be possibility to define the user who will be the owner of that file, so he can safely use it, without additional hacks I'm using currently to modify those permissions.

Is there a solution for that?