New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose kms timeout value via encryption config. #72540

Merged
merged 2 commits into from Jan 9, 2019

Conversation

@immutableT
Copy link
Contributor

immutableT commented Jan 3, 2019

What type of PR is this?

/kind feature
What this PR does / why we need it:
Expose, via EncryptionConfiguration, the timeout value that kube-apiserver will use when awaiting responses from kms-plugin.

This is a followup to 68585, were a timeout was added to calls from kube-apiserver to kms-plugin. This PR exposes the value of the timeout via EncryptionConfiguration. Providing the appropriate value of the timeout should be left to cluster managers who are familiar with the environment in which the cluster and the plugin operate. For example, KMS rooted in in a hardware device like (HSM/TPM) may have fairly long delays (seconds), while in memory KMS implementations would have timeouts in the range of milliseconds.
Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

kube-apiserver: When configuring integration with external KMS Providers, users  can supply timeout value (i.e. how long should kube-apiserver wait before giving up on a call to KMS).  (@immutableT )
@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 3, 2019

/assign @liggitt

@@ -123,4 +123,6 @@ type KMSConfiguration struct {
CacheSize int32
// endpoint is the gRPC server listening address, for example "unix:///var/run/kms-provider.sock".
Endpoint string
// Timeout for gRPC calls to kms-plugin (ex. 5s).

This comment has been minimized.

@liggitt

liggitt Jan 3, 2019

Member

document the default if unspecified

@@ -84,4 +84,6 @@ type KMSConfiguration struct {
CacheSize int32 `json:"cachesize,omitempty"`
// endpoint is the gRPC server listening address, for example "unix:///var/run/kms-provider.sock".
Endpoint string `json:"endpoint"`
// Timeout for gRPC calls to kms-plugin (ex. 5s).

This comment has been minimized.

@liggitt

liggitt Jan 3, 2019

Member

document the default if unspecified

@@ -123,4 +123,6 @@ type KMSConfiguration struct {
CacheSize int32
// endpoint is the gRPC server listening address, for example "unix:///var/run/kms-provider.sock".
Endpoint string
// Timeout for gRPC calls to kms-plugin (ex. 5s).
Timeout metav1.Duration

This comment has been minimized.

@liggitt

liggitt Jan 3, 2019

Member

should be *metav1.Duration so we can tell the difference between "0" and unspecified

@@ -84,4 +84,6 @@ type KMSConfiguration struct {
CacheSize int32 `json:"cachesize,omitempty"`
// endpoint is the gRPC server listening address, for example "unix:///var/run/kms-provider.sock".
Endpoint string `json:"endpoint"`
// Timeout for gRPC calls to kms-plugin (ex. 5s).
Timeout metav1.Duration `json:"timeout"`

This comment has been minimized.

@liggitt

liggitt Jan 3, 2019

Member

should be +optional and omitempty, right?

@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 3, 2019

@liggitt PTAL.

@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 4, 2019

/test pull-kubernetes-verify

@liggitt

This comment has been minimized.

Copy link
Member

liggitt commented Jan 4, 2019

The failure is legitimate. The generated files need to be regenerated and checked in

@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 4, 2019

/test pull-kubernetes-e2e-kops-aws

@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 4, 2019

thanks @liggitt

@liggitt

This comment has been minimized.

Copy link
Member

liggitt commented Jan 4, 2019

one comment on validation, then squash, lgtm otherwise

@k8s-ci-robot k8s-ci-robot added size/L and removed size/M labels Jan 4, 2019

@immutableT immutableT force-pushed the immutableT:expose-kms-timeout-in-config-v2 branch from f6bf95a to a4dc53c Jan 4, 2019

@@ -173,8 +173,16 @@ func GetPrefixTransformers(config *apiserverconfig.ResourceConfiguration) ([]val
return nil, fmt.Errorf("remote KMS provider can't use empty string as endpoint")
}

timeout := kmsPluginConnectionTimeout
if provider.KMS.Timeout != nil {
if provider.KMS.Timeout.Duration < 0 {

This comment has been minimized.

@liggitt

liggitt Jan 4, 2019

Member

what does a zero timeout do?

This comment has been minimized.

@immutableT

immutableT Jan 5, 2019

Author Contributor

@liggitt good catch - zero seems to make gRPC calls to kms-plugin fail no matter what - even when plugin is up and running before kube-apiserver makes its first call.
Based on that, changed the validation logic to prohibit zero.
Added unit test to cover timeout test cases; let me know if I should put them on a separate PR.

This comment has been minimized.

@immutableT

immutableT Jan 8, 2019

Author Contributor

@liggitt PTAL.

@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 4, 2019

/test pull-kubernetes-e2e-gce-device-plugin-gpu
/test pull-kubernetes-kubemark-e2e-gce-big

@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 5, 2019

/test pull-kubernetes-integration

@immutableT immutableT force-pushed the immutableT:expose-kms-timeout-in-config-v2 branch from a2c1836 to d1bf08d Jan 5, 2019

require timeout to be greater than zero.
add unit test to cover timeout behaviour.

@immutableT immutableT force-pushed the immutableT:expose-kms-timeout-in-config-v2 branch from d1bf08d to 39aca56 Jan 5, 2019

@liggitt

This comment has been minimized.

Copy link
Member

liggitt commented Jan 9, 2019

/lgtm
/approve

please detail the config field in the release note and open a doc PR to update the encryption docs for 1.14

@k8s-ci-robot k8s-ci-robot added the lgtm label Jan 9, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Jan 9, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: immutableT, liggitt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fejta-bot

This comment has been minimized.

Copy link

fejta-bot commented Jan 9, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

1 similar comment
@fejta-bot

This comment has been minimized.

Copy link

fejta-bot commented Jan 9, 2019

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot k8s-ci-robot merged commit 15521d0 into kubernetes:master Jan 9, 2019

19 checks passed

cla/linuxfoundation immutableT authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gke Skipped
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-e2e-kubeadm-gce Skipped
pull-kubernetes-godeps Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
tide In merge pool.
Details
@immutableT

This comment has been minimized.

Copy link
Contributor Author

immutableT commented Jan 10, 2019

Documentation PR 12158 submitted.
kubernetes/website#12158

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment