kubectl edit machine fails validations

[karan]

machines "karangoel-test1" was not valid:

• : Invalid value: "The edited file failed validation": [ValidationError(Machine.spec.providerConfig.value): unknown field "apiVersion" in io.k8s. apimachinery.pkg.runtime.RawExtension, ValidationError(Machine.spec.providerConfig.value): unknown field "kind" in io.k8s.apimachinery.pkg.runtime. RawExtension, ValidationError(Machine.spec.providerConfig.value): unknown field "terraformMachine" in io.k8s.apimachinery.pkg.runtime.RawExtension, ValidationError(Machine.spec.providerConfig.value): unknown field "terraformVariables" in io.k8s.apimachinery.pkg.runtime.RawExtension, ValidationError(Machine.spec.providerConfig.value): missing required field "Raw" in io.k8s.apimachinery.pkg.runtime.RawExtension]

To repro:

1. Deploy cluster
2. kubectl edit machine machineName
3. Edit something other than providerConfig (eg controlPlane version)

/cc sig-cluster-lifecycle

[krousey]

This is kubectl validation failing for some reason. I am able to successfully edit when I do kubectl edit machine <machine name> --validate=false and everything works as expected.

[karan]

Yes if I disable validation it works. We should look into why the validation is failing though.

[karan]

This seems to be an issue with newer kubectl. v1.9 and newer have this problem.

[rsdcastro]

Does this impact only Terraform or also GCP?

[karan]

It should impact gce as well.

I think @k4leung4 did some testing for this bug as well.

[k4leung4]

This is not provider specifics. As I understand it, this is an issue with the interaction of the kubectl and handling of custom types.
So it is problematic with apply/edits of machines/machinesets/machinedeployments.

[roberthbailey]

@pwittrock - is there a tracking bug for kubectl 1.9 not working properly with custom types?

[nikhita]

Noticed something similar with kubectl validation. It seems to have grown stricter since 1.9. For example,

If my deployment yaml contains spec.replicAs: 3:

• earlier kubectl versions do not complain, the deployment configuration is accepted and the json decoder unmarshals it as spec.replicas: 3 (unmarshalling is always case-insensitive).
• the newer kubectl versions check if the literal replicAs exists for the Deployment type, sees that it doesn't and complains that no such field replicAs exists. If --validate=false is used, the deployment configuration is accepted and the json decoder unmarshals normally as spec.replicas: 3.

Of course, this is true for all resources, not just deployments.

In this case, Machine.spec.providerConfig.value is of type *runtime.RawExtension:

cluster-api/pkg/apis/cluster/v1alpha1/common_types.go

Line 31 in a9f06b0

Value *runtime.RawExtension `json:"value,omitempty"`

RawExtension by itself does not have any field called apiVersion, etc:

https://github.com/kubernetes/kubernetes/blob/0ea07c40305afa845bc34eb6a73da960552c39b1/staging/src/k8s.io/apimachinery/pkg/runtime/types.go#L92-L100

Looks like kubectl notices that it does not have any field called apiVersion and complains at validation. When --validate=false is used, we skip this validation and the json decoder decodes correctly.

Afaik the only change to kubectl validation was moving to openAPI validation -- I'll check that further to see what could be going wrong and comment here if I find anything.

[pwittrock]

@seans3 Can you take a look?

[seans3]

Talked to @apelisse. Work in kube-openapi is already beginning to address this.

[seans3]

kubernetes/kube-openapi#25

[roberthbailey]

Since this is not a cluster api issues I'm going to bump it out of the alpha milestone.

[timothysc]

@detiber - please verify and, or document.

[detiber]

/help

[k8s-ci-robot]

@detiber:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ncdc]

This is most likely an issue that will be resolved with a change in kubernetes/kubernetes. I propose we move this out of v1alpha1. cc @detiber

[detiber]

/milestone Next

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[ncdc]

The referenced kube-openapi PR has been merged. I'm wondering if this is possibly fixed. Would someone have some time to test this issue and report back?

[seans3]

/remove-lifecycle stale

[timothysc]

/cc @michaelgugino

[detiber]

I just tested this with kubeadm version 1.13.1 and I am no longer able to replicate the issue. Closing.

/close

[k8s-ci-robot]

@detiber: Closing this issue.

In response to this:

I just tested this with kubeadm version 1.13.1 and I am no longer able to replicate the issue. Closing.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.