CRD cannot have a field named 'items'

[mfranczy]

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug

What happened:
I cannot have a resource which has a field named items it doesn't pass validation.
As an example I will modify sample-controller (https://github.com/kubernetes/sample-controller) and add items to artifacts/examples/example-foo.yaml:

apiVersion: samplecontroller.k8s.io/v1alpha1
kind: Foo
metadata:
  name: example-foo
spec:
  deploymentName: example-foo
  replicas: 1
  items: # <-- cannot be set
  - "foo"
  - "bar"

after executing:

kubectl create -f artifacts/examples/example-foo.yaml

I receive an error:

The Foo "example-foo" is invalid: []: Invalid value: map[string]interface {}{"apiVersion":"samplecontroller.k8s.io/v1alpha1", "kind":"Foo", "metadata":map[string]interface {}{"name":"example-foo", "namespace":"default", "generation":1, "creationTimestamp":"2018-09-10T10:31:05Z", "uid":"9f41ebca-b4e4-11e8-ab9b-525500d15501", "selfLink":"", "clusterName":""}, "spec":map[string]interface {}{"deploymentName":"example-foo", "items":[]interface {}{"foo", "bar"}, "replicas":1}}: validation failure list:
type in spec is required

Validation expects that I will set a type field as well, spec below doesn't return the error:

apiVersion: samplecontroller.k8s.io/v1alpha1
kind: Foo
metadata:
  name: example-foo
spec:
  deploymentName: example-foo
  replicas: 1
  items:
  - "foo"
  - "bar"
  type: array  # <-- with 'type' it works

After all, in my opinion, it is a bug. I think this connected to OpenApi schema validation, that items keyword should always be in pair with type.

What you expected to happen:
I would expect that I am able to use a field named items in CRD spec and it passes validation.
Similar to ConfigMap, it has an items field, https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#add-configmap-data-to-a-specific-path-in-the-volume.

How to reproduce it (as minimally and precisely as possible):
The easiest way is to use https://github.com/kubernetes/sample-controller,
change artifacts/examples/example-foo.yaml
from:

apiVersion: samplecontroller.k8s.io/v1alpha1
kind: Foo
metadata:
  name: example-foo
spec:
  deploymentName: example-foo
  replicas: 1

to:

apiVersion: samplecontroller.k8s.io/v1alpha1
kind: Foo
metadata:
  name: example-foo
spec:
  deploymentName: example-foo
  replicas: 1
  items:
  - "foo"
  - "bar"

then execute:

kubectl -f create artifacts/examples/crd-validation.yaml (it doesn't matter if you will change something here)

kubectl -f create artifacts/examples/example-foo.yaml

After mentioned steps, you should receive the error (btw you don't have to change the Foo structure to reproduce this)

Anything else we need to know?:
No.

Environment:

• Kubernetes version (use kubectl version):
  Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:13:03Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
  Server Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.4", GitCommit:"5ca598b4ba5abb89bb773071ce452e33fb66339d", GitTreeState:"clean", BuildDate:"2018-06-06T08:00:59Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
  Tested as well with k8s-1.11.0, the same problem occurred.
• OS (e.g. from /etc/os-release): Fedora27
• Kernel (e.g. uname -a) :4.17.19-100.fc27.x86_64

[mfranczy]

@kubernetes/sig-api-machinery

[mfranczy]

/sig api-machinery

[roycaihw]

cc @roycaihw @mbohlool

[liggitt]

I suspect it might be related to this:

kubernetes/staging/src/k8s.io/apimachinery/pkg/apis/meta/v1/unstructured/unstructured.go

Lines 53 to 60 in 36877da

	func (obj *Unstructured) IsList() bool {
	field, ok := obj.Object["items"]
	if !ok {
	return false
	}
	_, ok = field.([]interface{})
	return ok
	}

edit: nevermind... that is nested under spec, so shouldn't trigger IsList detection

[liggitt]

cc @sttts

[sttts]

The error comes from here:

kubernetes/vendor/github.com/go-openapi/validate/object_validator.go

Line 80 in 66a4e51

res.AddErrors(errors.Required("type", o.Path))

Is this some kind of JavaScript speciality that slices are represented as objects with items and type?

[liggitt]

Is this some kind of JavaScript speciality that slices are represented as objects with items and type?

none I've ever encountered

[sttts]

Haven't found it in JSON Schema (yet) https://json-schema.org/specification.html

[sttts]

Found this example https://json-schema.org/latest/json-schema-core.html#rfc.section.8.3.2, which looks concerningly similar to what we see.

[sttts]

Comes from this PR, without any reasoning: go-openapi/validate#35

[liggitt]

it looks like something that either should be done when validating a schema, or should pull type info from the validator when validating an object, but not arbitrarily pull type info from attributes of the object being validated

[sttts]

Bug in go-openapi I believe, misunderstanding of the OpenAPI spec: go-openapi/validate#35 (comment)

[nikhita]

/assign

will fix it upstream and report back here after that's done

[marun]

This problem has also been encountered in federation with the move to generate validation schema for federated template crds: kubernetes-retired/kubefed#469

[marun]

Will it be possible for this fix to land in 1.13 (whether initially or a patch release)?

[tamalsaha]

@nikhita / @sttts , is there an workaround for this issue? And am I right to assume that this will need a server side fix?

[tossmilestone]

/assign
I will take this issue.

[tossmilestone]

@nikhita Hi, do you have any plan to work on this?

[nikhita]

@nikhita Hi, do you have any plan to work on this?

I noticed that you've created the PR upstream, so if you'd like, feel free to take it up! :)

[tossmilestone]

Hi, @nikhita, after fix the bug in go-openapi, I can't find where the k8s validate the openapischema? Could you help to point out where is the validation code?
Thanks very much! 😄

[nikhita]

Just replied to your slack message :)

…

On Thu, Mar 21, 2019 at 12:57 PM Shaw Ho ***@***.***> wrote: Hi, @nikhita <https://github.com/nikhita>, after fix the bug in go-openapi, I can't find where the k8s validate the openapischema? Could you help to point out where is the validation code? Thanks very much! 😄 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#68466 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APXA0LOJCHil1UmFnK8ov2_ohd43CsyQks5vYzRWgaJpZM4WhMzK> .

[sttts]

Where is the PR?

[tossmilestone]

@sttts This is the PR go-openapi/validate#106, and I will submit a PR later in k8s.

[shivasuri]

Hi, I just ran into this issue on on k8s 1.14. What version is this fixed in, if any?

[liggitt]

Fixed in #76124 in 1.16