Invalid CRD generated when using pkg/status.Conditions

[kraman]

What did you do?

• Built operator-sdk from master
• Used pkg/status.Conditions in my CRD

What did you expect to see?
A valid CRD with status.conditions as a array of condition objects

What did you see instead? Under which circumstances?

The CustomResourceDefinition "..." is invalid: spec.validation.openAPIV3Schema.properties[status].properties[conditions].items: Required value: must be specified

The CRD has an additionalProperties under the conditions object instead of items.

Environment

• operator-sdk version: "v0.15.0-80-g373ebf37", commit: "373ebf3751b61a37f3589687043b0eda095cf156"

• go version: go1.14 darwin/amd64

• Kubernetes version information:

Client Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.3", GitCommit:"06ad960bfd03b39c8310aaf92d1e7c12ce618213", GitTreeState:"clean", BuildDate:"2020-02-13T18:08:14Z", GoVersion:"go1.13.8", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"17", GitVersion:"v1.17.2", GitCommit:"59603c6e503c87169aea6106f57b9f242f64df89", GitTreeState:"clean", BuildDate:"2020-01-18T23:22:30Z", GoVersion:"go1.13.5", Compiler:"gc", Platform:"linux/amd64"}

• Kubernetes cluster kind: kubeadm 1.17

• Are you writing your operator in ansible, helm, or go?

Go

[camilamacedo86]

The K8s API is warning you that the syntax used in your CRD is invalid.

The CustomResourceDefinition "..." is invalid: spec.validation.openAPIV3Schema.properties[status].properties[conditions].items: Required value: must be specified

I understand that you change it manually and not via the tool. It is recommended define the validations via annotations in the *_types.go and then, gen the CRDS. See : https://github.com/operator-framework/operator-sdk/blob/master/doc/user-guide.md#openapi-validation.

Please, let us know if we can close this one and if not please let us know what is missing as provide the snipe code of you CRD and _type.go.

[kraman]

@camilamacedo86 I did not modify the CRD directly. I used operator-sdk generate crds to do so.
Within my go types file I import "github.com/operator-framework/operator-sdk/pkg/status and the status struct has

   // Conditions
   // +operator-sdk:gen-csv:customresourcedefinitions.statusDescriptors=true
   // +operator-sdk:gen-csv:customresourcedefinitions.statusDescriptors.displayName="Conditions"
   Conditions status.Conditions `json:"conditions"`

I think something is wrong with how operator-sdk is generating the CRD for this type.

[camilamacedo86]

Hi @kraman,

// Conditions
// +operator-sdk:gen-csv:customresourcedefinitions.statusDescriptors=true
// +operator-sdk:gen-csv:customresourcedefinitions.statusDescriptors.displayName="Conditions"
Conditions status.Conditions json:"conditions"

The above annotations are used to just gen the CSV spec fields and not the CRD. See gen-csv
The error faced is over the openAPIV3Schema in your CRD. I think you are looking in the wrong place.

1. What is the CR applied which is facing the error?
2. Can you share the full error log?
3. Can you share its type.go definition?
4. Can you share its CRD used?

[joelanford]

@kraman Can you try adding this comment annotation in your _types.go file:

operator-sdk/pkg/status/conditions.go

Line 87 in 14e209e

// +kubebuilder:validation:Type=array

That annotation is declared on the Conditions type in the SDK, but maybe there's something weird going on where it needs to be re-declared in _types.go.

I'll try to reproduce this myself as well.

[joelanford]

Hmm, I'm unable to reproduce.

I did:

1. operator-sdk new status-operator --repo=example.com/status-operator
2. operator-sdk add api --api-version=example.com/v1 --kind=MyApp
3. Updated myapp_types.go to include the Conditions field in the Status struct with the same comments and +operator-sdk annotations
4. Ran operator-sdk generate k8s and operator-sdk generate crds successfully.

Here's the CRD produced:

apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
  name: myapps.example.com
spec:
  group: example.com
  names:
    kind: MyApp
    listKind: MyAppList
    plural: myapps
    singular: myapp
  scope: Namespaced
  subresources:
    status: {}
  validation:
    openAPIV3Schema:
      description: MyApp is the Schema for the myapps API
      properties:
        apiVersion:
          description: 'APIVersion defines the versioned schema of this representation
            of an object. Servers should convert recognized schemas to the latest
            internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
          type: string
        kind:
          description: 'Kind is a string value representing the REST resource this
            object represents. Servers may infer this from the endpoint the client
            submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
          type: string
        metadata:
          type: object
        spec:
          description: MyAppSpec defines the desired state of MyApp
          type: object
        status:
          description: MyAppStatus defines the observed state of MyApp
          properties:
            conditions:
              additionalProperties:
                description: "Condition represents an observation of an object's state.
                  Conditions are an extension mechanism intended to be used when the
                  details of an observation are not a priori known or would not apply
                  to all instances of a given Kind. \n Conditions should be added
                  to explicitly convey properties that users and components care about
                  rather than requiring those properties to be inferred from other
                  observations. Once defined, the meaning of a Condition can not be
                  changed arbitrarily - it becomes part of the API, and has the same
                  backwards- and forwards-compatibility concerns of any other part
                  of the API."
                properties:
                  lastTransitionTime:
                    format: date-time
                    type: string
                  message:
                    type: string
                  reason:
                    description: ConditionReason is intended to be a one-word, CamelCase
                      representation of the category of cause of the current status.
                      It is intended to be used in concise output, such as one-line
                      kubectl get output, and in summarizing occurrences of causes.
                    type: string
                  status:
                    type: string
                  type:
                    description: "ConditionType is the type of the condition and is
                      typically a CamelCased word or short phrase. \n Condition types
                      should indicate state in the \"abnormal-true\" polarity. For
                      example, if the condition indicates when a policy is invalid,
                      the \"is valid\" case is probably the norm, so the condition
                      should be called \"Invalid\"."
                    type: string
                required:
                - status
                - type
                type: object
              description: Conditions
              type: array
          required:
          - conditions
          type: object
      type: object
  version: v1
  versions:
  - name: v1
    served: true
    storage: true

[joelanford]

It may also be a Kubernetes version issue. I successfully created that CRD in a Kubernetes 1.16 server. I'll try 1.17 as well.

[kraman]

@joelanford if you run the CRD above through openapi validator it will tell you that conditions is of type: array and so should have an items property in it. additionalProperties is only valid for type: object

[kraman]

also, adding // +kubebuilder:validation:Type=array to my go types file did not change anything.

[camilamacedo86]

HI @kraman,

Could you please provide? Note that the annotations used by you have no relation with open API issue faced and/or the values gen in the CR.

1. What is the CR applied which is facing the error?
2. Can you share the full error log?
3. Can you share its type.go definition?
4. Can you share its CRD used?

[kraman]

@camilamacedo86 yes I realize. I don't think I need to create any annotations on my types go file because the issue is related to annotations in the pkg/status/conditions.go file.

Based on what I see in #2658 (comment), @joelanford has already recreated the problem.

K8s 1.16 doesn't error on this but 1.17 does.

full error:

sample-operator ➤ kubectl apply -f deploy/crds/sample.example.com_samplecluster_crd.yaml
The CustomResourceDefinition "sampleclusters.sample.example.com" is invalid: spec.validation.openAPIV3Schema.properties[status].properties[conditions].items: Required value: must be specified

I am reluctant to share my types file due to IP issues right now but since it has already been repro'd above it may no longer be necessary to do so. I will work on clean recreation if @joelanford 's repro above is insufficient.

[joelanford]

@kraman Thanks for confirming that it seems to be an issue with tighter validations in Kubernetes 1.17.

This may be something we can fix in the operator-sdk repo, by adding more kubebuilder annotations to the pkg/status/conditions.go file, but I need to experiment to know for sure. In the meantime, you may be able to work around it by adding validation annotations in your types.go file.

It's also possible that there is a missing feature and/or bug in the controller-tools project that we'll need. As you can see, the Conditions type is not actually a slice of conditions, so it seems like the go types parser isn't quite handling this case properly.