Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema Recognition issue in kubeconform v0.6.3 and above #247

Closed
AjanKG opened this issue Nov 23, 2023 · 4 comments
Closed

Schema Recognition issue in kubeconform v0.6.3 and above #247

AjanKG opened this issue Nov 23, 2023 · 4 comments

Comments

@AjanKG
Copy link

AjanKG commented Nov 23, 2023

Hello,

I am experiencing a schema recognition issue with kubeconform starting from version 0.6.3 and continuing in the latest version 0.6.4.
The tool fails to recognize the schema for Apigee resources (ApigeeEnvironment and ApigeeOrganization):

Actual Results:

kubeconform versions 0.6.3 and 0.6.4 fail to recognize the schema for these resources, resulting in validation error:

kubeconform -strict -summary -schema-location '/Users/kanagaratnam_a/schemas/master-standalone-strict/{{ .ResourceKind }}{{ .KindSuffix }}.json' -schema-location '/Users/kanagaratnam_a/schemas/{{ .ResourceKind }}.json' compiled
compiled/config-control/apigeeenvironment_apigeeenv-sample.yaml - ApigeeEnvironment apigeeenv-sample failed validation: could not find schema for ApigeeEnvironment
compiled/config-control/apigeeorganization_apigeeorganization-sample.yaml - ApigeeOrganization apigeeorganization-sample failed validation: could not find schema for ApigeeOrganization

The ApigeeOrganization CRD:

---
apiVersion: apigee.cnrm.cloud.google.com/v1beta1
kind: ApigeeOrganization
metadata:
  name: apigeeorganization-sample
spec:
  analyticsRegion: us-west1
  authorizedNetworkRef:
    name: apigeeorganization-dep
  displayName: Apigee Organization
  projectRef:
    name: "sample-apigee-ts"
  runtimeType: CLOUD

Expected Results:

kubeconform should correctly recognize and validate the schema of the given resource, as it did in version 0.6.2
I add Apigee Organization and Apigee Environment json schema generated

Apigee Organization:

{
  "properties": {
    "apiVersion": {
      "description": "apiVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources",
      "type": "string"
    },
    "kind": {
      "description": "kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds",
      "type": "string"
    },
    "metadata": {
      "type": "object"
    },
    "spec": {
      "properties": {
        "addonsConfig": {
          "description": "Addon configurations of the Apigee organization.",
          "properties": {
            "advancedApiOpsConfig": {
              "description": "Configuration for the Advanced API Ops add-on.",
              "properties": {
                "enabled": {
                  "description": "Flag that specifies whether the Advanced API Ops add-on is enabled.",
                  "type": "boolean"
                }
              },
              "type": "object",
              "additionalProperties": false
            },
            "monetizationConfig": {
              "description": "Configuration for the Monetization add-on.",
              "properties": {
                "enabled": {
                  "description": "Flag that specifies whether the Monetization add-on is enabled.",
                  "type": "boolean"
                }
              },
              "type": "object",
              "additionalProperties": false
            }
          },
          "type": "object",
          "additionalProperties": false
        },
        "analyticsRegion": {
          "description": "Immutable. Required. Primary GCP region for analytics data storage. For valid values, see (https://cloud.google.com/apigee/docs/api-platform/get-started/create-org).",
          "type": "string"
        },
        "authorizedNetworkRef": {
          "oneOf": [
            {
              "not": {
                "required": [
                  "external"
                ]
              },
              "required": [
                "name"
              ]
            },
            {
              "not": {
                "anyOf": [
                  {
                    "required": [
                      "name"
                    ]
                  },
                  {
                    "required": [
                      "namespace"
                    ]
                  }
                ]
              },
              "required": [
                "external"
              ]
            }
          ],
          "properties": {
            "external": {
              "description": "Compute Engine network used for Service Networking to be peered with Apigee runtime instances. See (https://cloud.google.com/vpc/docs/shared-vpc). To use a shared VPC network, use the following format: `projects/{host-project-id}/{region}/networks/{network-name}`. For example: `projects/my-sharedvpc-host/global/networks/mynetwork` **Note:** Not supported for Apigee hybrid.\n\nAllowed value: The Google Cloud resource name of a `ComputeNetwork` resource (format: `projects/{{project}}/global/networks/{{name}}`).",
              "type": "string"
            },
            "name": {
              "description": "Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names",
              "type": "string"
            },
            "namespace": {
              "description": "Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/",
              "type": "string"
            }
          },
          "type": "object",
          "additionalProperties": false
        },
        "description": {
          "description": "Description of the Apigee organization.",
          "type": "string"
        },
        "displayName": {
          "description": "Display name for the Apigee organization.",
          "type": "string"
        },
        "projectRef": {
          "description": "Immutable. The Project that this resource belongs to.",
          "oneOf": [
            {
              "not": {
                "required": [
                  "external"
                ]
              },
              "required": [
                "name"
              ]
            },
            {
              "not": {
                "anyOf": [
                  {
                    "required": [
                      "name"
                    ]
                  },
                  {
                    "required": [
                      "namespace"
                    ]
                  }
                ]
              },
              "required": [
                "external"
              ]
            }
          ],
          "properties": {
            "external": {
              "description": "Required. Name of the GCP project in which to associate the Apigee organization. Pass the information as a query parameter using the following structure in your request: projects/<project> Authorization requires the following IAM permission on the specified resource parent: apigee.organizations.create\n\nAllowed value: The Google Cloud resource name of a `Project` resource (format: `projects/{{name}}`).",
              "type": "string"
            },
            "name": {
              "description": "Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names",
              "type": "string"
            },
            "namespace": {
              "description": "Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/",
              "type": "string"
            }
          },
          "type": "object",
          "additionalProperties": false
        },
        "properties": {
          "additionalProperties": {
            "type": "string"
          },
          "description": "Properties defined in the Apigee organization profile.",
          "type": "object"
        },
        "resourceID": {
          "description": "Immutable. Optional. The service-generated name of the resource. Used for acquisition only. Leave unset to create a new resource.",
          "type": "string"
        },
        "runtimeDatabaseEncryptionKeyRef": {
          "oneOf": [
            {
              "not": {
                "required": [
                  "external"
                ]
              },
              "required": [
                "name"
              ]
            },
            {
              "not": {
                "anyOf": [
                  {
                    "required": [
                      "name"
                    ]
                  },
                  {
                    "required": [
                      "namespace"
                    ]
                  }
                ]
              },
              "required": [
                "external"
              ]
            }
          ],
          "properties": {
            "external": {
              "description": "Cloud KMS key name used for encrypting the data that is stored and replicated across runtime instances. Update is not allowed after the organization is created. Required when (#RuntimeType) is `TRIAL`, a Google-Managed encryption key will be used. For example: \"projects/foo/locations/us/keyRings/bar/cryptoKeys/baz\". **Note:** Not supported for Apigee hybrid.\n\nAllowed value: The Google Cloud resource name of a `KMSCryptoKey` resource (format: `{{selfLink}}`).",
              "type": "string"
            },
            "name": {
              "description": "Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names",
              "type": "string"
            },
            "namespace": {
              "description": "Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/",
              "type": "string"
            }
          },
          "type": "object",
          "additionalProperties": false
        },
        "runtimeType": {
          "description": "Immutable. Required. Runtime type of the Apigee organization based on the Apigee subscription purchased. Possible values: RUNTIME_TYPE_UNSPECIFIED, CLOUD, HYBRID",
          "type": "string"
        },
        "additionalProperties": false
      },
      "required": [
        "analyticsRegion",
        "projectRef",
        "runtimeType"
      ],
      "type": "object",
      "additionalProperties": false
    },
    "status": {
      "properties": {
        "billingType": {
          "description": "Output only. Billing type of the Apigee organization. See (https://cloud.google.com/apigee/pricing). Possible values: BILLING_TYPE_UNSPECIFIED, SUBSCRIPTION, EVALUATION",
          "type": "string"
        },
        "caCertificate": {
          "description": "Output only. Base64-encoded public certificate for the root CA of the Apigee organization. Valid only when (#RuntimeType) is `CLOUD`.",
          "type": "string"
        },
        "conditions": {
          "description": "Conditions represent the latest available observation of the resource's current state.",
          "items": {
            "properties": {
              "lastTransitionTime": {
                "description": "Last time the condition transitioned from one status to another.",
                "type": "string"
              },
              "message": {
                "description": "Human-readable message indicating details about last transition.",
                "type": "string"
              },
              "reason": {
                "description": "Unique, one-word, CamelCase reason for the condition's last transition.",
                "type": "string"
              },
              "status": {
                "description": "Status is the status of the condition. Can be True, False, Unknown.",
                "type": "string"
              },
              "type": {
                "description": "Type is the type of the condition.",
                "type": "string"
              }
            },
            "type": "object",
            "additionalProperties": false
          },
          "type": "array"
        },
        "createdAt": {
          "description": "Output only. Time that the Apigee organization was created in milliseconds since epoch.",
          "format": "int64",
          "type": "integer"
        },
        "environments": {
          "description": "Output only. List of environments in the Apigee organization.",
          "items": {
            "type": "string"
          },
          "type": "array"
        },
        "expiresAt": {
          "description": "Output only. Time that the Apigee organization is scheduled for deletion.",
          "format": "int64",
          "type": "integer"
        },
        "lastModifiedAt": {
          "description": "Output only. Time that the Apigee organization was last modified in milliseconds since epoch.",
          "format": "int64",
          "type": "integer"
        },
        "observedGeneration": {
          "description": "ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource.",
          "type": "integer"
        },
        "projectId": {
          "description": "Output only. Project ID associated with the Apigee organization.",
          "type": "string"
        },
        "state": {
          "description": "Output only. State of the organization. Values other than ACTIVE means the resource is not ready to use. Possible values: SNAPSHOT_STATE_UNSPECIFIED, MISSING, OK_DOCSTORE, OK_SUBMITTED, OK_EXTERNAL, DELETED",
          "type": "string"
        },
        "subscriptionType": {
          "description": "Output only. DEPRECATED: This will eventually be replaced by BillingType. Subscription type of the Apigee organization. Valid values include trial (free, limited, and for evaluation purposes only) or paid (full subscription has been purchased). See (https://cloud.google.com/apigee/pricing/). Possible values: SUBSCRIPTION_TYPE_UNSPECIFIED, PAID, TRIAL",
          "type": "string"
        }
      },
      "type": "object",
      "additionalProperties": false
    }
  },
  "required": [
    "spec"
  ],
  "type": "object"
}

Apigee Environment:

{
  "properties": {
    "apiVersion": {
      "description": "apiVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources",
      "type": "string"
    },
    "kind": {
      "description": "kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds",
      "type": "string"
    },
    "metadata": {
      "type": "object"
    },
    "spec": {
      "properties": {
        "apigeeOrganizationRef": {
          "description": "Immutable.",
          "oneOf": [
            {
              "not": {
                "required": [
                  "external"
                ]
              },
              "required": [
                "name"
              ]
            },
            {
              "not": {
                "anyOf": [
                  {
                    "required": [
                      "name"
                    ]
                  },
                  {
                    "required": [
                      "namespace"
                    ]
                  }
                ]
              },
              "required": [
                "external"
              ]
            }
          ],
          "properties": {
            "external": {
              "description": "The apigee organization for the resource\n\nAllowed value: The Google Cloud resource name of an `ApigeeOrganization` resource (format: `organizations/{{name}}`).",
              "type": "string"
            },
            "name": {
              "description": "Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names",
              "type": "string"
            },
            "namespace": {
              "description": "Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/",
              "type": "string"
            }
          },
          "type": "object",
          "additionalProperties": false
        },
        "description": {
          "description": "Optional. Description of the environment.",
          "type": "string"
        },
        "displayName": {
          "description": "Optional. Display name for this environment.",
          "type": "string"
        },
        "properties": {
          "additionalProperties": {
            "type": "string"
          },
          "description": "Optional. Key-value pairs that may be used for customizing the environment.",
          "type": "object"
        },
        "resourceID": {
          "description": "Immutable. Optional. The name of the resource. Used for creation and acquisition. When unset, the value of `metadata.name` is used as the default.",
          "type": "string"
        },
        "additionalProperties": false
      },
      "required": [
        "apigeeOrganizationRef"
      ],
      "type": "object",
      "additionalProperties": false
    },
    "status": {
      "properties": {
        "conditions": {
          "description": "Conditions represent the latest available observation of the resource's current state.",
          "items": {
            "properties": {
              "lastTransitionTime": {
                "description": "Last time the condition transitioned from one status to another.",
                "type": "string"
              },
              "message": {
                "description": "Human-readable message indicating details about last transition.",
                "type": "string"
              },
              "reason": {
                "description": "Unique, one-word, CamelCase reason for the condition's last transition.",
                "type": "string"
              },
              "status": {
                "description": "Status is the status of the condition. Can be True, False, Unknown.",
                "type": "string"
              },
              "type": {
                "description": "Type is the type of the condition.",
                "type": "string"
              }
            },
            "type": "object",
            "additionalProperties": false
          },
          "type": "array"
        },
        "createdAt": {
          "description": "Output only. Creation time of this environment as milliseconds since epoch.",
          "format": "int64",
          "type": "integer"
        },
        "lastModifiedAt": {
          "description": "Output only. Last modification time of this environment as milliseconds since epoch.",
          "format": "int64",
          "type": "integer"
        },
        "observedGeneration": {
          "description": "ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource.",
          "type": "integer"
        },
        "state": {
          "description": "Output only. State of the environment. Values other than ACTIVE means the resource is not ready to use. Possible values: STATE_UNSPECIFIED, CREATING, ACTIVE, DELETING",
          "type": "string"
        }
      },
      "type": "object",
      "additionalProperties": false
    }
  },
  "required": [
    "spec"
  ],
  "type": "object"
}
@michaelholtermann
Copy link

I'm pretty sure I run into the same issue

kubeconform -output tap -ignore-missing-schemas -strict -schema-location https://raw.githubusercontent.com/enercity/kubernetes-json-schema/master -schema-location /var/lib/crd-schemas/externalsecret_v1beta1.json k8s

validates all the usual CRDs & ExternalSecrets, but I was unable to add other CRDs.

Apparently, only 1 additional -schema-location is supported right now with v0.6.3.

@michaelholtermann
Copy link

We solved this issue by providing a folder of Schemas for the --schema-location parameter, instead of a concrete JSON:

kubeconform -output tap -ignore-missing-schemas -strict -schema-location https://raw.githubusercontent.com/enercity/kubernetes-json-schema/master -schema-location /var/lib/crd-schemas k8s

@yannh
Copy link
Owner

yannh commented Mar 3, 2024

Hi @michaelholtermann , the -schema-location is meant to be used not as a path to a particular file, but as a templated string, see documentation at https://github.com/yannh/kubeconform?tab=readme-ov-file#overriding-schemas-location

Especially: "if the -schema-location value ends with .json - Kubeconform assumes the value is a Go templated string that indicates how to search for JSON schemas."

You're not the first one to trip on the usability of this feature unfortunately 😬 Glad you found a solution!

@yannh yannh closed this as completed Mar 3, 2024
@pmialon
Copy link

pmialon commented May 15, 2024

The issue still persist.

Certainly there is an issue with the CRD but we didn't find a way to fix it.

Is there a way to find out, why the CRD that was found was rejected by kubeconform?

kubectl get crd apigeeorganizations.apigee.cnrm.cloud.google.com -o yaml > apigeeorganizations-crd.yaml

export FILENAME_FORMAT='{kind}-{group}-{version}'
openapi2jsonschema.py apigeeorganizations-crd.yaml
JSON schema written to apigeeorganization-apigee-v1beta1.json

kubeconform-0.6.6 -debug -summary -schema-location './{{ .ResourceKind }}{{ .KindSuffix }}.json' test.yaml
2024/05/15 13:38:07 using schema found at ./apigeeorganization-apigee-v1beta1.json
test.yaml - ApigeeOrganization apigeeorganization-sample failed validation: could not find schema for ApigeeOrganization
Summary: 1 resource found in 1 file - Valid: 0, Invalid: 0, Errors: 1, Skipped: 0

kubeconform-0.6.2 -debug -summary -schema-location './{{ .ResourceKind }}{{ .KindSuffix }}.json' test.yaml
2024/05/15 13:37:45 using schema found at ./apigeeorganization-apigee-v1beta1.json
Summary: 1 resource found in 1 file - Valid: 1, Invalid: 0, Errors: 0, Skipped: 0

You will find below the files to reproduce the regression we are observing.
apigeeorganizations-crd.yaml.txt
test.yaml.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants