envtest: no matches for kind "Infrastructure" in version "config.openshift.io/v1"

[gee4vee]

Dependency versions:
sigs.k8s.io/controller-runtime v0.6.3
github.com/openshift/api v0.0.0-20200326152221-912866ddb162

I'm getting the error above in my test even though I've added the relevant schema like so:

...
	configv1 "github.com/openshift/api/config/v1"
...
var _ = BeforeSuite(func(done Done) {
...
	err = configv1.AddToScheme(scheme.Scheme)
	Expect(err).NotTo(HaveOccurred())
...

I have some other schemas I've added including the one for our CRD, and those don't give me problems. The controller does start and begins reconciling our CRD. I also checked the version of the OpenShift API that I'm using to make sure it has Infrastructure and it does: https://pkg.go.dev/github.com/openshift/api@v0.0.0-20200623075207-eb651a5bb0ad/config/v1

Am I missing something else? Full code is here: https://github.com/IBM/starter-kit-operator/blob/operator-sdk-upgrade/controllers/suite_test.go

[ktravis]

I was also facing this issue without much indication what the problem was - I think I've just realized the problem, maybe someone can correct me if I have it wrong:

You are registering the object types with your scheme.Scheme, but there is nothing telling the (test) apiserver about the CRD itself. kubebuilder bootstraps a snippet that loads the CRDs you are defining and testing directly:

https://github.com/IBM/starter-kit-operator/blob/adffbc16c42d9d899432c67cd9c1e63a27729104/controllers/suite_test.go#L92-L94

But there is nothing to define the rest of those types with the apiserver (registering Go types with your scheme does not install the associated CRD, it's only telling your local client how to handle that type of object).

For this to work the way you are expecting I think you would need to include the CRDs and add them to that list - otherwise the test apiserver will always return an error when you send a request for a type it hasn't heard of.

Hope that helps.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[hfgbarrigas]

Any updates on this? Getting the same error. Using go mod vendor won't help because non go files are pruned.

Apart from crd paths, the environment has an attibute CRDs that looks promising but having issues using it since I can't find any documentation around it . Maybe @camilamacedo86 can shed some light?

[hfgbarrigas]

For those running into this, here's how to properly add the external crd manifests to envtest:

CRDDirectoryPaths: []string{
			filepath.Join("..", "config", "crd", "bases"),
			filepath.Join(build.Default.GOPATH, "pkg", "mod", "github.com", "owner", "theirs@version", "config", "crd", "bases"),
		},

[camilamacedo86]

Hi @hfgbarrigas,

Just to confirm. We can close this one since the error here was solved. by adding the correct path for the CRDs. Am I right?

[hfgbarrigas]

Hi @camilamacedo86,

I'd be inclined to close it. Though some documentation on how to do it and why using go modules would nice for future reference.

[estroz]

This would be a nice FAQ entry.

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-contributor-experience at kubernetes/community.
/lifecycle rotten

[fejta-bot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

[k8s-ci-robot]

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-contributor-experience at kubernetes/community.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[avrahams]

For those running into this, here's how to properly add the external crd manifests to envtest:

CRDDirectoryPaths: []string{
			filepath.Join("..", "config", "crd", "bases"),
			filepath.Join(build.Default.GOPATH, "pkg", "mod", "github.com", "owner", "theirs@version", "config", "crd", "bases"),
		},

@camilamacedo86
Will this resolve my issue in the sdk ?
operator-framework/operator-sdk#4434

Where can I find OpenShift Route CRD?

[mpreu]

@avrahams
There is none for Route. All OCP related CRDs are potentially to be found in the API repo, but some of the "older" OCP resources are not published this way. Recently was in contact with some Red Hat people and my information so far is, that there is no really immediate plan to change that.

There is a manual way though. One has to create a 'fake' CRD within the correct OCP API repository path and adapt the make targets.

E.g. for Routes, a CRD file route_crd.yaml with the following content has to be created in the path https://github.com/openshift/api/tree/master/route/v1:

apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  # name must be in the form: <plural>.<group>
  # <group> has to match the correspondig .spec field
  name: routes.route.openshift.io
spec:
  # group name to use for REST API: /apis/<group>/<version>
  group: route.openshift.io
  versions:
    - name: v1
      served: true
      storage: true
  scope: Namespaced
  subresources:
    # enable spec/status
    status: {}
  names:
    plural: routes
    singular: route
    kind: Route

Then the Makefile has to get this line:

$(call add-crd-gen,route,./route/v1,./route/v1,./route/v1)

Updating the CRDs with

make update-codegen-crds

populates the CRD stub with the correct OpenAPI validation content.

Unfortunately it seems to only works this way, as the API repo infrastructure already expects the basic CRD to be there (I am sure one can work around that somehow).

It works for other resources as well, but it is obviously not a very practical solution. So this is just a documentation comment for people experiencing similar problems with OCP related resources.

Just as a warning: In some of my tests I experienced validation errors populating envtest with these CRDs, but I did not investigate it further.

[avrahams]

Thank you @mpreu this worked