Need references/documentation for unit testing

[kaushiksrinivas]

Hi

Looking for documentation/examples on unit testing the operator sdk project.
links referred in some of the bugs seems to be no more available
https://github.com/operator-framework/operator-sdk/blob/master/doc/user/unit-testing.md

Can anyone provide documentation link for unit testing and any example implementations ?
Thanks in advance
Kaushik.

[estroz]

We recommend following this kubebuilder testing guide, which we should link in documentation.

/kind documentation
/milestone v1.1.0

[camilamacedo86]

Hi @kaushiksrinivas,

However, the same doc that you are looking for is in : https://sdk.operatorframework.io/docs/golang/legacy/unit-testing/. Note that SDK is in a process to be integrated with KB which means that its project layouts will be aligned. More info : Integrating Kubebuilder and Operator SDK.

So, for the new layout test files for your controller are scaffold and then, you can follow up the docs suggested by @estroz as well.

[kaushiksrinivas]

Hi @camilamacedo86
"Note that SDK is in a process to be integrated with KB which means that its project layouts will be aligned."
Does this mean operator-sdk would get changed to align with kubebuilder ?

Currently if anyone is already using operator-sdk, what changes are expected ?
Thanks,
Kaushik

[camilamacedo86]

Hi @kaushiksrinivas,

Does this mean operator-sdk would get changed to align with kubebuilder?

Yes, for 1.0.0 the legacy layout will no longer be supported.

Currently if anyone is already using operator-sdk, what changes are expected?

You can check the migration guide to know how to upgrade your projects. See the PR #3543 for we shape it better. Note that in the first comment has a link where you can preview the doc.

See that Can anyone provide documentation link for unit testing and any example implementations ? which shows answered and we already started to discuss another topic. Ideally, we should have one question/bug/feature per issue. So, let's it open for we track the need to add the unit test doc. However, feel free to raise another issue as well

[britoaldan]

hi @camilamacedo86, @estroz @joelanford
The above comments are a bit confusing,
Need clarifications for the below concerns,

Operator SDK includes an e2e testing framework that simplifies testing your operator against an actual cluster.
Kubebuilder includes an envtest package that allows operator developers to run simple tests with a standalone etcd and apiserver.
Reference: #1758

Does this mean that the e2e test framework of operator-sdk will be removed and no longer supported and one needs to follow the kube-builder way of testing ?

Does this mean https://sdk.operatorframework.io/docs/golang/legacy/unit-testing/. this is no longer valid with latest version(v0.19) of operator-sdk ?

[camilamacedo86]

Hi @britoaldan,

I flagged it to be discussed in the Bug Triage meeting. Please, feel free to attend.
We will update it after the meeting as well.

[kaushiksrinivas]

Hi @camilamacedo86
Can you please let us know, how do we attend these meetings ?

[camilamacedo86]

Hi @kaushiksrinivas, @beautytiger,

See the details over the meetings in: https://github.com/operator-framework/community

As an update from the discussion made see:

• For 1.0.0 we have the Breaking change: Removed test subcommand and the test framework pkg/test. (#3409)
• Also, we would like to recommend use controller-runtime's envtest framework instead of it.
• Note that in the new layout the projects are scaffolded tests using the envtest
• And then, it is important to highlight that the test-framework and the old unit doc uses the client-fake which is not actually really testing the changes. It is a subject that has been discussed a lot in the Kubernetes community and indeed it was flagged in the past to be deprecated but now was decided that it will not be removed from controller-runtime but then will be required clarifies its limitations. More info: Reconsider deprecation of pkg/client/fake kubernetes-sigs/controller-runtime#768. It is one of the reasons for the envtest has been used in the Kubernetes projects per default as been recommended to be used.

We will keep this issue to track the need to provide the docs for the tests.

[britoaldan]

Hi @camilamacedo86

If one builds operator with v0.19 with the new scaffolding structure.
when we upgrade to 1.0.0 will it be a smooth upgrade, if not what is expected to break or to be migrated again.
could you please provide some pointers.

Also we see envtest is for integration testing, will there be a framework available for unit test ?

[camilamacedo86]

Hi @britoaldan,

Following the comments.

If one builds operator with v0.19 with the new scaffolding structure.

You can check the migration guide for GO already. However, it requires some fixes and updates. Because of this, I'd suggest you check and follow it via the changes made in the PR #3543. See that there you will find the preview link for the doc https://deploy-preview-3543--operator-sdk.netlify.app/docs/building-operators/golang/project_migration_guide/.

when we upgrade to 1.0.0 will it be a smooth upgrade, if not what is expected to break or to be migrated again.

See that SDK so far is in a version < 1.0.0 which means by convention initial development phase which allows breaking changes. However, the goal is to release 1.0.0 and then you will be less impact by breaking changes. The next release with breaking changes would need to be bumped to 2.0.0 following the semver

Also we see envtest is for integration testing, will there be a framework available for the unit test?

Now, the SDK will generate tests for your projects. These tests are using ginkgo/ and gomega, look its imports:

	. "github.com/onsi/ginkgo"
	. "github.com/onsi/gomega"

I'd recommend you follow up and use that since this stack is adopted across the k8ss projects used by it and SDK now. To know more about how to test check the doc : https://book.kubebuilder.io/reference/writing-tests.html?highlight=controller,test#writing-controller-tests

[kaushiksrinivas]

Hi @camilamacedo86
Any update on this since v1.0.0 is already out ?

Is there any unit test framework which can be used or one must follow the link
https://book.kubebuilder.io/reference/writing-tests.html?highlight=controller,test#writing-controller-tests
for writing tests for operators in 1.0 sdk version.

Thanks,
Kaushik.

[camilamacedo86]

HI @kaushiksrinivas,

The documentation in upstream was updated with an example. Also, see that a PR was raised to share the info which was already provided here via the docs as well #3823. Also, we are clarifying on it that scorecard and kuttl also can be used to test the project.