'make test' and 'make integ-test' failing

[sumitd2]

What happened?

Hi team,

I am trying to build and test the cass-operator in a fresh Ubuntu 22.04 container, following the directions in the Readme, but make test is failing:

make test output

test -s /sumit/cass-operator/bin/controller-gen && /sumit/cass-operator/bin/controller-gen --version | grep -q v0.12.0 || \
GOBIN=/sumit/cass-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.12.0
/sumit/cass-operator/bin/controller-gen "crd:generateEmbeddedObjectMeta=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
/sumit/cass-operator/bin/controller-gen object:headerFile="hack/boilerplate.go.txt" paths="./..."
go fmt ./...
go vet ./...
test -s /sumit/cass-operator/bin/golangci-lint || curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s v1.52.2
/sumit/cass-operator/bin/golangci-lint run ./...
pkg/mocks/Client.go:26:12: _m.Called undefined (type *Client has no field or method Called) (typecheck)
        ret := _m.Called(_ca...)
                  ^
pkg/mocks/Client.go:47:12: _m.Called undefined (type *Client has no field or method Called) (typecheck)
        ret := _m.Called(_ca...)
                  ^
pkg/mocks/Client.go:68:12: _m.Called undefined (type *Client has no field or method Called) (typecheck)
        ret := _m.Called(_ca...)
                  ^
pkg/mocks/HttpClient.go:18:12: _m.Called undefined (type *HttpClient has no field or method Called) (typecheck)
        ret := _m.Called(req)
                  ^
pkg/mocks/HttpClient.go:52:26: mock.AssertExpectations undefined (type *HttpClient has no field or method AssertExpectations) (typecheck)
        t.Cleanup(func() { mock.AssertExpectations(t) })
                                ^
pkg/mocks/SubResourceClient.go:27:12: _m.Called undefined (type *SubResourceClient has no field or method Called) (typecheck)
        ret := _m.Called(_ca...)
                  ^
pkg/mocks/SubResourceClient.go:48:12: _m.Called undefined (type *SubResourceClient has no field or method Called) (typecheck)
        ret := _m.Called(_ca...)
                  ^
pkg/mocks/SubResourceClient.go:69:12: _m.Called undefined (type *SubResourceClient has no field or method Called) (typecheck)
        ret := _m.Called(_ca...)
                  ^
pkg/mocks/SubResourceClient.go:112:26: mock.AssertExpectations undefined (type *SubResourceClient has no field or method AssertExpectations) (typecheck)
        t.Cleanup(func() { mock.AssertExpectations(t) })
                                ^
pkg/mocks/helper.go:37:26: mock.AssertExpectations undefined (type *Client has no field or method AssertExpectations) (typecheck)
        t.Cleanup(func() { mock.AssertExpectations(t) })
                                ^
internal/envtest/statefulset_controller.go:38:14: r.Get undefined (type *StatefulSetReconciler has no field or method Get) (typecheck)
        if err := r.Get(ctx, req.NamespacedName, &sts); err != nil {
                    ^
tests/util/ginkgo/lib.go:221:2: undefined: Expect (typecheck)
        Expect(err).ToNot(HaveOccurred())
        ^
tests/util/ginkgo/lib.go:231:2: undefined: Expect (typecheck)
        Expect(err).ToNot(HaveOccurred())
        ^
tests/util/ginkgo/lib.go:245:2: undefined: Expect (typecheck)
        Expect(err).ToNot(HaveOccurred())
        ^
apis/cassandra/v1beta1/webhook_suite_test.go:53:9: undefined: BeforeSuite (typecheck)
var _ = BeforeSuite(func() {
        ^
apis/cassandra/v1beta1/webhook_suite_test.go:119:9: undefined: AfterSuite (typecheck)
var _ = AfterSuite(func() {
        ^
apis/cassandra/v1beta1/cassandradatacenter_types.go:955:12: dc.Name undefined (type *CassandraDatacenter has no field or method Name) (typecheck)
        return dc.Name
                  ^
apis/cassandra/v1beta1/webhook_suite_test.go:49:2: undefined: RegisterFailHandler (typecheck)
        RegisterFailHandler(Fail)
        ^
apis/cassandra/v1beta1/webhook_suite_test.go:50:2: undefined: RunSpecs (typecheck)
        RunSpecs(t, "Webhook Suite")
        ^
apis/cassandra/v1beta1/webhook_suite_test.go:54:37: undefined: GinkgoWriter (typecheck)
        logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true)))
                                           ^
apis/cassandra/v1beta1/webhook_suite_test.go:58:2: undefined: By (typecheck)
        By("bootstrapping test environment")
        ^
apis/cassandra/v1beta1/webhook_suite_test.go:108:2: undefined: Eventually (typecheck)
        Eventually(func() error {
        ^
apis/cassandra/v1beta1/webhook_suite_test.go:115:12: undefined: Succeed (typecheck)
        }).Should(Succeed())
                  ^
apis/cassandra/v1beta1/webhook_suite_test.go:121:2: undefined: By (typecheck)
        By("tearing down the test environment")
        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:67:9: undefined: Describe (typecheck)
var _ = Describe("CassandraDatacenter tests", func() {
        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:221:29: undefined: Gomega (typecheck)
func verifyOwnerReference(g Gomega, ownerRef metav1.OwnerReference, dcName string) {
                            ^
internal/controllers/cassandra/suite_test.go:58:9: undefined: BeforeSuite (typecheck)
var _ = BeforeSuite(func() {
        ^
internal/controllers/cassandra/suite_test.go:141:9: undefined: AfterSuite (typecheck)
var _ = AfterSuite(func() {
        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:54:2: undefined: Eventually (typecheck)
        Eventually(func(g Gomega) {
        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:60:95: undefined: Succeed (typecheck)
        }).WithTimeout(20 * time.Second).WithPolling(200 * time.Millisecond).WithContext(ctx).Should(Succeed())
                                                                                                     ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:58:45: undefined: Succeed (typecheck)
                g.Expect(k8sClient.Get(ctx, key, &dc)).To(Succeed())
                                                          ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:59:52: undefined: Equal (typecheck)
                g.Expect(dc.Status.CassandraOperatorProgress).To(Equal(state))
                                                                 ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:68:2: undefined: Describe (typecheck)
        Describe("Creating a new datacenter", func() {
        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:69:3: undefined: Context (typecheck)
                Context("Single datacenter", func() {
                ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:70:4: undefined: BeforeEach (typecheck)
                        BeforeEach(func() {
                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:80:4: undefined: AfterEach (typecheck)
                        AfterEach(func() {
                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:88:4: undefined: When (typecheck)
                        When("There is a single rack and a single node", func() {
                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:89:5: undefined: It (typecheck)
                                It("should end up in a Ready state", func(ctx SpecContext) {
                                ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:101:5: undefined: It (typecheck)
                                It("should be able to scale up", func(ctx SpecContext) {
                                ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:113:6: undefined: By (typecheck)
                                        By("Updating the size to 3")
                                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:117:6: undefined: Eventually (typecheck)
                                        Eventually(func(g Gomega) {
                                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:128:4: undefined: When (typecheck)
                        When("There are multiple nodes in a single rack", func() {
                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:129:5: undefined: It (typecheck)
                                It("should end up in a Ready state", func(ctx SpecContext) {
                                ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:143:4: undefined: When (typecheck)
                        When("There are multiple nodes in multiple racks", func() {
                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:165:34: undefined: Equal (typecheck)
                g.Expect(len(stsAll.Items)).To(Equal(rackCount))
                                               ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:172:36: undefined: Equal (typecheck)
                        g.Expect(len(podList.Items)).To(Equal(podsPerSts))
                                                        ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:193:39: undefined: BeTrue (typecheck)
                g.Expect(errors.IsNotFound(err)).To(BeTrue())
                                                    ^
internal/controllers/cassandra/cassandradatacenter_controller_test.go:215:47: undefined: BeNil (typecheck)
                        g.Expect(pvc.GetDeletionTimestamp()).ToNot(BeNil())
                                                                   ^
internal/controllers/cassandra/suite_test.go:54:2: undefined: RegisterFailHandler (typecheck)
        RegisterFailHandler(Fail)
        ^
internal/controllers/cassandra/suite_test.go:55:2: undefined: RunSpecs (typecheck)
        RunSpecs(t, "Controller Suite")
        ^
make: *** [Makefile:129: lint] Error 1

What did you expect to happen?

The tests should pass.

How can we reproduce it (as minimally and precisely as possible)?

Start a fresh Ubuntu 22.04 container:

1. Install Golang 1.21.5
2. Install docker 25.0.3
3. Install kind 0.17.0
4. Install kubectl 1.29.2
5. Install kustomize 5.3.0
6. Insyall yq 4.40.7
7. Clone source 1.18.2
8. make build
9. make test
10. make docker-build docker-kind
11. kubectl apply -f operator/k8s-flavors/kind/rancher-local-path-storage.yaml
12. make deploy
13. make integ-test

cass-operator version

1.18.2

Kubernetes version

1.29.2

Method of installation

No response

Anything else we need to know?

make integ-test is also failing if i skip the make test

[burmanm]

I think I fixed this in commit f309eb4 , the problem you're hitting is too old version of golangci-lint.

[sumitd2]

@burmanm Thanks, golangci-lint version does fix make test.
Now I am trying to run the integration tests, but there is an issue with make deploy:

test -s /sumit/cass-operator/bin/controller-gen && /sumit/cass-operator/bin/controller-gen --version | grep -q v0.12.0 || \
GOBIN=/sumit/cass-operator/bin go install sigs.k8s.io/controller-tools/cmd/controller-gen@v0.12.0
/sumit/cass-operator/bin/controller-gen "crd:generateEmbeddedObjectMeta=true" rbac:roleName=manager-role webhook paths="./..." output:crd:artifacts:config=config/crd/bases
test -s /sumit/cass-operator/bin/kustomize || { curl -s "https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh" | bash -s -- 5.0.3 /sumit/cass-operator/bin; }
cd config/manager && /sumit/cass-operator/bin/kustomize edit set image controller=k8ssandra/cass-operator:v1.18.2-dev.f1f25eb-20240216
LOG_IMG=k8ssandra/system-logger:v1.18.2-dev.f1f25eb-20240216 yq eval -i '.images.system-logger = env(LOG_IMG)' config/manager/image_config.yaml
kubectl apply --force-conflicts --server-side -k config/deployments/default
# Warning: 'bases' is deprecated. Please use 'resources' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesJson6902' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'patchesStrategicMerge' is deprecated. Please use 'patches' instead. Run 'kustomize edit fix' to update your Kustomization automatically.
# Warning: 'vars' is deprecated. Please use 'replacements' instead. [EXPERIMENTAL] Run 'kustomize edit fix' to update your Kustomization automatically.
namespace/cass-operator serverside-applied
customresourcedefinition.apiextensions.k8s.io/cassandradatacenters.cassandra.datastax.com serverside-applied
customresourcedefinition.apiextensions.k8s.io/cassandratasks.control.k8ssandra.io serverside-applied
serviceaccount/cass-operator-controller-manager serverside-applied
role.rbac.authorization.k8s.io/cass-operator-leader-election-role serverside-applied
role.rbac.authorization.k8s.io/cass-operator-manager-role serverside-applied
clusterrole.rbac.authorization.k8s.io/cass-operator-manager-role serverside-applied
rolebinding.rbac.authorization.k8s.io/cass-operator-leader-election-rolebinding serverside-applied
rolebinding.rbac.authorization.k8s.io/cass-operator-manager-rolebinding serverside-applied
clusterrolebinding.rbac.authorization.k8s.io/cass-operator-manager-res-rolebinding serverside-applied
configmap/cass-operator-manager-config serverside-applied
service/cass-operator-webhook-service serverside-applied
deployment.apps/cass-operator-controller-manager serverside-applied
validatingwebhookconfiguration.admissionregistration.k8s.io/cass-operator-validating-webhook-configuration serverside-applied
Error from server (InternalError): Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": dial tcp 10.96.158.2:443: connect: connection refused
Error from server (InternalError): Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/mutate?timeout=10s": dial tcp 10.96.158.2:443: connect: connection refused
make: *** [Makefile:203: deploy] Error 1

[burmanm]

Your cert-manager isn't running.

[sumitd2]

@burmanm Thanks for the reply. The cert manager was running, but the kind cluster was started with kind_config_6_workers.yaml. When I changed it to kind_config_3_workers.yaml it started working. The kind pods were crashing, indicating there weren't enough resources on the VM.

[burmanm]

Closing stale tickets, reopen if there's anything else.