Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run integration tests with ginkgo instead of go test and ensure logs go to GinkgoWriter instead of stdout #1726

Merged
merged 2 commits into from
Mar 22, 2024
Merged

Run integration tests with ginkgo instead of go test and ensure logs go to GinkgoWriter instead of stdout #1726

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5f45f3c
OSSM-6158 Run integration tests with ginkgo instead of go test
luksa Mar 22, 2024
dc4b3fb
OSSM-6159 Ensure integration tests log to GinkgoWriter instead of stdout
luksa Mar 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile.core.mk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -152,7 +152,7 @@ test.unit: ## Run unit tests.
.PHONY: test.integration
test.integration: envtest ## Run integration tests located in the tests/integration directory.
KUBEBUILDER_ASSETS="$(shell $(ENVTEST) use $(ENVTEST_K8S_VERSION) --bin-dir $(LOCALBIN) -p path)" \
go test --tags=integration $(VERBOSE_FLAG) ./tests/integration/...
go run github.com/onsi/ginkgo/v2/ginkgo --tags=integration $(VERBOSE_FLAG) ./tests/integration/...
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just tried this locally and the output is too much verbose. :(

also, does this run ginkgo from master? This might break us at some point? Should we stick to a released version instead?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, too much output due to a bug: https://issues.redhat.com/browse/OSSM-6159

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As for running ginkgo from master. We use the same approach in the e2e tests. So if this need to be fixed, we should do it in both places (in a follow up PR).

Copy link
Contributor Author

@luksa luksa Mar 22, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, it looks like go run honors whatever is in go.mod, so we're actually running v2.15.0:

[luksa@p1 sail-operator]$ go run github.com/onsi/ginkgo/v2/ginkgo version
Ginkgo Version 2.15.0

[luksa@p1 sail-operator]$ cd ..

[luksa@p1 maistra]$ go run github.com/onsi/ginkgo/v2/ginkgo version
no required module provides package github.com/onsi/ginkgo/v2/ginkgo: go.mod file not found in current directory or any parent directory; see 'go help modules'

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jwendell I've just pushed another commit that fixes the logs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree, but as you say: "it is what it is" :)

Ginkgo has "specs" not tests. The list of specs gets built from a hierarchy of test "nodes". These specs are then executed (in a single test function that you must write). Unlike functions that are organized into packages, the specs are just one giant sequential list. And they are all executed by your test function.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Is there a way to show the progress a little more verbose, instead of just showing small dots?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. There's only -v, but this displays everything that is logged by the test. I don't see any other option that would e.g. only print the specs with nothing else.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok. It is what it is, should we revert to using plain old go tests then? /me runs :D

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We wouldn't be reverting. We had ginkgo tests from the very start. :)


.PHONY: test.scorecard
test.scorecard: operator-sdk ## Run the operator scorecard test.
Expand Down
5 changes: 3 additions & 2 deletions pkg/test/environment.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
package test

import (
"io"
"path"

"github.com/istio-ecosystem/sail-operator/pkg/common"
Expand All @@ -26,8 +27,8 @@ import (
"sigs.k8s.io/controller-runtime/pkg/log/zap"
)

func SetupEnv() (*envtest.Environment, client.Client, *rest.Config) {
logf.SetLogger(zap.New(zap.UseDevMode(true)))
func SetupEnv(logWriter io.Writer) (*envtest.Environment, client.Client, *rest.Config) {
logf.SetLogger(zap.New(zap.WriteTo(logWriter), zap.UseDevMode(true)))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funny story, in Brazil, "zap" is a short for "Whatsapp", the messenger app.


testEnv := &envtest.Environment{
CRDDirectoryPaths: []string{path.Join(common.RepositoryRoot, "chart", "crds")},
Expand Down
5 changes: 1 addition & 4 deletions tests/integration/api/suite_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ import (
ctrl "sigs.k8s.io/controller-runtime"
"sigs.k8s.io/controller-runtime/pkg/client"
"sigs.k8s.io/controller-runtime/pkg/envtest"
logf "sigs.k8s.io/controller-runtime/pkg/log"
"sigs.k8s.io/controller-runtime/pkg/log/zap"
)

// These tests use Ginkgo (BDD-style Go testing framework). Refer to
Expand All @@ -58,8 +56,7 @@ func TestAPIs(t *testing.T) {
}

var _ = BeforeSuite(func() {
testEnv, k8sClient, cfg = test.SetupEnv()
logf.SetLogger(zap.New(zap.WriteTo(GinkgoWriter), zap.UseDevMode(true)))
testEnv, k8sClient, cfg = test.SetupEnv(GinkgoWriter)

mgr, err := ctrl.NewManager(cfg, ctrl.Options{
Scheme: scheme.Scheme,
Expand Down