-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
docs: Advise running ginkgo in verbose for e2e tests #15060
docs: Advise running ginkgo in verbose for e2e tests #15060
Conversation
We should always recommend to run end-to-end tests with ginkgo in verbose mode. Otherwise the provisioning may get stuck downloading some image without the contributor noticing. Signed-off-by: Paul Chaignon <paul@cilium.io>
@@ -64,15 +64,15 @@ To run all of the runtime tests, execute the following command from the ``test`` | |||
|
|||
:: | |||
|
|||
ginkgo --focus="Runtime" -noColor | |||
ginkgo -v --focus="Runtime" -noColor |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also wondering why we put -noColor
everywhere. Such a shame to run in a black and white world...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also wondering
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find -test.v
useful also, did you consider adding it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't know about -test.v
. What's the difference?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's equivalent of go test -v
, so if you have anything that uses test logger etc, it will be included. I don't know what it'd mean for Cilium tests, but I just use it to be sure when I run any gingko suite.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If it makes any difference for Cilium tests, then yes, we should probably add it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not so sure. Let's merge this and move along!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good.
Should we make it verbose by default in the future, with an option to reduce log instead?
@qmonnet It's a ginkgo option, so I'm not sure we can. If we can, then yes we should, absolutely. |
Maybe doable by accessing |
Apparently I can get it verbose with the following patch: Default to verbosediff --git a/test/config/config.go b/test/config/config.go
index 17c7964aea76..1f05067b33ef 100644
--- a/test/config/config.go
+++ b/test/config/config.go
@@ -55,6 +55,7 @@ type CiliumTestConfigType struct {
// node. If false, some tests will silently skip multinode checks.
Multinode bool
RunQuarantined bool
+ NoVerbose bool
Help bool
}
@@ -107,6 +108,8 @@ func (c *CiliumTestConfigType) ParseFlags() {
"Enable tests across multiple nodes. If disabled, such tests may silently pass")
flagset.BoolVar(&c.RunQuarantined, "cilium.runQuarantined", false,
"Run tests that are under quarantine.")
+ flagset.BoolVar(&c.NoVerbose, "cilium.noVerbose", false,
+ "Disable verbose mode, otherwise enforced by default.")
flagset.BoolVar(&c.Help, "cilium.help", false, "Display this help message.")
args := make([]string, 0, len(os.Args))
diff --git a/test/ginkgo-ext/scopes.go b/test/ginkgo-ext/scopes.go
index 2253f764f17e..582a5b8bd770 100644
--- a/test/ginkgo-ext/scopes.go
+++ b/test/ginkgo-ext/scopes.go
@@ -120,6 +120,10 @@ func init() {
config.Flags(commandFlags, "ginkgo", true)
commandFlags.Parse(args)
ciliumTestConfig.CiliumTestConfig.ParseFlags()
+
+ if !ciliumTestConfig.CiliumTestConfig.NoVerbose {
+ config.DefaultReporterConfig.Verbose = true
+ }
}
func (s *scope) isUnset() bool { I'll open a PR with this change so we can discuss it further. [Edit: #15184] |
We should always recommend to run end-to-end tests with ginkgo in verbose mode. Otherwise the provisioning may get stuck downloading some image without the contributor noticing.