test: Fix artifact collection for bad log failures #16489
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When a test fails because a bad log message is found [1], the per-test artifacts are not collected and exposed in the Jenkins UI. This issue is visible at [2] at the time of writing this. We only collect per-test artifacts for failing tests, which we detect with ginkgo.CurrentGinkgoTestDescription().Failed. Unfortunately, this variable is not updated when the test fails in a JustAfterEach block. That's the case for the bad log messages, which are almost always checked from a JustAfterEach block. This commit fixes that issue by using the dedicated function TestFailed instead of ginkgo.CurrentGinkgoTestDescription().Failed. At the time of writing this, the result can be seen at [3]. This issue was discovered before in da25f94 ("k8sT: Clean up services in JustAfterEach"), but the fix was not extended to the existing code at the time. 1 - https://github.com/cilium/cilium/blob/v1.10.0/test/helpers/cons.go#L291 2 - https://jenkins.cilium.io/job/Cilium-PR-Runtime-4.9/4917/testReport/junit/(root)/Suite-runtime/RuntimeFQDNPolicies_toFQDNs_populates_toCIDRSet__data_from_proxy__L3_dependent_L7_HTTP_with_toFQDN_updates_proxy_policy/ 3 - https://jenkins.cilium.io/job/Cilium-PR-K8s-1.20-kernel-4.19/637/testReport/junit/Suite-k8s-1/20/K8sServicesTest_Checks_service_across_nodes_Supports_IPv4_fragments/ Signed-off-by: Paul Chaignon <paul@cilium.io>
pchaigno
added
kind/bug/CI
maintainer-s-little-helper
bot
added
dont-merge/needs-release-note-label
maintainer-s-little-helper
bot
removed
the
dont-merge/needs-release-note-label
nbusseneau
approved these changes
Jun 9, 2021
|
I've already validated at https://jenkins.cilium.io/job/Cilium-PR-K8s-1.20-kernel-4.19/637/testReport/junit/Suite-k8s-1/20/K8sServicesTest_Checks_service_across_nodes_Supports_IPv4_fragments/ that this fix works, so no need to run the full test suite. Marking as ready to merge. |
pchaigno
added
the
ready-to-merge
|
Marking for backports as it can significantly help flake debugging. |
joestringer
moved this from Needs backport from master
to Backport done to v1.9
in 1.9.10
This was referenced Sep 1, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When a test fails because a bad log message is found, the per-test artifacts are not collected and exposed in the Jenkins UI.
We only collect per-test artifacts for failing tests, which we detect with
ginkgo.CurrentGinkgoTestDescription().Failed. Unfortunately, this variable is not updated when the test fails in aJustAfterEachblock. That's the case for the bad log messages, which are almost always checked from aJustAfterEachblock.This pull request fixes that issue by using the dedicated function
TestFailed()instead ofginkgo.CurrentGinkgoTestDescription().Failed.This issue was discovered by @joestringer before in da25f94 ("k8sT: Clean up services in JustAfterEach"), but the fix was not extended to the existing code at the time.
Before:
After:
