CI: K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath #25255

maintainer-s-little-helper · 2023-05-04T03:37:43Z

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc001599020>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-6l2gc echo-bc5cf6ffb-n8zv6]",
        err: <*errors.errorString | 0xc000e678c0>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-6l2gc echo-bc5cf6ffb-n8zv6]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

11:20:44 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
11:20:44 STEP: Ensuring the namespace kube-system exists
11:20:44 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
11:20:44 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
11:20:44 STEP: Deleting pods [echo-bc5cf6ffb-6l2gc,echo-bc5cf6ffb-n8zv6] in namespace default
11:20:44 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-6l2gc,echo-bc5cf6ffb-n8zv6)
11:20:44 STEP: Deleting cep [echo-bc5cf6ffb-6l2gc,echo-bc5cf6ffb-n8zv6] in namespace default
11:20:44 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-6l2gc,echo-bc5cf6ffb-n8zv6)
11:20:54 STEP: Unable to delete pods echo-bc5cf6ffb-6l2gc with 'kubectl -n default delete pods echo-bc5cf6ffb-6l2gc': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-6l2gc" deleted
	 
Stderr:
 	 

11:20:54 STEP: Unable to delete pods echo-bc5cf6ffb-n8zv6 with 'kubectl -n default delete pods echo-bc5cf6ffb-n8zv6': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-n8zv6" deleted
	 
Stderr:
 	 

11:20:55 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
11:21:08 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
11:21:22 STEP: Cleaning bpf_lxc build files
11:21:22 STEP: Building bpf_lxc object file
11:21:24 STEP: Running the verifier test script with bpf_lxc
11:21:25 STEP: Cleaning bpf_lxc build files
11:21:25 STEP: Building bpf_lxc object file
11:21:27 STEP: Running the verifier test script with bpf_lxc
11:21:27 STEP: Cleaning bpf_lxc build files
11:21:27 STEP: Building bpf_lxc object file
11:21:29 STEP: Running the verifier test script with bpf_lxc
11:21:29 STEP: Cleaning bpf_host build files
11:21:29 STEP: Building bpf_host object file
11:21:33 STEP: Running the verifier test script with bpf_host
11:21:35 STEP: Cleaning bpf_host build files
11:21:36 STEP: Building bpf_host object file
11:21:38 STEP: Running the verifier test script with bpf_host
11:21:39 STEP: Cleaning bpf_host build files
11:21:39 STEP: Building bpf_host object file
11:21:41 STEP: Running the verifier test script with bpf_host
11:21:42 STEP: Cleaning bpf_xdp build files
11:21:42 STEP: Building bpf_xdp object file
11:21:45 STEP: Running the verifier test script with bpf_xdp
11:21:45 STEP: Cleaning bpf_overlay build files
11:21:46 STEP: Building bpf_overlay object file
11:21:48 STEP: Running the verifier test script with bpf_overlay
11:21:49 STEP: Cleaning bpf_sock build files
11:21:49 STEP: Building bpf_sock object file
11:21:49 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-03T11:21:50Z====
11:21:50 STEP: Running AfterEach for block EntireTestsuite

11:21:50 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc001599020>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-6l2gc echo-bc5cf6ffb-n8zv6]",
        err: <*errors.errorString | 0xc000e678c0>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-6l2gc echo-bc5cf6ffb-n8zv6]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//31/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_31_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/31/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

The text was updated successfully, but these errors were encountered:

maintainer-s-little-helper · 2023-05-05T03:33:38Z

PR #25125 hit this flake with 89.37% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0000bee40>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-dbdpv echo-67574b6655-mxqjg]",
        err: <*errors.errorString | 0xc0017ec110>{
            s: "Pods are still terminating: [echo-67574b6655-dbdpv echo-67574b6655-mxqjg]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

02:44:17 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
02:44:17 STEP: Ensuring the namespace kube-system exists
02:44:17 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
02:44:17 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
02:44:17 STEP: Deleting pods [echo-67574b6655-dbdpv,echo-67574b6655-mxqjg] in namespace default
02:44:17 STEP: Waiting for 2 deletes to return (echo-67574b6655-dbdpv,echo-67574b6655-mxqjg)
02:44:17 STEP: Deleting cep [echo-67574b6655-dbdpv,echo-67574b6655-mxqjg] in namespace default
02:44:17 STEP: Waiting for 2 deletes to return (echo-67574b6655-dbdpv,echo-67574b6655-mxqjg)
02:44:27 STEP: Unable to delete pods echo-67574b6655-mxqjg with 'kubectl -n default delete pods echo-67574b6655-mxqjg': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-mxqjg" deleted
	 
Stderr:
 	 

02:44:27 STEP: Unable to delete pods echo-67574b6655-dbdpv with 'kubectl -n default delete pods echo-67574b6655-dbdpv': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-dbdpv" deleted
	 
Stderr:
 	 

02:44:27 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
02:44:37 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
02:44:52 STEP: Cleaning bpf_lxc build files
02:44:52 STEP: Building bpf_lxc object file
02:44:55 STEP: Running the verifier test script with bpf_lxc
02:44:55 STEP: Cleaning bpf_lxc build files
02:44:55 STEP: Building bpf_lxc object file
02:44:57 STEP: Running the verifier test script with bpf_lxc
02:44:57 STEP: Cleaning bpf_lxc build files
02:44:57 STEP: Building bpf_lxc object file
02:44:59 STEP: Running the verifier test script with bpf_lxc
02:44:59 STEP: Cleaning bpf_host build files
02:44:59 STEP: Building bpf_host object file
02:45:03 STEP: Running the verifier test script with bpf_host
02:45:05 STEP: Cleaning bpf_host build files
02:45:06 STEP: Building bpf_host object file
02:45:08 STEP: Running the verifier test script with bpf_host
02:45:09 STEP: Cleaning bpf_host build files
02:45:09 STEP: Building bpf_host object file
02:45:12 STEP: Running the verifier test script with bpf_host
02:45:12 STEP: Cleaning bpf_xdp build files
02:45:13 STEP: Building bpf_xdp object file
02:45:15 STEP: Running the verifier test script with bpf_xdp
02:45:16 STEP: Cleaning bpf_overlay build files
02:45:16 STEP: Building bpf_overlay object file
02:45:19 STEP: Running the verifier test script with bpf_overlay
02:45:19 STEP: Cleaning bpf_sock build files
02:45:19 STEP: Building bpf_sock object file
02:45:20 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-05T02:45:20Z====
02:45:20 STEP: Running AfterEach for block EntireTestsuite

02:45:20 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0000bee40>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-dbdpv echo-67574b6655-mxqjg]",
        err: <*errors.errorString | 0xc0017ec110>{
            s: "Pods are still terminating: [echo-67574b6655-dbdpv echo-67574b6655-mxqjg]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//1988/artifact/531fa95d_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//1988/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_1988_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/1988/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

Due to this flake: #25255 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

maintainer-s-little-helper · 2023-05-10T05:13:00Z

PR #25346 hit this flake with 88.22% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000808540>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-22hcp echo-67574b6655-fxl2p]",
        err: <*errors.errorString | 0xc000033720>{
            s: "Pods are still terminating: [echo-67574b6655-22hcp echo-67574b6655-fxl2p]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

04:23:46 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
04:23:46 STEP: Ensuring the namespace kube-system exists
04:23:46 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
04:23:46 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
04:23:46 STEP: Deleting cep [echo-67574b6655-22hcp,echo-67574b6655-fxl2p] in namespace default
04:23:46 STEP: Waiting for 2 deletes to return (echo-67574b6655-22hcp,echo-67574b6655-fxl2p)
04:23:46 STEP: Deleting pods [echo-67574b6655-22hcp,echo-67574b6655-fxl2p] in namespace default
04:23:46 STEP: Waiting for 2 deletes to return (echo-67574b6655-22hcp,echo-67574b6655-fxl2p)
04:23:56 STEP: Unable to delete pods echo-67574b6655-fxl2p with 'kubectl -n default delete pods echo-67574b6655-fxl2p': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-fxl2p" deleted
	 
Stderr:
 	 

04:23:56 STEP: Unable to delete pods echo-67574b6655-22hcp with 'kubectl -n default delete pods echo-67574b6655-22hcp': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-22hcp" deleted
	 
Stderr:
 	 

04:23:56 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
04:24:17 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
04:24:31 STEP: Cleaning bpf_lxc build files
04:24:31 STEP: Building bpf_lxc object file
04:24:34 STEP: Running the verifier test script with bpf_lxc
04:24:34 STEP: Cleaning bpf_lxc build files
04:24:35 STEP: Building bpf_lxc object file
04:24:36 STEP: Running the verifier test script with bpf_lxc
04:24:36 STEP: Cleaning bpf_lxc build files
04:24:37 STEP: Building bpf_lxc object file
04:24:38 STEP: Running the verifier test script with bpf_lxc
04:24:38 STEP: Cleaning bpf_host build files
04:24:38 STEP: Building bpf_host object file
04:24:42 STEP: Running the verifier test script with bpf_host
04:24:44 STEP: Cleaning bpf_host build files
04:24:45 STEP: Building bpf_host object file
04:24:47 STEP: Running the verifier test script with bpf_host
04:24:48 STEP: Cleaning bpf_host build files
04:24:48 STEP: Building bpf_host object file
04:24:50 STEP: Running the verifier test script with bpf_host
04:24:51 STEP: Cleaning bpf_xdp build files
04:24:51 STEP: Building bpf_xdp object file
04:24:54 STEP: Running the verifier test script with bpf_xdp
04:24:55 STEP: Cleaning bpf_overlay build files
04:24:55 STEP: Building bpf_overlay object file
04:24:57 STEP: Running the verifier test script with bpf_overlay
04:24:58 STEP: Cleaning bpf_sock build files
04:24:58 STEP: Building bpf_sock object file
04:24:59 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-10T04:24:59Z====
04:24:59 STEP: Running AfterEach for block EntireTestsuite

04:24:59 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000808540>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-22hcp echo-67574b6655-fxl2p]",
        err: <*errors.errorString | 0xc000033720>{
            s: "Pods are still terminating: [echo-67574b6655-22hcp echo-67574b6655-fxl2p]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2080/artifact/cc3a77bb_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2080/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2080_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2080/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-10T05:43:39Z

PR #25346 hit this flake with 94.10% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0007abde0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-5kmbm echo-bc5cf6ffb-h52kz]",
        err: <*errors.errorString | 0xc0017c8360>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-5kmbm echo-bc5cf6ffb-h52kz]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

04:48:26 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
04:48:26 STEP: Ensuring the namespace kube-system exists
04:48:26 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
04:48:26 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
04:48:26 STEP: Deleting pods [echo-bc5cf6ffb-5kmbm,echo-bc5cf6ffb-h52kz] in namespace default
04:48:26 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-5kmbm,echo-bc5cf6ffb-h52kz)
04:48:26 STEP: Deleting cep [echo-bc5cf6ffb-5kmbm,echo-bc5cf6ffb-h52kz] in namespace default
04:48:26 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-5kmbm,echo-bc5cf6ffb-h52kz)
04:48:36 STEP: Unable to delete pods echo-bc5cf6ffb-h52kz with 'kubectl -n default delete pods echo-bc5cf6ffb-h52kz': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-h52kz" deleted
	 
Stderr:
 	 

04:48:36 STEP: Unable to delete pods echo-bc5cf6ffb-5kmbm with 'kubectl -n default delete pods echo-bc5cf6ffb-5kmbm': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-5kmbm" deleted
	 
Stderr:
 	 

04:48:36 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
04:48:52 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
04:49:05 STEP: Cleaning bpf_lxc build files
04:49:05 STEP: Building bpf_lxc object file
04:49:08 STEP: Running the verifier test script with bpf_lxc
04:49:08 STEP: Cleaning bpf_lxc build files
04:49:08 STEP: Building bpf_lxc object file
04:49:10 STEP: Running the verifier test script with bpf_lxc
04:49:10 STEP: Cleaning bpf_lxc build files
04:49:10 STEP: Building bpf_lxc object file
04:49:12 STEP: Running the verifier test script with bpf_lxc
04:49:12 STEP: Cleaning bpf_host build files
04:49:12 STEP: Building bpf_host object file
04:49:16 STEP: Running the verifier test script with bpf_host
04:49:18 STEP: Cleaning bpf_host build files
04:49:18 STEP: Building bpf_host object file
04:49:21 STEP: Running the verifier test script with bpf_host
04:49:22 STEP: Cleaning bpf_host build files
04:49:22 STEP: Building bpf_host object file
04:49:24 STEP: Running the verifier test script with bpf_host
04:49:25 STEP: Cleaning bpf_xdp build files
04:49:25 STEP: Building bpf_xdp object file
04:49:28 STEP: Running the verifier test script with bpf_xdp
04:49:28 STEP: Cleaning bpf_overlay build files
04:49:29 STEP: Building bpf_overlay object file
04:49:31 STEP: Running the verifier test script with bpf_overlay
04:49:32 STEP: Cleaning bpf_sock build files
04:49:32 STEP: Building bpf_sock object file
04:49:32 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-10T04:49:33Z====
04:49:33 STEP: Running AfterEach for block EntireTestsuite

04:49:33 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0007abde0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-5kmbm echo-bc5cf6ffb-h52kz]",
        err: <*errors.errorString | 0xc0017c8360>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-5kmbm echo-bc5cf6ffb-h52kz]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//37/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_37_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/37/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-10T14:54:00Z

PR #25346 hit this flake with 89.66% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00053bc80>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-pwzvb echo-67574b6655-z5hbn]",
        err: <*errors.errorString | 0xc0000b0e90>{
            s: "Pods are still terminating: [echo-67574b6655-pwzvb echo-67574b6655-z5hbn]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

14:04:00 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
14:04:00 STEP: Ensuring the namespace kube-system exists
14:04:01 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
14:04:01 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
14:04:01 STEP: Deleting cep [echo-67574b6655-pwzvb,echo-67574b6655-z5hbn] in namespace default
14:04:01 STEP: Waiting for 2 deletes to return (echo-67574b6655-pwzvb,echo-67574b6655-z5hbn)
14:04:01 STEP: Deleting pods [echo-67574b6655-pwzvb,echo-67574b6655-z5hbn] in namespace default
14:04:01 STEP: Waiting for 2 deletes to return (echo-67574b6655-pwzvb,echo-67574b6655-z5hbn)
14:04:11 STEP: Unable to delete pods echo-67574b6655-z5hbn with 'kubectl -n default delete pods echo-67574b6655-z5hbn': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-z5hbn" deleted
	 
Stderr:
 	 

14:04:11 STEP: Unable to delete pods echo-67574b6655-pwzvb with 'kubectl -n default delete pods echo-67574b6655-pwzvb': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-pwzvb" deleted
	 
Stderr:
 	 

14:04:11 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
14:04:28 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
14:04:41 STEP: Cleaning bpf_lxc build files
14:04:42 STEP: Building bpf_lxc object file
14:04:44 STEP: Running the verifier test script with bpf_lxc
14:04:45 STEP: Cleaning bpf_lxc build files
14:04:45 STEP: Building bpf_lxc object file
14:04:46 STEP: Running the verifier test script with bpf_lxc
14:04:47 STEP: Cleaning bpf_lxc build files
14:04:47 STEP: Building bpf_lxc object file
14:04:48 STEP: Running the verifier test script with bpf_lxc
14:04:49 STEP: Cleaning bpf_host build files
14:04:49 STEP: Building bpf_host object file
14:04:53 STEP: Running the verifier test script with bpf_host
14:04:55 STEP: Cleaning bpf_host build files
14:04:55 STEP: Building bpf_host object file
14:04:57 STEP: Running the verifier test script with bpf_host
14:04:59 STEP: Cleaning bpf_host build files
14:04:59 STEP: Building bpf_host object file
14:05:01 STEP: Running the verifier test script with bpf_host
14:05:02 STEP: Cleaning bpf_xdp build files
14:05:02 STEP: Building bpf_xdp object file
14:05:05 STEP: Running the verifier test script with bpf_xdp
14:05:05 STEP: Cleaning bpf_overlay build files
14:05:05 STEP: Building bpf_overlay object file
14:05:08 STEP: Running the verifier test script with bpf_overlay
14:05:08 STEP: Cleaning bpf_sock build files
14:05:08 STEP: Building bpf_sock object file
14:05:09 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-10T14:05:09Z====
14:05:09 STEP: Running AfterEach for block EntireTestsuite

14:05:09 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00053bc80>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-pwzvb echo-67574b6655-z5hbn]",
        err: <*errors.errorString | 0xc0000b0e90>{
            s: "Pods are still terminating: [echo-67574b6655-pwzvb echo-67574b6655-z5hbn]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2097/artifact/7c261106_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2097/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2097_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2097/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-10T14:58:31Z

PR #25346 hit this flake with 93.95% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000b88940>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-6ghwz echo-bc5cf6ffb-rw2nd]",
        err: <*errors.errorString | 0xc000af00e0>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-6ghwz echo-bc5cf6ffb-rw2nd]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

14:04:06 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
14:04:06 STEP: Ensuring the namespace kube-system exists
14:04:06 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
14:04:06 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
14:04:06 STEP: Deleting pods [echo-bc5cf6ffb-6ghwz,echo-bc5cf6ffb-rw2nd] in namespace default
14:04:06 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-6ghwz,echo-bc5cf6ffb-rw2nd)
14:04:06 STEP: Deleting cep [echo-bc5cf6ffb-6ghwz,echo-bc5cf6ffb-rw2nd] in namespace default
14:04:06 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-6ghwz,echo-bc5cf6ffb-rw2nd)
14:04:16 STEP: Unable to delete pods echo-bc5cf6ffb-6ghwz with 'kubectl -n default delete pods echo-bc5cf6ffb-6ghwz': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-6ghwz" deleted
	 
Stderr:
 	 

14:04:16 STEP: Unable to delete pods echo-bc5cf6ffb-rw2nd with 'kubectl -n default delete pods echo-bc5cf6ffb-rw2nd': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-rw2nd" deleted
	 
Stderr:
 	 

14:04:16 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
14:04:31 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
14:04:45 STEP: Cleaning bpf_lxc build files
14:04:45 STEP: Building bpf_lxc object file
14:04:48 STEP: Running the verifier test script with bpf_lxc
14:04:48 STEP: Cleaning bpf_lxc build files
14:04:49 STEP: Building bpf_lxc object file
14:04:50 STEP: Running the verifier test script with bpf_lxc
14:04:50 STEP: Cleaning bpf_lxc build files
14:04:51 STEP: Building bpf_lxc object file
14:04:52 STEP: Running the verifier test script with bpf_lxc
14:04:52 STEP: Cleaning bpf_host build files
14:04:52 STEP: Building bpf_host object file
14:04:56 STEP: Running the verifier test script with bpf_host
14:04:58 STEP: Cleaning bpf_host build files
14:04:59 STEP: Building bpf_host object file
14:05:01 STEP: Running the verifier test script with bpf_host
14:05:02 STEP: Cleaning bpf_host build files
14:05:02 STEP: Building bpf_host object file
14:05:04 STEP: Running the verifier test script with bpf_host
14:05:05 STEP: Cleaning bpf_xdp build files
14:05:05 STEP: Building bpf_xdp object file
14:05:08 STEP: Running the verifier test script with bpf_xdp
14:05:09 STEP: Cleaning bpf_overlay build files
14:05:09 STEP: Building bpf_overlay object file
14:05:11 STEP: Running the verifier test script with bpf_overlay
14:05:12 STEP: Cleaning bpf_sock build files
14:05:12 STEP: Building bpf_sock object file
14:05:12 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-10T14:05:13Z====
14:05:13 STEP: Running AfterEach for block EntireTestsuite

14:05:13 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000b88940>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-6ghwz echo-bc5cf6ffb-rw2nd]",
        err: <*errors.errorString | 0xc000af00e0>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-6ghwz echo-bc5cf6ffb-rw2nd]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//38/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_38_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/38/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

In the K8sDatapathConfig tests "echo-svc" deployment Pods are failing to terminate while waiting to terminate all Pods. These come from a deployment, that shouldn't be applied in this suite. Presumably failure to delete is being cause by the Deployment controller restarting the Pods as they're deleted. To try to fix this, going to wait for applied yamls in K8sServices to be fully deleted, including finalizers. Addresses: cilium#25255 Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com>

maintainer-s-little-helper · 2023-05-11T09:27:41Z

PR #25346 hit this flake with 88.94% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0005fb3e0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-5h5c6 echo-67574b6655-8hdhl]",
        err: <*errors.errorString | 0xc000153ec0>{
            s: "Pods are still terminating: [echo-67574b6655-5h5c6 echo-67574b6655-8hdhl]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

08:38:47 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
08:38:47 STEP: Ensuring the namespace kube-system exists
08:38:48 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
08:38:48 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
08:38:48 STEP: Deleting cep [echo-67574b6655-5h5c6,echo-67574b6655-8hdhl] in namespace default
08:38:48 STEP: Waiting for 2 deletes to return (echo-67574b6655-5h5c6,echo-67574b6655-8hdhl)
08:38:48 STEP: Deleting pods [echo-67574b6655-5h5c6,echo-67574b6655-8hdhl] in namespace default
08:38:48 STEP: Waiting for 2 deletes to return (echo-67574b6655-5h5c6,echo-67574b6655-8hdhl)
08:38:58 STEP: Unable to delete pods echo-67574b6655-8hdhl with 'kubectl -n default delete pods echo-67574b6655-8hdhl': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-8hdhl" deleted
	 
Stderr:
 	 

08:38:58 STEP: Unable to delete pods echo-67574b6655-5h5c6 with 'kubectl -n default delete pods echo-67574b6655-5h5c6': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-5h5c6" deleted
	 
Stderr:
 	 

08:38:58 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
08:39:12 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
08:39:26 STEP: Cleaning bpf_lxc build files
08:39:26 STEP: Building bpf_lxc object file
08:39:29 STEP: Running the verifier test script with bpf_lxc
08:39:30 STEP: Cleaning bpf_lxc build files
08:39:30 STEP: Building bpf_lxc object file
08:39:31 STEP: Running the verifier test script with bpf_lxc
08:39:32 STEP: Cleaning bpf_lxc build files
08:39:32 STEP: Building bpf_lxc object file
08:39:33 STEP: Running the verifier test script with bpf_lxc
08:39:33 STEP: Cleaning bpf_host build files
08:39:34 STEP: Building bpf_host object file
08:39:38 STEP: Running the verifier test script with bpf_host
08:39:40 STEP: Cleaning bpf_host build files
08:39:40 STEP: Building bpf_host object file
08:39:42 STEP: Running the verifier test script with bpf_host
08:39:44 STEP: Cleaning bpf_host build files
08:39:44 STEP: Building bpf_host object file
08:39:46 STEP: Running the verifier test script with bpf_host
08:39:47 STEP: Cleaning bpf_xdp build files
08:39:47 STEP: Building bpf_xdp object file
08:39:50 STEP: Running the verifier test script with bpf_xdp
08:39:50 STEP: Cleaning bpf_overlay build files
08:39:50 STEP: Building bpf_overlay object file
08:39:53 STEP: Running the verifier test script with bpf_overlay
08:39:53 STEP: Cleaning bpf_sock build files
08:39:54 STEP: Building bpf_sock object file
08:39:54 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-11T08:39:55Z====
08:39:55 STEP: Running AfterEach for block EntireTestsuite

08:39:55 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0005fb3e0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-5h5c6 echo-67574b6655-8hdhl]",
        err: <*errors.errorString | 0xc000153ec0>{
            s: "Pods are still terminating: [echo-67574b6655-5h5c6 echo-67574b6655-8hdhl]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2114/artifact/181eb1ac_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2114/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2114_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2114/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

In the K8sDatapathConfig tests "echo-svc" deployment Pods are failing to terminate while waiting to terminate all Pods. These come from a deployment, that shouldn't be applied in this suite. Presumably failure to delete is being cause by the Deployment controller restarting the Pods as they're deleted. To try to fix this, going to wait for applied yamls in K8sServices to be fully deleted, including finalizers. Addresses: #25255 Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com>

Due to this flake: #25255 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

maintainer-s-little-helper · 2023-05-12T12:02:56Z

PR #25346 hit this flake with 88.79% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000f978a0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-l72dj echo-67574b6655-tx5qj]",
        err: <*errors.errorString | 0xc000ca0120>{
            s: "Pods are still terminating: [echo-67574b6655-l72dj echo-67574b6655-tx5qj]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

11:13:23 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
11:13:23 STEP: Ensuring the namespace kube-system exists
11:13:23 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
11:13:23 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
11:13:23 STEP: Deleting cep [echo-67574b6655-l72dj,echo-67574b6655-tx5qj] in namespace default
11:13:23 STEP: Waiting for 2 deletes to return (echo-67574b6655-l72dj,echo-67574b6655-tx5qj)
11:13:23 STEP: Deleting pods [echo-67574b6655-l72dj,echo-67574b6655-tx5qj] in namespace default
11:13:23 STEP: Waiting for 2 deletes to return (echo-67574b6655-l72dj,echo-67574b6655-tx5qj)
11:13:33 STEP: Unable to delete pods echo-67574b6655-tx5qj with 'kubectl -n default delete pods echo-67574b6655-tx5qj': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-tx5qj" deleted
	 
Stderr:
 	 

11:13:33 STEP: Unable to delete pods echo-67574b6655-l72dj with 'kubectl -n default delete pods echo-67574b6655-l72dj': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-l72dj" deleted
	 
Stderr:
 	 

11:13:33 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
11:13:44 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
11:13:58 STEP: Cleaning bpf_lxc build files
11:13:59 STEP: Building bpf_lxc object file
11:14:02 STEP: Running the verifier test script with bpf_lxc
11:14:03 STEP: Cleaning bpf_lxc build files
11:14:03 STEP: Building bpf_lxc object file
11:14:04 STEP: Running the verifier test script with bpf_lxc
11:14:04 STEP: Cleaning bpf_lxc build files
11:14:05 STEP: Building bpf_lxc object file
11:14:06 STEP: Running the verifier test script with bpf_lxc
11:14:06 STEP: Cleaning bpf_host build files
11:14:07 STEP: Building bpf_host object file
11:14:10 STEP: Running the verifier test script with bpf_host
11:14:12 STEP: Cleaning bpf_host build files
11:14:13 STEP: Building bpf_host object file
11:14:15 STEP: Running the verifier test script with bpf_host
11:14:16 STEP: Cleaning bpf_host build files
11:14:16 STEP: Building bpf_host object file
11:14:18 STEP: Running the verifier test script with bpf_host
11:14:19 STEP: Cleaning bpf_xdp build files
11:14:19 STEP: Building bpf_xdp object file
11:14:22 STEP: Running the verifier test script with bpf_xdp
11:14:23 STEP: Cleaning bpf_overlay build files
11:14:23 STEP: Building bpf_overlay object file
11:14:25 STEP: Running the verifier test script with bpf_overlay
11:14:26 STEP: Cleaning bpf_sock build files
11:14:26 STEP: Building bpf_sock object file
11:14:26 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-12T11:14:27Z====
11:14:27 STEP: Running AfterEach for block EntireTestsuite

11:14:27 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000f978a0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-l72dj echo-67574b6655-tx5qj]",
        err: <*errors.errorString | 0xc000ca0120>{
            s: "Pods are still terminating: [echo-67574b6655-l72dj echo-67574b6655-tx5qj]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2144/artifact/23eb9c85_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2144/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2144_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2144/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-12T14:06:38Z

PR #25125 hit this flake with 94.24% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00089f520>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-hmk6x echo-bc5cf6ffb-k7fr6]",
        err: <*errors.errorString | 0xc0012f3370>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-hmk6x echo-bc5cf6ffb-k7fr6]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

13:11:21 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
13:11:21 STEP: Ensuring the namespace kube-system exists
13:11:21 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
13:11:21 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
13:11:21 STEP: Deleting pods [echo-bc5cf6ffb-hmk6x,echo-bc5cf6ffb-k7fr6] in namespace default
13:11:21 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-hmk6x,echo-bc5cf6ffb-k7fr6)
13:11:21 STEP: Deleting cep [echo-bc5cf6ffb-hmk6x,echo-bc5cf6ffb-k7fr6] in namespace default
13:11:21 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-hmk6x,echo-bc5cf6ffb-k7fr6)
13:11:31 STEP: Unable to delete pods echo-bc5cf6ffb-k7fr6 with 'kubectl -n default delete pods echo-bc5cf6ffb-k7fr6': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-k7fr6" deleted
	 
Stderr:
 	 

13:11:31 STEP: Unable to delete pods echo-bc5cf6ffb-hmk6x with 'kubectl -n default delete pods echo-bc5cf6ffb-hmk6x': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-hmk6x" deleted
	 
Stderr:
 	 

13:11:32 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
13:11:47 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
13:12:00 STEP: Cleaning bpf_lxc build files
13:12:00 STEP: Building bpf_lxc object file
13:12:03 STEP: Running the verifier test script with bpf_lxc
13:12:03 STEP: Cleaning bpf_lxc build files
13:12:04 STEP: Building bpf_lxc object file
13:12:05 STEP: Running the verifier test script with bpf_lxc
13:12:05 STEP: Cleaning bpf_lxc build files
13:12:06 STEP: Building bpf_lxc object file
13:12:07 STEP: Running the verifier test script with bpf_lxc
13:12:07 STEP: Cleaning bpf_host build files
13:12:07 STEP: Building bpf_host object file
13:12:11 STEP: Running the verifier test script with bpf_host
13:12:13 STEP: Cleaning bpf_host build files
13:12:14 STEP: Building bpf_host object file
13:12:16 STEP: Running the verifier test script with bpf_host
13:12:17 STEP: Cleaning bpf_host build files
13:12:17 STEP: Building bpf_host object file
13:12:20 STEP: Running the verifier test script with bpf_host
13:12:20 STEP: Cleaning bpf_xdp build files
13:12:20 STEP: Building bpf_xdp object file
13:12:23 STEP: Running the verifier test script with bpf_xdp
13:12:24 STEP: Cleaning bpf_overlay build files
13:12:24 STEP: Building bpf_overlay object file
13:12:26 STEP: Running the verifier test script with bpf_overlay
13:12:27 STEP: Cleaning bpf_sock build files
13:12:27 STEP: Building bpf_sock object file
13:12:28 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-12T13:12:28Z====
13:12:28 STEP: Running AfterEach for block EntireTestsuite

13:12:28 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00089f520>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-hmk6x echo-bc5cf6ffb-k7fr6]",
        err: <*errors.errorString | 0xc0012f3370>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-hmk6x echo-bc5cf6ffb-k7fr6]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//44/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_44_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/44/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-12T17:19:56Z

PR #25125 hit this flake with 93.66% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000597ba0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-489mb echo-bc5cf6ffb-rqn4z]",
        err: <*errors.errorString | 0xc00057e640>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-489mb echo-bc5cf6ffb-rqn4z]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

16:26:09 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
16:26:09 STEP: Ensuring the namespace kube-system exists
16:26:09 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
16:26:09 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
16:26:09 STEP: Deleting pods [echo-bc5cf6ffb-489mb,echo-bc5cf6ffb-rqn4z] in namespace default
16:26:09 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-489mb,echo-bc5cf6ffb-rqn4z)
16:26:09 STEP: Deleting cep [echo-bc5cf6ffb-489mb,echo-bc5cf6ffb-rqn4z] in namespace default
16:26:09 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-489mb,echo-bc5cf6ffb-rqn4z)
16:26:19 STEP: Unable to delete pods echo-bc5cf6ffb-rqn4z with 'kubectl -n default delete pods echo-bc5cf6ffb-rqn4z': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-rqn4z" deleted
	 
Stderr:
 	 

16:26:19 STEP: Unable to delete pods echo-bc5cf6ffb-489mb with 'kubectl -n default delete pods echo-bc5cf6ffb-489mb': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-489mb" deleted
	 
Stderr:
 	 

16:26:19 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
16:26:39 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
16:26:53 STEP: Cleaning bpf_lxc build files
16:26:54 STEP: Building bpf_lxc object file
16:26:56 STEP: Running the verifier test script with bpf_lxc
16:26:57 STEP: Cleaning bpf_lxc build files
16:26:57 STEP: Building bpf_lxc object file
16:26:58 STEP: Running the verifier test script with bpf_lxc
16:26:59 STEP: Cleaning bpf_lxc build files
16:26:59 STEP: Building bpf_lxc object file
16:27:00 STEP: Running the verifier test script with bpf_lxc
16:27:01 STEP: Cleaning bpf_host build files
16:27:01 STEP: Building bpf_host object file
16:27:05 STEP: Running the verifier test script with bpf_host
16:27:07 STEP: Cleaning bpf_host build files
16:27:07 STEP: Building bpf_host object file
16:27:09 STEP: Running the verifier test script with bpf_host
16:27:10 STEP: Cleaning bpf_host build files
16:27:11 STEP: Building bpf_host object file
16:27:13 STEP: Running the verifier test script with bpf_host
16:27:13 STEP: Cleaning bpf_xdp build files
16:27:14 STEP: Building bpf_xdp object file
16:27:16 STEP: Running the verifier test script with bpf_xdp
16:27:17 STEP: Cleaning bpf_overlay build files
16:27:17 STEP: Building bpf_overlay object file
16:27:19 STEP: Running the verifier test script with bpf_overlay
16:27:20 STEP: Cleaning bpf_sock build files
16:27:20 STEP: Building bpf_sock object file
16:27:21 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-12T16:27:21Z====
16:27:21 STEP: Running AfterEach for block EntireTestsuite

16:27:21 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000597ba0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-489mb echo-bc5cf6ffb-rqn4z]",
        err: <*errors.errorString | 0xc00057e640>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-489mb echo-bc5cf6ffb-rqn4z]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//45/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_45_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/45/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-12T20:19:15Z

PR #25427 hit this flake with 94.53% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000159f20>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-s7hbt echo-bc5cf6ffb-xs2kn]",
        err: <*errors.errorString | 0xc001181200>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-s7hbt echo-bc5cf6ffb-xs2kn]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

19:26:01 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
19:26:01 STEP: Ensuring the namespace kube-system exists
19:26:01 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
19:26:02 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
19:26:02 STEP: Deleting cep [echo-bc5cf6ffb-s7hbt,echo-bc5cf6ffb-xs2kn] in namespace default
19:26:02 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-s7hbt,echo-bc5cf6ffb-xs2kn)
19:26:02 STEP: Deleting pods [echo-bc5cf6ffb-s7hbt,echo-bc5cf6ffb-xs2kn] in namespace default
19:26:02 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-s7hbt,echo-bc5cf6ffb-xs2kn)
19:26:12 STEP: Unable to delete pods echo-bc5cf6ffb-xs2kn with 'kubectl -n default delete pods echo-bc5cf6ffb-xs2kn': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-xs2kn" deleted
	 
Stderr:
 	 

19:26:12 STEP: Unable to delete pods echo-bc5cf6ffb-s7hbt with 'kubectl -n default delete pods echo-bc5cf6ffb-s7hbt': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-s7hbt" deleted
	 
Stderr:
 	 

19:26:12 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
19:26:23 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
19:26:37 STEP: Cleaning bpf_lxc build files
19:26:39 STEP: Building bpf_lxc object file
19:26:41 STEP: Running the verifier test script with bpf_lxc
19:26:42 STEP: Cleaning bpf_lxc build files
19:26:42 STEP: Building bpf_lxc object file
19:26:44 STEP: Running the verifier test script with bpf_lxc
19:26:44 STEP: Cleaning bpf_lxc build files
19:26:44 STEP: Building bpf_lxc object file
19:26:46 STEP: Running the verifier test script with bpf_lxc
19:26:46 STEP: Cleaning bpf_host build files
19:26:46 STEP: Building bpf_host object file
19:26:50 STEP: Running the verifier test script with bpf_host
19:26:52 STEP: Cleaning bpf_host build files
19:26:52 STEP: Building bpf_host object file
19:26:55 STEP: Running the verifier test script with bpf_host
19:26:56 STEP: Cleaning bpf_host build files
19:26:56 STEP: Building bpf_host object file
19:26:58 STEP: Running the verifier test script with bpf_host
19:26:59 STEP: Cleaning bpf_xdp build files
19:26:59 STEP: Building bpf_xdp object file
19:27:02 STEP: Running the verifier test script with bpf_xdp
19:27:02 STEP: Cleaning bpf_overlay build files
19:27:03 STEP: Building bpf_overlay object file
19:27:05 STEP: Running the verifier test script with bpf_overlay
19:27:06 STEP: Cleaning bpf_sock build files
19:27:06 STEP: Building bpf_sock object file
19:27:06 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-12T19:27:07Z====
19:27:07 STEP: Running AfterEach for block EntireTestsuite

19:27:07 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000159f20>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-s7hbt echo-bc5cf6ffb-xs2kn]",
        err: <*errors.errorString | 0xc001181200>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-s7hbt echo-bc5cf6ffb-xs2kn]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//47/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_47_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/47/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-12T21:14:31Z

PR #25427 hit this flake with 89.22% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000769520>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-7gfhd echo-67574b6655-kfh9c]",
        err: <*errors.errorString | 0xc0004f7930>{
            s: "Pods are still terminating: [echo-67574b6655-7gfhd echo-67574b6655-kfh9c]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

20:26:04 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
20:26:04 STEP: Ensuring the namespace kube-system exists
20:26:04 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
20:26:04 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
20:26:04 STEP: Deleting cep [echo-67574b6655-7gfhd,echo-67574b6655-kfh9c] in namespace default
20:26:04 STEP: Waiting for 2 deletes to return (echo-67574b6655-7gfhd,echo-67574b6655-kfh9c)
20:26:04 STEP: Deleting pods [echo-67574b6655-7gfhd,echo-67574b6655-kfh9c] in namespace default
20:26:04 STEP: Waiting for 2 deletes to return (echo-67574b6655-7gfhd,echo-67574b6655-kfh9c)
20:26:14 STEP: Unable to delete pods echo-67574b6655-7gfhd with 'kubectl -n default delete pods echo-67574b6655-7gfhd': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-7gfhd" deleted
	 
Stderr:
 	 

20:26:14 STEP: Unable to delete pods echo-67574b6655-kfh9c with 'kubectl -n default delete pods echo-67574b6655-kfh9c': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-kfh9c" deleted
	 
Stderr:
 	 

20:26:14 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
20:26:29 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
20:26:43 STEP: Cleaning bpf_lxc build files
20:26:43 STEP: Building bpf_lxc object file
20:26:46 STEP: Running the verifier test script with bpf_lxc
20:26:47 STEP: Cleaning bpf_lxc build files
20:26:47 STEP: Building bpf_lxc object file
20:26:48 STEP: Running the verifier test script with bpf_lxc
20:26:49 STEP: Cleaning bpf_lxc build files
20:26:49 STEP: Building bpf_lxc object file
20:26:50 STEP: Running the verifier test script with bpf_lxc
20:26:51 STEP: Cleaning bpf_host build files
20:26:51 STEP: Building bpf_host object file
20:26:55 STEP: Running the verifier test script with bpf_host
20:26:57 STEP: Cleaning bpf_host build files
20:26:57 STEP: Building bpf_host object file
20:26:59 STEP: Running the verifier test script with bpf_host
20:27:01 STEP: Cleaning bpf_host build files
20:27:01 STEP: Building bpf_host object file
20:27:03 STEP: Running the verifier test script with bpf_host
20:27:04 STEP: Cleaning bpf_xdp build files
20:27:04 STEP: Building bpf_xdp object file
20:27:07 STEP: Running the verifier test script with bpf_xdp
20:27:07 STEP: Cleaning bpf_overlay build files
20:27:07 STEP: Building bpf_overlay object file
20:27:10 STEP: Running the verifier test script with bpf_overlay
20:27:10 STEP: Cleaning bpf_sock build files
20:27:11 STEP: Building bpf_sock object file
20:27:11 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-12T20:27:12Z====
20:27:12 STEP: Running AfterEach for block EntireTestsuite

20:27:12 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000769520>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-7gfhd echo-67574b6655-kfh9c]",
        err: <*errors.errorString | 0xc0004f7930>{
            s: "Pods are still terminating: [echo-67574b6655-7gfhd echo-67574b6655-kfh9c]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2162/artifact/442a9ba6_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2162/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2162_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2162/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-13T08:20:49Z

PR #25409 hit this flake with 88.51% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc001889520>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-g8rtf echo-67574b6655-n7c8p]",
        err: <*errors.errorString | 0xc000893330>{
            s: "Pods are still terminating: [echo-67574b6655-g8rtf echo-67574b6655-n7c8p]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

07:31:08 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
07:31:08 STEP: Ensuring the namespace kube-system exists
07:31:08 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
07:31:08 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
07:31:08 STEP: Deleting pods [echo-67574b6655-g8rtf,echo-67574b6655-n7c8p] in namespace default
07:31:08 STEP: Waiting for 2 deletes to return (echo-67574b6655-g8rtf,echo-67574b6655-n7c8p)
07:31:08 STEP: Deleting cep [echo-67574b6655-g8rtf,echo-67574b6655-n7c8p] in namespace default
07:31:08 STEP: Waiting for 2 deletes to return (echo-67574b6655-g8rtf,echo-67574b6655-n7c8p)
07:31:18 STEP: Unable to delete pods echo-67574b6655-n7c8p with 'kubectl -n default delete pods echo-67574b6655-n7c8p': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-n7c8p" deleted
	 
Stderr:
 	 

07:31:18 STEP: Unable to delete pods echo-67574b6655-g8rtf with 'kubectl -n default delete pods echo-67574b6655-g8rtf': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-g8rtf" deleted
	 
Stderr:
 	 

07:31:18 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
07:31:34 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
07:31:47 STEP: Cleaning bpf_lxc build files
07:31:48 STEP: Building bpf_lxc object file
07:31:50 STEP: Running the verifier test script with bpf_lxc
07:31:51 STEP: Cleaning bpf_lxc build files
07:31:51 STEP: Building bpf_lxc object file
07:31:52 STEP: Running the verifier test script with bpf_lxc
07:31:53 STEP: Cleaning bpf_lxc build files
07:31:53 STEP: Building bpf_lxc object file
07:31:54 STEP: Running the verifier test script with bpf_lxc
07:31:55 STEP: Cleaning bpf_host build files
07:31:55 STEP: Building bpf_host object file
07:31:59 STEP: Running the verifier test script with bpf_host
07:32:01 STEP: Cleaning bpf_host build files
07:32:01 STEP: Building bpf_host object file
07:32:04 STEP: Running the verifier test script with bpf_host
07:32:05 STEP: Cleaning bpf_host build files
07:32:05 STEP: Building bpf_host object file
07:32:07 STEP: Running the verifier test script with bpf_host
07:32:08 STEP: Cleaning bpf_xdp build files
07:32:08 STEP: Building bpf_xdp object file
07:32:11 STEP: Running the verifier test script with bpf_xdp
07:32:12 STEP: Cleaning bpf_overlay build files
07:32:12 STEP: Building bpf_overlay object file
07:32:14 STEP: Running the verifier test script with bpf_overlay
07:32:15 STEP: Cleaning bpf_sock build files
07:32:15 STEP: Building bpf_sock object file
07:32:16 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-13T07:32:16Z====
07:32:16 STEP: Running AfterEach for block EntireTestsuite

07:32:16 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc001889520>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-g8rtf echo-67574b6655-n7c8p]",
        err: <*errors.errorString | 0xc000893330>{
            s: "Pods are still terminating: [echo-67574b6655-g8rtf echo-67574b6655-n7c8p]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2167/artifact/ad127378_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2167/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2167_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2167/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-15T10:10:04Z

PR #25346 hit this flake with 87.93% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0008a79c0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-5gdql echo-67574b6655-kw8q8]",
        err: <*errors.errorString | 0xc000637be0>{
            s: "Pods are still terminating: [echo-67574b6655-5gdql echo-67574b6655-kw8q8]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

09:20:18 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
09:20:18 STEP: Ensuring the namespace kube-system exists
09:20:19 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
09:20:19 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
09:20:19 STEP: Deleting pods [echo-67574b6655-5gdql,echo-67574b6655-kw8q8] in namespace default
09:20:19 STEP: Waiting for 2 deletes to return (echo-67574b6655-5gdql,echo-67574b6655-kw8q8)
09:20:19 STEP: Deleting cep [echo-67574b6655-5gdql,echo-67574b6655-kw8q8] in namespace default
09:20:19 STEP: Waiting for 2 deletes to return (echo-67574b6655-5gdql,echo-67574b6655-kw8q8)
09:20:29 STEP: Unable to delete pods echo-67574b6655-kw8q8 with 'kubectl -n default delete pods echo-67574b6655-kw8q8': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-kw8q8" deleted
	 
Stderr:
 	 

09:20:29 STEP: Unable to delete pods echo-67574b6655-5gdql with 'kubectl -n default delete pods echo-67574b6655-5gdql': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-5gdql" deleted
	 
Stderr:
 	 

09:20:29 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
09:20:41 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
09:20:56 STEP: Cleaning bpf_lxc build files
09:20:58 STEP: Building bpf_lxc object file
09:21:01 STEP: Running the verifier test script with bpf_lxc
09:21:02 STEP: Cleaning bpf_lxc build files
09:21:02 STEP: Building bpf_lxc object file
09:21:03 STEP: Running the verifier test script with bpf_lxc
09:21:04 STEP: Cleaning bpf_lxc build files
09:21:04 STEP: Building bpf_lxc object file
09:21:05 STEP: Running the verifier test script with bpf_lxc
09:21:06 STEP: Cleaning bpf_host build files
09:21:06 STEP: Building bpf_host object file
09:21:10 STEP: Running the verifier test script with bpf_host
09:21:12 STEP: Cleaning bpf_host build files
09:21:12 STEP: Building bpf_host object file
09:21:14 STEP: Running the verifier test script with bpf_host
09:21:16 STEP: Cleaning bpf_host build files
09:21:16 STEP: Building bpf_host object file
09:21:18 STEP: Running the verifier test script with bpf_host
09:21:19 STEP: Cleaning bpf_xdp build files
09:21:19 STEP: Building bpf_xdp object file
09:21:22 STEP: Running the verifier test script with bpf_xdp
09:21:22 STEP: Cleaning bpf_overlay build files
09:21:22 STEP: Building bpf_overlay object file
09:21:25 STEP: Running the verifier test script with bpf_overlay
09:21:25 STEP: Cleaning bpf_sock build files
09:21:26 STEP: Building bpf_sock object file
09:21:26 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-15T09:21:27Z====
09:21:27 STEP: Running AfterEach for block EntireTestsuite

09:21:27 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0008a79c0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-5gdql echo-67574b6655-kw8q8]",
        err: <*errors.errorString | 0xc000637be0>{
            s: "Pods are still terminating: [echo-67574b6655-5gdql echo-67574b6655-kw8q8]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2192/artifact/ecd56b17_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2192/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2192_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2192/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-15T13:21:12Z

PR #25346 hit this flake with 89.66% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00044bdc0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-4prsr echo-67574b6655-v6sng]",
        err: <*errors.errorString | 0xc00047be10>{
            s: "Pods are still terminating: [echo-67574b6655-4prsr echo-67574b6655-v6sng]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

12:30:36 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
12:30:36 STEP: Ensuring the namespace kube-system exists
12:30:36 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
12:30:36 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
12:30:36 STEP: Deleting cep [echo-67574b6655-4prsr,echo-67574b6655-v6sng] in namespace default
12:30:36 STEP: Waiting for 2 deletes to return (echo-67574b6655-4prsr,echo-67574b6655-v6sng)
12:30:36 STEP: Deleting pods [echo-67574b6655-4prsr,echo-67574b6655-v6sng] in namespace default
12:30:36 STEP: Waiting for 2 deletes to return (echo-67574b6655-4prsr,echo-67574b6655-v6sng)
12:30:46 STEP: Unable to delete pods echo-67574b6655-v6sng with 'kubectl -n default delete pods echo-67574b6655-v6sng': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-v6sng" deleted
	 
Stderr:
 	 

12:30:46 STEP: Unable to delete pods echo-67574b6655-4prsr with 'kubectl -n default delete pods echo-67574b6655-4prsr': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-4prsr" deleted
	 
Stderr:
 	 

12:30:46 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
12:31:01 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
12:31:15 STEP: Cleaning bpf_lxc build files
12:31:15 STEP: Building bpf_lxc object file
12:31:18 STEP: Running the verifier test script with bpf_lxc
12:31:18 STEP: Cleaning bpf_lxc build files
12:31:18 STEP: Building bpf_lxc object file
12:31:20 STEP: Running the verifier test script with bpf_lxc
12:31:20 STEP: Cleaning bpf_lxc build files
12:31:20 STEP: Building bpf_lxc object file
12:31:22 STEP: Running the verifier test script with bpf_lxc
12:31:22 STEP: Cleaning bpf_host build files
12:31:22 STEP: Building bpf_host object file
12:31:26 STEP: Running the verifier test script with bpf_host
12:31:28 STEP: Cleaning bpf_host build files
12:31:28 STEP: Building bpf_host object file
12:31:31 STEP: Running the verifier test script with bpf_host
12:31:32 STEP: Cleaning bpf_host build files
12:31:32 STEP: Building bpf_host object file
12:31:34 STEP: Running the verifier test script with bpf_host
12:31:35 STEP: Cleaning bpf_xdp build files
12:31:35 STEP: Building bpf_xdp object file
12:31:38 STEP: Running the verifier test script with bpf_xdp
12:31:38 STEP: Cleaning bpf_overlay build files
12:31:39 STEP: Building bpf_overlay object file
12:31:41 STEP: Running the verifier test script with bpf_overlay
12:31:42 STEP: Cleaning bpf_sock build files
12:31:42 STEP: Building bpf_sock object file
12:31:42 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-15T12:31:43Z====
12:31:43 STEP: Running AfterEach for block EntireTestsuite

12:31:43 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00044bdc0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-4prsr echo-67574b6655-v6sng]",
        err: <*errors.errorString | 0xc00047be10>{
            s: "Pods are still terminating: [echo-67574b6655-4prsr echo-67574b6655-v6sng]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2196/artifact/55e9d3c2_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2196/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2196_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2196/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-15T18:43:51Z

PR #25346 hit this flake with 89.51% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00058a480>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-889zv echo-67574b6655-c5j92]",
        err: <*errors.errorString | 0xc002204cb0>{
            s: "Pods are still terminating: [echo-67574b6655-889zv echo-67574b6655-c5j92]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

17:43:17 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
17:43:17 STEP: Ensuring the namespace kube-system exists
17:43:17 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
17:43:17 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
17:43:17 STEP: Deleting pods [echo-67574b6655-889zv,echo-67574b6655-c5j92] in namespace default
17:43:17 STEP: Waiting for 2 deletes to return (echo-67574b6655-889zv,echo-67574b6655-c5j92)
17:43:17 STEP: Deleting cep [echo-67574b6655-889zv,echo-67574b6655-c5j92] in namespace default
17:43:17 STEP: Waiting for 2 deletes to return (echo-67574b6655-889zv,echo-67574b6655-c5j92)
17:43:27 STEP: Unable to delete pods echo-67574b6655-c5j92 with 'kubectl -n default delete pods echo-67574b6655-c5j92': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-c5j92" deleted
	 
Stderr:
 	 

17:43:27 STEP: Unable to delete pods echo-67574b6655-889zv with 'kubectl -n default delete pods echo-67574b6655-889zv': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-889zv" deleted
	 
Stderr:
 	 

17:43:27 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
17:43:45 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
17:43:58 STEP: Cleaning bpf_lxc build files
17:43:59 STEP: Building bpf_lxc object file
17:44:01 STEP: Running the verifier test script with bpf_lxc
17:44:02 STEP: Cleaning bpf_lxc build files
17:44:02 STEP: Building bpf_lxc object file
17:44:04 STEP: Running the verifier test script with bpf_lxc
17:44:04 STEP: Cleaning bpf_lxc build files
17:44:04 STEP: Building bpf_lxc object file
17:44:06 STEP: Running the verifier test script with bpf_lxc
17:44:06 STEP: Cleaning bpf_host build files
17:44:06 STEP: Building bpf_host object file
17:44:10 STEP: Running the verifier test script with bpf_host
17:44:12 STEP: Cleaning bpf_host build files
17:44:12 STEP: Building bpf_host object file
17:44:14 STEP: Running the verifier test script with bpf_host
17:44:16 STEP: Cleaning bpf_host build files
17:44:16 STEP: Building bpf_host object file
17:44:18 STEP: Running the verifier test script with bpf_host
17:44:19 STEP: Cleaning bpf_xdp build files
17:44:19 STEP: Building bpf_xdp object file
17:44:22 STEP: Running the verifier test script with bpf_xdp
17:44:22 STEP: Cleaning bpf_overlay build files
17:44:22 STEP: Building bpf_overlay object file
17:44:25 STEP: Running the verifier test script with bpf_overlay
17:44:25 STEP: Cleaning bpf_sock build files
17:44:26 STEP: Building bpf_sock object file
17:44:26 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-15T17:44:27Z====
17:44:27 STEP: Running AfterEach for block EntireTestsuite

17:44:27 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00058a480>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-889zv echo-67574b6655-c5j92]",
        err: <*errors.errorString | 0xc002204cb0>{
            s: "Pods are still terminating: [echo-67574b6655-889zv echo-67574b6655-c5j92]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2205/artifact/2678c176_K8sDatapathConfig_Host_firewall_With_VXLAN.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2205/artifact/2f5d5560_K8sDatapathConfig_Host_firewall_With_VXLAN_and_endpoint_routes.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2205/artifact/869a3c66_K8sDatapathConfig_Host_firewall_With_native_routing_and_endpoint_routes.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2205/artifact/e796306d_K8sDatapathConfig_Host_firewall_With_native_routing.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2205/artifact/ef6b81c0_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2205/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2205_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2205/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-15T20:28:06Z

PR #25454 hit this flake with 88.79% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00070b760>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-76sst echo-67574b6655-vrqbg]",
        err: <*errors.errorString | 0xc001d8ab60>{
            s: "Pods are still terminating: [echo-67574b6655-76sst echo-67574b6655-vrqbg]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

19:38:46 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
19:38:46 STEP: Ensuring the namespace kube-system exists
19:38:46 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
19:38:46 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
19:38:46 STEP: Deleting pods [echo-67574b6655-76sst,echo-67574b6655-vrqbg] in namespace default
19:38:46 STEP: Waiting for 2 deletes to return (echo-67574b6655-76sst,echo-67574b6655-vrqbg)
19:38:46 STEP: Deleting cep [echo-67574b6655-76sst,echo-67574b6655-vrqbg] in namespace default
19:38:46 STEP: Waiting for 2 deletes to return (echo-67574b6655-76sst,echo-67574b6655-vrqbg)
19:38:56 STEP: Unable to delete pods echo-67574b6655-vrqbg with 'kubectl -n default delete pods echo-67574b6655-vrqbg': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-vrqbg" deleted
	 
Stderr:
 	 

19:38:56 STEP: Unable to delete pods echo-67574b6655-76sst with 'kubectl -n default delete pods echo-67574b6655-76sst': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-76sst" deleted
	 
Stderr:
 	 

19:38:57 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
19:39:07 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
19:39:21 STEP: Cleaning bpf_lxc build files
19:39:23 STEP: Building bpf_lxc object file
19:39:25 STEP: Running the verifier test script with bpf_lxc
19:39:26 STEP: Cleaning bpf_lxc build files
19:39:26 STEP: Building bpf_lxc object file
19:39:28 STEP: Running the verifier test script with bpf_lxc
19:39:28 STEP: Cleaning bpf_lxc build files
19:39:28 STEP: Building bpf_lxc object file
19:39:30 STEP: Running the verifier test script with bpf_lxc
19:39:30 STEP: Cleaning bpf_host build files
19:39:30 STEP: Building bpf_host object file
19:39:34 STEP: Running the verifier test script with bpf_host
19:39:36 STEP: Cleaning bpf_host build files
19:39:36 STEP: Building bpf_host object file
19:39:38 STEP: Running the verifier test script with bpf_host
19:39:40 STEP: Cleaning bpf_host build files
19:39:40 STEP: Building bpf_host object file
19:39:42 STEP: Running the verifier test script with bpf_host
19:39:43 STEP: Cleaning bpf_xdp build files
19:39:43 STEP: Building bpf_xdp object file
19:39:46 STEP: Running the verifier test script with bpf_xdp
19:39:46 STEP: Cleaning bpf_overlay build files
19:39:46 STEP: Building bpf_overlay object file
19:39:49 STEP: Running the verifier test script with bpf_overlay
19:39:49 STEP: Cleaning bpf_sock build files
19:39:49 STEP: Building bpf_sock object file
19:39:50 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-15T19:39:51Z====
19:39:51 STEP: Running AfterEach for block EntireTestsuite

19:39:51 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc00070b760>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-76sst echo-67574b6655-vrqbg]",
        err: <*errors.errorString | 0xc001d8ab60>{
            s: "Pods are still terminating: [echo-67574b6655-76sst echo-67574b6655-vrqbg]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2207/artifact/d0f2e398_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2207/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2207_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2207/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-16T07:47:03Z

PR #25424 hit this flake with 93.81% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000848160>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-5nlmh echo-bc5cf6ffb-6dhrk]",
        err: <*errors.errorString | 0xc0003fdf70>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-5nlmh echo-bc5cf6ffb-6dhrk]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

06:52:47 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
06:52:47 STEP: Ensuring the namespace kube-system exists
06:52:47 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
06:52:47 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
06:52:47 STEP: Deleting pods [echo-bc5cf6ffb-5nlmh,echo-bc5cf6ffb-6dhrk] in namespace default
06:52:47 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-5nlmh,echo-bc5cf6ffb-6dhrk)
06:52:47 STEP: Deleting cep [echo-bc5cf6ffb-5nlmh,echo-bc5cf6ffb-6dhrk] in namespace default
06:52:47 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-5nlmh,echo-bc5cf6ffb-6dhrk)
06:52:57 STEP: Unable to delete pods echo-bc5cf6ffb-6dhrk with 'kubectl -n default delete pods echo-bc5cf6ffb-6dhrk': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-6dhrk" deleted
	 
Stderr:
 	 

06:52:57 STEP: Unable to delete pods echo-bc5cf6ffb-5nlmh with 'kubectl -n default delete pods echo-bc5cf6ffb-5nlmh': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-5nlmh" deleted
	 
Stderr:
 	 

06:52:57 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
06:53:08 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
06:53:22 STEP: Cleaning bpf_lxc build files
06:53:23 STEP: Building bpf_lxc object file
06:53:26 STEP: Running the verifier test script with bpf_lxc
06:53:26 STEP: Cleaning bpf_lxc build files
06:53:27 STEP: Building bpf_lxc object file
06:53:28 STEP: Running the verifier test script with bpf_lxc
06:53:28 STEP: Cleaning bpf_lxc build files
06:53:29 STEP: Building bpf_lxc object file
06:53:30 STEP: Running the verifier test script with bpf_lxc
06:53:30 STEP: Cleaning bpf_host build files
06:53:30 STEP: Building bpf_host object file
06:53:34 STEP: Running the verifier test script with bpf_host
06:53:36 STEP: Cleaning bpf_host build files
06:53:36 STEP: Building bpf_host object file
06:53:38 STEP: Running the verifier test script with bpf_host
06:53:40 STEP: Cleaning bpf_host build files
06:53:40 STEP: Building bpf_host object file
06:53:42 STEP: Running the verifier test script with bpf_host
06:53:43 STEP: Cleaning bpf_xdp build files
06:53:43 STEP: Building bpf_xdp object file
06:53:46 STEP: Running the verifier test script with bpf_xdp
06:53:46 STEP: Cleaning bpf_overlay build files
06:53:46 STEP: Building bpf_overlay object file
06:53:49 STEP: Running the verifier test script with bpf_overlay
06:53:49 STEP: Cleaning bpf_sock build files
06:53:50 STEP: Building bpf_sock object file
06:53:50 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-16T06:53:51Z====
06:53:51 STEP: Running AfterEach for block EntireTestsuite

06:53:51 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000848160>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-5nlmh echo-bc5cf6ffb-6dhrk]",
        err: <*errors.errorString | 0xc0003fdf70>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-5nlmh echo-bc5cf6ffb-6dhrk]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//53/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_53_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/53/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-16T10:56:05Z

PR #25424 hit this flake with 89.37% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc001249360>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-7kwpg echo-67574b6655-cqh4r]",
        err: <*errors.errorString | 0xc000613570>{
            s: "Pods are still terminating: [echo-67574b6655-7kwpg echo-67574b6655-cqh4r]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.25-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

10:06:32 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
10:06:32 STEP: Ensuring the namespace kube-system exists
10:06:32 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
10:06:32 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
10:06:32 STEP: Deleting cep [echo-67574b6655-7kwpg,echo-67574b6655-cqh4r] in namespace default
10:06:32 STEP: Waiting for 2 deletes to return (echo-67574b6655-7kwpg,echo-67574b6655-cqh4r)
10:06:32 STEP: Deleting pods [echo-67574b6655-7kwpg,echo-67574b6655-cqh4r] in namespace default
10:06:32 STEP: Waiting for 2 deletes to return (echo-67574b6655-7kwpg,echo-67574b6655-cqh4r)
10:06:42 STEP: Unable to delete pods echo-67574b6655-7kwpg with 'kubectl -n default delete pods echo-67574b6655-7kwpg': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-7kwpg" deleted
	 
Stderr:
 	 

10:06:42 STEP: Unable to delete pods echo-67574b6655-cqh4r with 'kubectl -n default delete pods echo-67574b6655-cqh4r': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-67574b6655-cqh4r" deleted
	 
Stderr:
 	 

10:06:42 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
10:06:53 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
10:07:08 STEP: Cleaning bpf_lxc build files
10:07:09 STEP: Building bpf_lxc object file
10:07:11 STEP: Running the verifier test script with bpf_lxc
10:07:12 STEP: Cleaning bpf_lxc build files
10:07:12 STEP: Building bpf_lxc object file
10:07:14 STEP: Running the verifier test script with bpf_lxc
10:07:14 STEP: Cleaning bpf_lxc build files
10:07:14 STEP: Building bpf_lxc object file
10:07:16 STEP: Running the verifier test script with bpf_lxc
10:07:16 STEP: Cleaning bpf_host build files
10:07:16 STEP: Building bpf_host object file
10:07:20 STEP: Running the verifier test script with bpf_host
10:07:22 STEP: Cleaning bpf_host build files
10:07:22 STEP: Building bpf_host object file
10:07:24 STEP: Running the verifier test script with bpf_host
10:07:26 STEP: Cleaning bpf_host build files
10:07:26 STEP: Building bpf_host object file
10:07:28 STEP: Running the verifier test script with bpf_host
10:07:29 STEP: Cleaning bpf_xdp build files
10:07:29 STEP: Building bpf_xdp object file
10:07:32 STEP: Running the verifier test script with bpf_xdp
10:07:33 STEP: Cleaning bpf_overlay build files
10:07:33 STEP: Building bpf_overlay object file
10:07:35 STEP: Running the verifier test script with bpf_overlay
10:07:36 STEP: Cleaning bpf_sock build files
10:07:36 STEP: Building bpf_sock object file
10:07:37 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-16T10:07:37Z====
10:07:37 STEP: Running AfterEach for block EntireTestsuite

10:07:37 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc001249360>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-67574b6655-7kwpg echo-67574b6655-cqh4r]",
        err: <*errors.errorString | 0xc000613570>{
            s: "Pods are still terminating: [echo-67574b6655-7kwpg echo-67574b6655-cqh4r]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2222/artifact/e9397955_K8sUpdates_Tests_upgrade_and_downgrade_from_a_Cilium_stable_image_to_master.zip
https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19//2222/artifact/test_results_Cilium-PR-K8s-1.25-kernel-4.19_2222_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.25-kernel-4.19/2222/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-16T16:01:44Z

PR #25421 hit this flake with 93.38% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0007aa700>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-8lck9 echo-bc5cf6ffb-kmmhh]",
        err: <*errors.errorString | 0xc0012268d0>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-8lck9 echo-bc5cf6ffb-kmmhh]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

15:07:39 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
15:07:39 STEP: Ensuring the namespace kube-system exists
15:07:40 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
15:07:40 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
15:07:40 STEP: Deleting pods [echo-bc5cf6ffb-8lck9,echo-bc5cf6ffb-kmmhh] in namespace default
15:07:40 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-8lck9,echo-bc5cf6ffb-kmmhh)
15:07:40 STEP: Deleting cep [echo-bc5cf6ffb-8lck9,echo-bc5cf6ffb-kmmhh] in namespace default
15:07:40 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-8lck9,echo-bc5cf6ffb-kmmhh)
15:07:50 STEP: Unable to delete pods echo-bc5cf6ffb-8lck9 with 'kubectl -n default delete pods echo-bc5cf6ffb-8lck9': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-8lck9" deleted
	 
Stderr:
 	 

15:07:50 STEP: Unable to delete pods echo-bc5cf6ffb-kmmhh with 'kubectl -n default delete pods echo-bc5cf6ffb-kmmhh': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-kmmhh" deleted
	 
Stderr:
 	 

15:07:50 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
15:08:04 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
15:08:17 STEP: Cleaning bpf_lxc build files
15:08:18 STEP: Building bpf_lxc object file
15:08:20 STEP: Running the verifier test script with bpf_lxc
15:08:21 STEP: Cleaning bpf_lxc build files
15:08:21 STEP: Building bpf_lxc object file
15:08:22 STEP: Running the verifier test script with bpf_lxc
15:08:23 STEP: Cleaning bpf_lxc build files
15:08:23 STEP: Building bpf_lxc object file
15:08:24 STEP: Running the verifier test script with bpf_lxc
15:08:25 STEP: Cleaning bpf_host build files
15:08:25 STEP: Building bpf_host object file
15:08:29 STEP: Running the verifier test script with bpf_host
15:08:31 STEP: Cleaning bpf_host build files
15:08:31 STEP: Building bpf_host object file
15:08:33 STEP: Running the verifier test script with bpf_host
15:08:34 STEP: Cleaning bpf_host build files
15:08:35 STEP: Building bpf_host object file
15:08:37 STEP: Running the verifier test script with bpf_host
15:08:37 STEP: Cleaning bpf_xdp build files
15:08:38 STEP: Building bpf_xdp object file
15:08:40 STEP: Running the verifier test script with bpf_xdp
15:08:41 STEP: Cleaning bpf_overlay build files
15:08:41 STEP: Building bpf_overlay object file
15:08:43 STEP: Running the verifier test script with bpf_overlay
15:08:44 STEP: Cleaning bpf_sock build files
15:08:44 STEP: Building bpf_sock object file
15:08:45 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-16T15:08:45Z====
15:08:45 STEP: Running AfterEach for block EntireTestsuite

15:08:45 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc0007aa700>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-8lck9 echo-bc5cf6ffb-kmmhh]",
        err: <*errors.errorString | 0xc0012268d0>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-8lck9 echo-bc5cf6ffb-kmmhh]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//58/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_58_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/58/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

maintainer-s-little-helper · 2023-05-16T19:31:36Z

PR #25421 hit this flake with 93.09% similarity:

Click to show.

Test Name

K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath

Failure Output

FAIL: terminating containers are not deleted after timeout

Stacktrace

Click to show.

/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/test/ginkgo-ext/scopes.go:415
terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000b0bda0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-gsnbm echo-bc5cf6ffb-zw569]",
        err: <*errors.errorString | 0xc00027f750>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-gsnbm echo-bc5cf6ffb-zw569]",
        },
    }
to be nil
/home/jenkins/workspace/Cilium-PR-K8s-1.24-kernel-4.19/src/github.com/cilium/cilium/vendor/github.com/onsi/ginkgo/internal/leafnodes/runner.go:64

Standard Output

Click to show.

Standard Error

Click to show.

18:37:54 STEP: Running BeforeAll block for EntireTestsuite K8sDatapathVerifier
18:37:54 STEP: Ensuring the namespace kube-system exists
18:37:54 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs")
18:37:54 STEP: WaitforPods(namespace="kube-system", filter="-l k8s-app=cilium-test-logs") => <nil>
18:37:54 STEP: Deleting cep [echo-bc5cf6ffb-gsnbm,echo-bc5cf6ffb-zw569] in namespace default
18:37:54 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-gsnbm,echo-bc5cf6ffb-zw569)
18:37:54 STEP: Deleting pods [echo-bc5cf6ffb-gsnbm,echo-bc5cf6ffb-zw569] in namespace default
18:37:54 STEP: Waiting for 2 deletes to return (echo-bc5cf6ffb-gsnbm,echo-bc5cf6ffb-zw569)
18:38:04 STEP: Unable to delete pods echo-bc5cf6ffb-zw569 with 'kubectl -n default delete pods echo-bc5cf6ffb-zw569': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-zw569" deleted
	 
Stderr:
 	 

18:38:04 STEP: Unable to delete pods echo-bc5cf6ffb-gsnbm with 'kubectl -n default delete pods echo-bc5cf6ffb-gsnbm': Exitcode: -1 
Err: signal: killed
Stdout:
 	 pod "echo-bc5cf6ffb-gsnbm" deleted
	 
Stderr:
 	 

18:38:04 STEP: WaitForSinglePod(namespace="default", podname="test-verifier")
18:38:16 STEP: waitForSinglePod(namespace="default", podname="test-verifier", minRequired=1) => <nil>
18:38:30 STEP: Cleaning bpf_lxc build files
18:38:31 STEP: Building bpf_lxc object file
18:38:34 STEP: Running the verifier test script with bpf_lxc
18:38:35 STEP: Cleaning bpf_lxc build files
18:38:35 STEP: Building bpf_lxc object file
18:38:36 STEP: Running the verifier test script with bpf_lxc
18:38:37 STEP: Cleaning bpf_lxc build files
18:38:37 STEP: Building bpf_lxc object file
18:38:38 STEP: Running the verifier test script with bpf_lxc
18:38:39 STEP: Cleaning bpf_host build files
18:38:39 STEP: Building bpf_host object file
18:38:43 STEP: Running the verifier test script with bpf_host
18:38:45 STEP: Cleaning bpf_host build files
18:38:45 STEP: Building bpf_host object file
18:38:47 STEP: Running the verifier test script with bpf_host
18:38:49 STEP: Cleaning bpf_host build files
18:38:49 STEP: Building bpf_host object file
18:38:51 STEP: Running the verifier test script with bpf_host
18:38:52 STEP: Cleaning bpf_xdp build files
18:38:52 STEP: Building bpf_xdp object file
18:38:54 STEP: Running the verifier test script with bpf_xdp
18:38:55 STEP: Cleaning bpf_overlay build files
18:38:55 STEP: Building bpf_overlay object file
18:38:58 STEP: Running the verifier test script with bpf_overlay
18:38:58 STEP: Cleaning bpf_sock build files
18:38:58 STEP: Building bpf_sock object file
18:38:59 STEP: Running the verifier test script with bpf_sock
=== Test Finished at 2023-05-16T18:38:59Z====
18:38:59 STEP: Running AfterEach for block EntireTestsuite

18:38:59 STEP: Running AfterAll block for EntireTestsuite K8sDatapathVerifier
FAIL: terminating containers are not deleted after timeout
Expected
    <*fmt.wrapError | 0xc000b0bda0>: {
        msg: "Pods are still not deleted after a timeout: 4m0s timeout expired: Pods are still terminating: [echo-bc5cf6ffb-gsnbm echo-bc5cf6ffb-zw569]",
        err: <*errors.errorString | 0xc00027f750>{
            s: "Pods are still terminating: [echo-bc5cf6ffb-gsnbm echo-bc5cf6ffb-zw569]",
        },
    }
to be nil

ZIP Links:

Click to show.

https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19//60/artifact/test_results_Cilium-PR-K8s-1.24-kernel-4.19_60_BDD-Test-PR.zip

Jenkins URL: https://jenkins.cilium.io/job/Cilium-PR-K8s-1.24-kernel-4.19/60/

If this is a duplicate of an existing flake, comment 'Duplicate of #<issue-number>' and close this issue.

[ upstream commit 6878133 ] In the K8sDatapathConfig tests "echo-svc" deployment Pods are failing to terminate while waiting to terminate all Pods. These come from a deployment, that shouldn't be applied in this suite. Presumably failure to delete is being cause by the Deployment controller restarting the Pods as they're deleted. To try to fix this, going to wait for applied yamls in K8sServices to be fully deleted, including finalizers. Addresses: #25255 Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com> Signed-off-by: Gilberto Bertin <jibi@cilium.io>

Due to this flake: #25255 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

Quarantine K8sDatapathVerifier until #25255 is fixed. Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

[ upstream commit 6878133 ] In the K8sDatapathConfig tests "echo-svc" deployment Pods are failing to terminate while waiting to terminate all Pods. These come from a deployment, that shouldn't be applied in this suite. Presumably failure to delete is being cause by the Deployment controller restarting the Pods as they're deleted. To try to fix this, going to wait for applied yamls in K8sServices to be fully deleted, including finalizers. Addresses: #25255 Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com> Signed-off-by: Gilberto Bertin <jibi@cilium.io>

tommyp1ckles · 2023-05-30T21:58:53Z

I don't have time to work on this at the moment.

github-actions · 2023-07-30T01:50:31Z

This issue has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.

github-actions · 2023-08-14T01:45:00Z

This issue has not seen any activity since it was marked stale.
Closing.

[ upstream commit 6878133 ] In the K8sDatapathConfig tests "echo-svc" deployment Pods are failing to terminate while waiting to terminate all Pods. These come from a deployment, that shouldn't be applied in this suite. Presumably failure to delete is being cause by the Deployment controller restarting the Pods as they're deleted. To try to fix this, going to wait for applied yamls in K8sServices to be fully deleted, including finalizers. Addresses: #25255 Signed-off-by: Tom Hadlaw <tom.hadlaw@isovalent.com> Signed-off-by: Gilberto Bertin <jibi@cilium.io>

maintainer-s-little-helper bot added the ci/flake This is a known failure that occurs in the tree. Please investigate me! label May 4, 2023

maintainer-s-little-helper bot mentioned this issue May 4, 2023

identities: fix incorrectly overwriting the labels of kube-apiserver IPs #25241

Merged

maintainer-s-little-helper bot mentioned this issue May 5, 2023

chore(deps): update hubble cli to v0.11.5 (v1.13) (patch) #25125

Merged

1 task

michi-covalent added a commit that referenced this issue May 8, 2023

Quarantine K8sDatapathVerifier test

c7a39be

Due to this flake: #25255 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

michi-covalent mentioned this issue May 8, 2023

Quarantine K8sDatapathVerifier test #25318

Closed

michi-covalent assigned tommyp1ckles May 8, 2023

tommyp1ckles mentioned this issue May 9, 2023

test/k8s: for services test, wait for all applied manifests to delete #25341

Merged

maintainer-s-little-helper bot mentioned this issue May 10, 2023

v1.13 Backports 2023-05-10 #25346

Merged

14 tasks

michi-covalent added a commit that referenced this issue May 12, 2023

Quarantine K8sDatapathVerifier

cbf8865

Due to this flake: #25255 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

maintainer-s-little-helper bot mentioned this issue May 12, 2023

backport (v1.13): policy: Fix Deny Precedence Bug #25427

Merged

maintainer-s-little-helper bot mentioned this issue May 13, 2023

v1.13 Backports 2023-05-12 #25409

Merged

2 tasks

maintainer-s-little-helper bot mentioned this issue May 15, 2023

[v1.13] Update CNI to 1.3.0 #25454

Merged

1 task

maintainer-s-little-helper bot mentioned this issue May 16, 2023

v1.13 backports 2023-05-12 #25424

Merged

1 task

maintainer-s-little-helper bot mentioned this issue May 16, 2023

chore(deps): update docker.io/library/golang:1.19.8 docker digest to 9f2dd04 (v1.13) #25421

Merged

1 task

michi-covalent added a commit that referenced this issue May 18, 2023

Quarantine K8sDatapathVerifier

4d38b8a

Due to this flake: #25255 Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

michi-covalent added a commit that referenced this issue May 18, 2023

Quarantine K8sDatapathVerifier

9fb6710

Quarantine K8sDatapathVerifier until #25255 is fixed. Signed-off-by: Michi Mutsuzaki <michi@isovalent.com>

michi-covalent mentioned this issue May 18, 2023

Quarantine K8sDatapathVerifier #25536

Closed

tommyp1ckles removed their assignment May 30, 2023

github-actions bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Jul 30, 2023

github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Aug 14, 2023

CI: K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath #25255

CI: K8sDatapathVerifier Runs the kernel verifier against Cilium's BPF datapath #25255

Comments

maintainer-s-little-helper bot commented May 4, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 5, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 10, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 10, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 10, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 10, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 11, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 12, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 12, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 12, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 12, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 12, 2023

Test Name

Failure Output

Stacktrace

Standard Output

Standard Error

maintainer-s-little-helper bot commented May 13, 2023

Test Name

Failure Output

Stacktrace

Standard Output