ci-kubernetes-e2e-gci-gke-alpha-features: broken test run #38020
Assignees
Labels
area/test-infra
kind/flake
Categorizes issue or PR as related to a flaky test.
priority/backlog
Higher priority than priority/awaiting-more-evidence.
sig/network
Categorizes an issue or PR as relevant to SIG Network.
Milestone
Comments
k8s-github-robot
added
kind/flake
|
Run so broken it didn't make JUnit output! |
|
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gke-alpha-features/5961/ Failed: [k8s.io] ESIPP [Slow] should work from pods [Feature:ExternalTrafficLocalOnly] {Kubernetes e2e suite} Failed: Test {e2e.go} Issues about this test specifically: #33361 #38663 #39788 #39877 #40371 #40469 #40478 #40483 #40668 #41048 Failed: DiffResources {e2e.go} Issues about this test specifically: #33373 #33416 #34060 #40437 #40454 #42510 Failed: list nodes {e2e.go} Issues about this test specifically: #38667 |
|
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gke-alpha-features/5970/ Failed: [k8s.io] ESIPP [Slow] should only target nodes with endpoints [Feature:ExternalTrafficLocalOnly] {Kubernetes e2e suite} Issues about this test specifically: #38819 Failed: [k8s.io] ESIPP [Slow] should work from pods [Feature:ExternalTrafficLocalOnly] {Kubernetes e2e suite} Failed: Test {e2e.go} Issues about this test specifically: #33361 #38663 #39788 #39877 #40371 #40469 #40478 #40483 #40668 #41048 Failed: DiffResources {e2e.go} Issues about this test specifically: #33373 #33416 #34060 #40437 #40454 #42510 |
|
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gke-alpha-features/5990/ Failed: [k8s.io] ESIPP [Slow] should work from pods [Feature:ExternalTrafficLocalOnly] {Kubernetes e2e suite} Failed: [k8s.io] Kubectl alpha client [k8s.io] Kubectl run ScheduledJob should create a ScheduledJob {Kubernetes e2e suite} Failed: Test {e2e.go} Issues about this test specifically: #33361 #38663 #39788 #39877 #40371 #40469 #40478 #40483 #40668 #41048 Failed: DiffResources {e2e.go} Issues about this test specifically: #33373 #33416 #34060 #40437 #40454 #42510 |
grodrigues3
added
the
sig/network
|
https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gke-alpha-features/6234/ Failed: DiffResources {e2e.go} Issues about this test specifically: #33373 #33416 #34060 #40437 #40454 #42510 Failed: [k8s.io] ESIPP [Slow] should work from pods [Feature:ExternalTrafficLocalOnly] {Kubernetes e2e suite} Failed: [k8s.io] ESIPP [Slow] should handle updates to source ip annotation [Feature:ExternalTrafficLocalOnly] {Kubernetes e2e suite} Issues about this test specifically: #38448 Failed: Test {e2e.go} Issues about this test specifically: #33361 #38663 #39788 #39877 #40371 #40469 #40478 #40483 #40668 #41048 |
|
Took a look at this, ci-kubernetes-e2e-gke-alpha-features-release-1.4 failed roughly 100% due the leaked healthCheck resource (and hit quotaExceeded): ci-kubernetes-e2e-gke-alpha-features has similar issue, as that project already has 39 healthCheck resource leaked. I manually deleted the leaked healthChecks and would like to see how it goes. And this does not block v1.6. @krzyzacy Is the GCP resource cleanup mechanism not working here? |
|
seems that's not part of the list of resources yet, I'll add health-check there. |
|
@MrHohn I saw health-check, http-health-check, https-health-check, which ones are we using? Or all of them? |
|
Should be |
|
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a temporary error. The following address(es) deferred:
curtis.l.bates@gmail.com
Domain imwiz.com has exceeded the max emails per hour (157/150 (104%)) allowed. Message will be reattempted later
------- This is a copy of the message, including all the headers. ------
Received: from github-smtp2-ext6.iad.github.net ([192.30.252.197]:50992 helo=github-smtp2b-ext-cp1-prd.iad.github.net)
by box969.bluehost.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256)
(Exim 4.87)
(envelope-from <noreply@github.com>)
id 1cnrXP-0000je-0o
for dev@imwiz.com; Tue, 14 Mar 2017 12:51:20 -0600
Date: Tue, 14 Mar 2017 11:40:55 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=github.com;
s=pf2014; t=1489516855;
bh=4hQMpgDfuYI2P/SvD6nHcV+mHulWXXuOdGVKdfCITwM=;
h=From:Reply-To:To:Cc:In-Reply-To:References:Subject:List-ID:
List-Archive:List-Post:List-Unsubscribe:From;
b=mR6wVMqBEgpURvcB8pk+FU2QhsSa+OnYhq3OC2ys4kide07AelIPKM9l4oSEHqzj1
ZdkhbuF2WlEeUeyUlWoAqQU7LqKScXXd8jmFeiOInlbMKtG3ShNdpD8H+MunTytxPW
mEW9Nzfd+NINwx+VjTjlkhiJ+syo/gkw5//gghlA=
From: Zihong Zheng <notifications@github.com>
Reply-To: kubernetes/kubernetes <reply@reply.github.com>
To: kubernetes/kubernetes <kubernetes@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Message-ID: <kubernetes/kubernetes/issues/38020/286520388@github.com>
In-Reply-To: <kubernetes/kubernetes/issues/38020@github.com>
References: <kubernetes/kubernetes/issues/38020@github.com>
Subject: Re: [kubernetes/kubernetes] ci-kubernetes-e2e-gci-gke-alpha-features:
broken test run (#38020)
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="--==_mimepart_58c83937b25ac_e843f88acfcbc2c161523";
charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
X-GitHub-Sender: MrHohn
X-GitHub-Recipient: falenn
X-GitHub-Reason: subscribed
List-ID: kubernetes/kubernetes <kubernetes.kubernetes.github.com>
List-Archive: https://github.com/kubernetes/kubernetes
List-Post: <mailto:reply@reply.github.com>
List-Unsubscribe: <mailto:unsub+000ab60a26bbb7103a802e5f169cab2daac3ec9508c29ead92cf0000000114dffb3792a169ce0b84e4ed@reply.github.com>,
<https://github.com/notifications/unsubscribe/AAq2CmNs92Zzq3kifc35aw3Dhpqx5tI-ks5rlt83gaJpZM4LDKn5>
X-Auto-Response-Suppress: All
X-GitHub-Recipient-Address: dev@imwiz.com
X-Spam-Status: No, score=-1.6
X-Spam-Score: -15
X-Spam-Bar: -
X-Ham-Report: Spam detection software, running on the system "box969.bluehost.com",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
root\@localhost for details.
Content preview: Took a look at this, ci-kubernetes-e2e-gke-alpha-features-release-1.4
failed roughly 100% due the leaked healthCheck resource (and hit quotaExceeded):
``` Failed to ensure health check for localized service a9e504c6408ca11e7a13a42010af0000
on node port 32673: googleapi: Error 403: Quota 'HEALTH_CHECKS' exceeded.
Limit: 50.0, quotaExceeded ``` ci-kubernetes-e2e-gke-alpha-features has similar
issue, as that project already has 39 healthCheck resource leaked. I manually
deleted the leaked healthChecks and would like to see how it goes. [...]
Content analysis details: (-1.6 points, 4.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_PASS SPF: sender matches SPF record
0.0 HTML_MESSAGE BODY: HTML included in message
0.0 HTML_IMAGE_ONLY_32 BODY: HTML: images with 2800-3200 bytes of words
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
domain
-2.8 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2)
[192.30.252.197 listed in wl.mailspike.net]
1.2 AWL AWL: Adjusted score from AWL reputation of From: address
X-Spam-Flag: NO
----==_mimepart_58c83937b25ac_e843f88acfcbc2c161523
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Took a look at this, ci-kubernetes-e2e-gke-alpha-features-release-1.4 failed roughly 100% due the leaked healthCheck resource (and hit quotaExceeded):
```
Failed to ensure health check for localized service a9e504c6408ca11e7a13a42010af0000 on node port 32673: googleapi: Error 403: Quota 'HEALTH_CHECKS' exceeded. Limit: 50.0, quotaExceeded
```
ci-kubernetes-e2e-gke-alpha-features has similar issue, as that project already has 39 healthCheck resource leaked. I manually deleted the leaked healthChecks and would like to see how it goes.
And this does not block v1.6.
@krzyzacy Is the GCP resource cleanup mechanism not working here?
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#38020 (comment)
----==_mimepart_58c83937b25ac_e843f88acfcbc2c161523
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<p>Took a look at this, ci-kubernetes-e2e-gke-alpha-features-release-1.4 =
failed roughly 100% due the leaked healthCheck resource (and hit quotaExc=
eeded):</p>
<pre><code>Failed to ensure health check for localized service a9e504c640=
8ca11e7a13a42010af0000 on node port 32673: googleapi: Error 403: Quota 'H=
EALTH_CHECKS' exceeded. Limit: 50.0, quotaExceeded
</code></pre>
<p>ci-kubernetes-e2e-gke-alpha-features has similar issue, as that projec=
t already has 39 healthCheck resource leaked. I manually deleted the leak=
ed healthChecks and would like to see how it goes.</p>
<p>And this does not block v1.6.</p>
<p><a href=3D"https://github.com/krzyzacy" class=3D"user-mention">@Krzyza=
cy</a> Is the GCP resource cleanup mechanism not working here?</p>
<p style=3D"font-size:small;-webkit-text-size-adjust:none;color:#666;">&m=
dash;<br />You are receiving this because you are subscribed to this thre=
ad.<br />Reply to this email directly, <a href=3D"https://github.com/kube=
rnetes/kubernetes/issues/38020#issuecomment-286520388">view it on GitHub<=
/a>, or <a href=3D"https://github.com/notifications/unsubscribe-auth/AAq2=
Cg16qEmpI2etOQbtViepUXHKSS6Xks5rlt83gaJpZM4LDKn5">mute the thread</a>.<im=
g alt=3D"" height=3D"1" src=3D"https://github.com/notifications/beacon/AA=
q2CmL_9_A4DLLADOyf8dpZKvnwIV6oks5rlt83gaJpZM4LDKn5.gif" width=3D"1" /></p=
<div itemscope itemtype=3D"http://schema.org/EmailMessage">
<div itemprop=3D"action" itemscope itemtype=3D"http://schema.org/ViewActi=
on">
<link itemprop=3D"url" href=3D"https://github.com/kubernetes/kubernetes=
/issues/38020#issuecomment-286520388"></link>
<meta itemprop=3D"name" content=3D"View Issue"></meta>
</div>
<meta itemprop=3D"description" content=3D"View this Issue on GitHub"></me=
ta>
</div>
<script type=3D"application/json" data-scope=3D"inboxmarkup">{"api_versio=
n":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name"=
:"GitHub"},"entity":{"external_key":"github/kubernetes/kubernetes","title=
":"kubernetes/kubernetes","subtitle":"GitHub repository","main_image_url"=
:"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d8=
8-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubuse=
rcontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.=
png","action":{"name":"Open in GitHub","url":"https://github.com/kubernet=
es/kubernetes"}},"updates":{"snippets":[{"icon":"PERSON","message":"@Mrho=
hn in #38020: Took a look at this, ci-kubernetes-e2e-gke-alpha-features-r=
elease-1.4 failed roughly 100% due the leaked healthCheck resource (and h=
it quotaExceeded):\r\n```\r\nFailed to ensure health check for localized =
service a9e504c6408ca11e7a13a42010af0000 on node port 32673: googleapi: E=
rror 403: Quota 'HEALTH_CHECKS' exceeded. Limit: 50.0, quotaExceeded\r\n`=
``\r\nci-kubernetes-e2e-gke-alpha-features has similar issue, as that pro=
ject already has 39 healthCheck resource leaked. I manually deleted the l=
eaked healthChecks and would like to see how it goes.\r\n\r\nAnd this doe=
s not block v1.6.\r\n\r\n@krzyzacy Is the GCP resource cleanup mechanism =
not working here?"}],"action":{"name":"View Issue","url":"https://github.=
com/kubernetes/kubernetes/issues/38020#issuecomment-286520388"}}}</script=
…=
----==_mimepart_58c83937b25ac_e843f88acfcbc2c161523--
|
lol |
|
This message was created automatically by mail delivery software.
A message that you sent could not be delivered to one or more of its
recipients. This is a temporary error. The following address(es) deferred:
curtis.l.bates@gmail.com
Domain imwiz.com has exceeded the max emails per hour (176/150 (117%)) allowed. Message will be reattempted later
…------- This is a copy of the message, including all the headers. ------
Received: from o7.sgmail.github.com ([167.89.101.198]:14224)
by box969.bluehost.com with esmtps (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128)
(Exim 4.87)
(envelope-from <bounces+848413-5c7e-dev=imwiz.com@sgmail.github.com>)
id 1cnrdB-0005ev-1x
for dev@imwiz.com; Tue, 14 Mar 2017 12:57:18 -0600
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=github.com;
h=from:reply-to:to:cc:in-reply-to:references:subject:mime-version:content-type:content-transfer-encoding:list-id:list-archive:list-post:list-unsubscribe;
s=s20150108; bh=LfpdaXm6EuJRCVTqc+Tx2YK1DNE=; b=R/FX2AdITCenUyb8
cEEbapRthtNe41d16zSACrTD7RFGJ0mrMtQwusHJ/nWxVzJ/hUEazuAIvM+9smui
m6jAamxstkplOicpOZLxaj1w8f2Mn7uIyhzj/l4ifvAahnijNvxziqgt+ctM1lQ0
PZgkLIWUzFG0/MYbh00tfNlNW88=
Received: by filter0443p1mdw1.sendgrid.net with SMTP id filter0443p1mdw1-26925-58C83CAF-4B
2017-03-14 18:55:43.542050759 +0000 UTC
Received: from github-smtp2b-ext-cp1-prd.iad.github.net (github-smtp2b-ext-cp1-prd.iad.github.net [192.30.253.17])
by ismtpd0006p1iad1.sendgrid.net (SG) with ESMTP id 1ti1UCdeQrqdaugS6HwRJw
for <dev@imwiz.com>; Tue, 14 Mar 2017 18:55:43.508 +0000 (UTC)
Date: Tue, 14 Mar 2017 11:55:43 -0700
From: Zihong Zheng <notifications@github.com>
Reply-To: kubernetes/kubernetes <reply@reply.github.com>
To: kubernetes/kubernetes <kubernetes@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Message-ID: <kubernetes/kubernetes/issues/38020/286524810@github.com>
In-Reply-To: <kubernetes/kubernetes/issues/38020@github.com>
References: <kubernetes/kubernetes/issues/38020@github.com>
Subject: Re: [kubernetes/kubernetes] ci-kubernetes-e2e-gci-gke-alpha-features:
broken test run (#38020)
Mime-Version: 1.0
Content-Type: multipart/alternative;
boundary="--==_mimepart_58c83caf60dce_5d0a3ff430137c3497413";
charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: list
X-GitHub-Sender: MrHohn
X-GitHub-Recipient: falenn
X-GitHub-Reason: subscribed
List-ID: kubernetes/kubernetes <kubernetes.kubernetes.github.com>
List-Archive: https://github.com/kubernetes/kubernetes
List-Post: <mailto:reply@reply.github.com>
List-Unsubscribe: <mailto:unsub+000ab60aba62d3698779e35e1ba56e23e2e53b71cc63666992cf0000000114dffeaf92a169ce0b84e4ed@reply.github.com>,
<https://github.com/notifications/unsubscribe/AAq2Cj0REEK-N56NaNgveF8uV3sKlzG6ks5rluKvgaJpZM4LDKn5>
X-Auto-Response-Suppress: All
X-GitHub-Recipient-Address: dev@imwiz.com
X-SG-EID: APO41b8ovafPb3SK9rw3vN9M8f950ywLVs1eKTDzrbjfBNzLmFmW7f9kxxB0HSoN12ffd+nxbfJhZL
4sGH7Y+4Q9KTDJNbf+jOjTXRUxe4CP0DVmj6Fc60e1n96bem99aPrtf6FvHlDnA+6DEMr/oA5iEnJd
vo1cRbEULqREZzztoiXsCqCk9pMd/6Wmj28mkOEM6lPECYd6OmhJx2vgSQ==
X-Spam-Status: No, score=0.4
X-Spam-Score: 4
X-Spam-Bar: /
X-Ham-Report: Spam detection software, running on the system "box969.bluehost.com",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
root\@localhost for details.
Content preview: Should be `health-check` ([doc](https://cloud.google.com/compute/docs/load-balancing/health-checks))?
Seems like the other two are legacy. -- You are receiving this because you
are subscribed to this thread. Reply to this email directly or view it on
GitHub: #38020 (comment)
[...]
Content analysis details: (0.4 points, 4.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 SPF_PASS SPF: sender matches SPF record
-0.0 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain
1.3 HTML_IMAGE_ONLY_24 BODY: HTML: images with 2000-2400 bytes of words
0.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's
domain
-0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4)
[167.89.101.198 listed in wl.mailspike.net]
-0.0 RCVD_IN_MSPIKE_WL Mailspike good senders
-0.8 AWL AWL: Adjusted score from AWL reputation of From: address
X-Spam-Flag: NO
----==_mimepart_58c83caf60dce_5d0a3ff430137c3497413
Content-Type: text/plain;
charset=UTF-8
Content-Transfer-Encoding: 7bit
Should be `health-check` ([doc](https://cloud.google.com/compute/docs/load-balancing/health-checks))? Seems like the other two are legacy.
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
#38020 (comment)
----==_mimepart_58c83caf60dce_5d0a3ff430137c3497413
Content-Type: text/html;
charset=UTF-8
Content-Transfer-Encoding: 7bit
<p>Should be <code>health-check</code> (<a href="https://cloud.google.com/compute/docs/load-balancing/health-checks">doc</a>)? Seems like the other two are legacy.</p>
<p style="font-size:small;-webkit-text-size-adjust:none;color:#666;">—<br />You are receiving this because you are subscribed to this thread.<br />Reply to this email directly, <a href="#38020 (comment)">view it on GitHub</a>, or <a href="https://github.com/notifications/unsubscribe-auth/AAq2CiCHq3i1WDYHLE9z16oYW6sWHVpgks5rluKvgaJpZM4LDKn5">mute the thread</a>.<img alt="" height="1" src="https://github.com/notifications/beacon/AAq2Coac2DiLBsF6SFDIDzHdjAK_0xVMks5rluKvgaJpZM4LDKn5.gif" width="1" /></p>
<div itemscope itemtype="http://schema.org/EmailMessage">
<div itemprop="action" itemscope itemtype="http://schema.org/ViewAction">
<link itemprop="url" href="#38020 (comment)"></link>
<meta itemprop="name" content="View Issue"></meta>
</div>
<meta itemprop="description" content="View this Issue on GitHub"></meta>
</div>
<script type="application/json" data-scope="inboxmarkup">{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/kubernetes/kubernetes","title":"kubernetes/kubernetes","subtitle":"GitHub repository","main_image_url":"https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open in GitHub","url":"https://github.com/kubernetes/kubernetes"}},"updates":{"snippets":[{"icon":"PERSON","message":"@MrHohn in #38020: Should be `health-check` ([doc](https://cloud.google.com/compute/docs/load-balancing/health-checks))? Seems like the other two are legacy."}],"action":{"name":"View Issue","url":"#38020 (comment)"}}}</script>
----==_mimepart_58c83caf60dce_5d0a3ff430137c3497413--
|
|
that's the reason you don't want to subscribe for everything in k/k 😈 |
|
/assign |
|
Suite seems relatively stable so moving to v1.7 |
![https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open](https://cloud.githubusercontent.com/assets/143418/17495839/a5054eac-5d88-11e6-95fc-7290892c7bb5.png","avatar_image_url":"https://cloud.githubusercontent.com/assets/143418/15842166/7c72db34-2c0b-11e6-9aed-b52498112777.png","action":{"name":"Open){kind=link}
|
/close close via kubernetes/test-infra#2245 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Failed: https://k8s-gubernator.appspot.com/build/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gke-alpha-features/689/
Run so broken it didn't make JUnit output!
The text was updated successfully, but these errors were encountered: