New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[e2e failure] "should function for client IP based session affinity" constantly fails #53741

Closed
MrHohn opened this Issue Oct 11, 2017 · 6 comments

Comments

Projects
None yet
3 participants
@MrHohn
Member

MrHohn commented Oct 11, 2017

Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug
/priority failing-test
/sig network

What happened:
From https://k8s-testgrid.appspot.com/google-gce#gci-gce-slow&sort-by-failures=&width=5,
Services should function for client IP based session affinity: udp/http seems to be broken right after they were added by #50447.

Example error message:

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:225
Oct 11 16:39:42.406: Expect endpoints: map[], got: map[netserver-1:{}]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:222

and

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:249
Oct 11 16:37:49.106: test session affinity, cost time: 1m31.742990485s
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:248

Anything else we need to know?:
@m1093782566 Is it as expected?

@m1093782566

This comment has been minimized.

Show comment
Hide comment
@m1093782566

m1093782566 Oct 12, 2017

Member

@MrHohn

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:249
Oct 11 16:37:49.106: test session affinity, cost time: 1m31.742990485s
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:248

This is a mistake. Sorry for my debug lines sneak in. 1fd653f

I was just wondering why this test case doesn't run in upstream CI, so I added this debug line. Now, I see because this test case only runs in ci-kubernetes-e2e-gci-gce-slow.

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:225
Oct 11 16:39:42.406: Expect endpoints: map[], got: map[netserver-1:{}]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:222

This check failed unexpectedly, let me try to find out why. I am working on it and will let you know soon when I have some updates.

Member

m1093782566 commented Oct 12, 2017

@MrHohn

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:249
Oct 11 16:37:49.106: test session affinity, cost time: 1m31.742990485s
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:248

This is a mistake. Sorry for my debug lines sneak in. 1fd653f

I was just wondering why this test case doesn't run in upstream CI, so I added this debug line. Now, I see because this test case only runs in ci-kubernetes-e2e-gci-gce-slow.

/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:225
Oct 11 16:39:42.406: Expect endpoints: map[], got: map[netserver-1:{}]
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/network/networking.go:222

This check failed unexpectedly, let me try to find out why. I am working on it and will let you know soon when I have some updates.

@MrHohn

This comment has been minimized.

Show comment
Hide comment
@MrHohn

MrHohn Oct 12, 2017

Member

@m1093782566 Thanks. These two tests don't run on pull jobs as they are marked as [slow], though https://k8s-testgrid.appspot.com/google-gce#gci-gce-slow, https://k8s-testgrid.appspot.com/google-gce#gci-gce-alpha-features (and other CIs that include [slow]) would have them running.

Member

MrHohn commented Oct 12, 2017

@m1093782566 Thanks. These two tests don't run on pull jobs as they are marked as [slow], though https://k8s-testgrid.appspot.com/google-gce#gci-gce-slow, https://k8s-testgrid.appspot.com/google-gce#gci-gce-alpha-features (and other CIs that include [slow]) would have them running.

@m1093782566

This comment has been minimized.

Show comment
Hide comment
@m1093782566

m1093782566 Oct 12, 2017

Member

These two tests don't run on pull jobs as they are marked as [slow]

Do you mean when I raise a PR, these test cases will not be triggered? My question is how can I make sure my follow-up PR can fix this bug? Where can I access the log message when I raise a new PR?

Member

m1093782566 commented Oct 12, 2017

These two tests don't run on pull jobs as they are marked as [slow]

Do you mean when I raise a PR, these test cases will not be triggered? My question is how can I make sure my follow-up PR can fix this bug? Where can I access the log message when I raise a new PR?

@m1093782566

This comment has been minimized.

Show comment
Hide comment
Member

m1093782566 commented Oct 12, 2017

@MrHohn

This comment has been minimized.

Show comment
Hide comment
@MrHohn

MrHohn Oct 12, 2017

Member

Do you mean when I raise a PR, these test cases will not be triggered? My question is how can I make sure my follow-up PR can fix this bug? Where can I access the log message when I raise a new PR?

Yep, that is the current behavior. A hacky way to make it run on PR --- add a temporary commit to remove the word [Slow] from test name so some pull jobs (pull-kubernetes-e2e-gce-etcd3 I think?) will run it. And you could view logs by clicking the Details button.

Member

MrHohn commented Oct 12, 2017

Do you mean when I raise a PR, these test cases will not be triggered? My question is how can I make sure my follow-up PR can fix this bug? Where can I access the log message when I raise a new PR?

Yep, that is the current behavior. A hacky way to make it run on PR --- add a temporary commit to remove the word [Slow] from test name so some pull jobs (pull-kubernetes-e2e-gce-etcd3 I think?) will run it. And you could view logs by clicking the Details button.

@MrHohn

This comment has been minimized.

Show comment
Hide comment
@MrHohn

MrHohn Oct 12, 2017

Member

Can I see the updated test results via directly visiting https://k8s-testgrid.appspot.com/google-gce#gci-gce-slow, https://k8s-testgrid.appspot.com/google-gce#gci-gce-alpha-features?

Nope, this CI only includes changes that are merged. It is basically a cronjob that runs tests against head.

Member

MrHohn commented Oct 12, 2017

Can I see the updated test results via directly visiting https://k8s-testgrid.appspot.com/google-gce#gci-gce-slow, https://k8s-testgrid.appspot.com/google-gce#gci-gce-alpha-features?

Nope, this CI only includes changes that are merged. It is basically a cronjob that runs tests against head.

k8s-merge-robot added a commit that referenced this issue Oct 26, 2017

Merge pull request #53760 from m1093782566/session-ci
Automatic merge from submit-queue (batch tested with PRs 53760, 48996, 51267, 54414). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Fix CI error for service session affinity

**What this PR does / why we need it**:

Fix CI error for service session affinity -- debug

**Which issue this PR fixes**: 

fixes #53741

**Special notes for your reviewer**:

I remove the [slow] tag so that these test cases can be run in PR request. We may need to add back the [slow] tag when this PR is ready to get in.

**Release note**:

```release-note
NONE
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment