New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kubetest: A Few UX bugs improvements around build reqiurements, setting apiserver IP addres, skeleton/local mode guidance. #73859

Open
jayunit100 opened this Issue Feb 8, 2019 · 3 comments

Comments

Projects
None yet
3 participants
@jayunit100
Copy link
Member

jayunit100 commented Feb 8, 2019

Testing on master now, this from the 1.11.3 tag, so these bugs may be obsolete but in general from conversation in sig-testing, some ux improvments around help for baremetal clusters and more verbose error logging were discussed so maybe fold those into these UX improvements.

What happened:

Kubetest against a generic cluster seems to not work using basic options.

What you expected to happen:

E2E tests would run against the default cluster in my kubeconfig file.

How to reproduce it (as minimally and precisely as possible):

  1. With local mode:
➜  kubernetes git:(1bfeeb6f21) ✗ kubetest --test --kubeconfig=/Users/jayv/.kube/config --provider=local
  1. Results in a failure but doesnt complain that the kubeconfig file doesnt exist.
  2. Running this with the right kubeconfig file also doesnt work... so looks like maybe its just not getting used at all.
2019/02/08 11:16:38 process.go:153: Running: ./hack/e2e-internal/e2e-status.sh
Local doesn't need special preparations for e2e tests
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.11", GitCommit:"1bfeeb6f212135a22dc787b73e1980e5bccef13d", GitTreeState:"clean", BuildDate:"2019-02-08T16:51:58Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}                        
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}                          
2019/02/08 11:16:39 process.go:155: Step './hack/e2e-internal/e2e-status.sh' finished in 1.079404546s
2019/02/08 11:16:39 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version
2019/02/08 11:16:40 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 763.71723ms
2019/02/08 11:16:40 process.go:153: Running: ./hack/ginkgo-e2e.sh
Setting up for KUBERNETES_PROVIDER="local".
Local doesn't need special preparations for e2e tests
Feb  8 11:16:40.616: INFO: Overriding default scale value of zero to 1
Feb  8 11:16:40.617: INFO: Overriding default milliseconds value of zero to 5000
I0208 11:16:40.719868   18600 e2e.go:331] Starting e2e run "1024670e-2bd6-11e9-b082-8c8590b99efa" on Ginkgo node 1
Running Suite: Kubernetes e2e suite
===================================
Random Seed: 1549653400 - Will randomize all specs
Will run 610 of 853 specs

Feb  8 11:16:40.740: INFO: >>> kubeConfig: /Users/jayv/.kube/config
Feb  8 11:16:40.747: INFO: Waiting up to 4h0m0s for all (but 0) nodes to be schedulable
Feb  8 11:16:40.749: INFO: Unexpected error listing nodes: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse&resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused                                                      
Feb  8 11:16:40.750: INFO: Unexpected error occurred: Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse&resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused                                                           
Failure [0.011 seconds]
[BeforeSuite] BeforeSuite 
/go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/test/e2e/e2e.go:138

  Expected error:
      <*url.Error | 0xc420d0a030>: {
          Op: "Get",
          URL: "http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse&resourceVersion=0",
          Err: {
              Op: "dial",
              Net: "tcp",
              Source: nil,
              Addr: {
                  IP: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 255, 255, 127, 0, 0, 1],
                  Port: 8080,
                  Zone: "",
              },
              Err: {
                  Syscall: "getsockopt",
                  Err: 0x3d,
              },
          },
      }
      Get http://127.0.0.1:8080/api/v1/nodes?fieldSelector=spec.unschedulable%3Dfalse&resourceVersion=0: dial tcp 127.0.0.1:8080: getsockopt: connection refused                                                                                                           
  not to have occurred

  1. Similar, with skeleton mode:
➜  kubernetes git:(1bfeeb6f21) ✗ kubetest --test --kubeconfig=/Users/jayv/.kube/config --provider=skeleton
2019/02/08 11:35:23 process.go:153: Running: ./hack/e2e-internal/e2e-status.sh
Skeleton Provider: prepare-e2e not implemented
Client Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.11", GitCommit:"1bfeeb6f212135a22dc787b73e1980e5bccef13d", GitTreeState:"clean", BuildDate:"2019-02-08T16:51:58Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"9", GitVersion:"v1.9.6", GitCommit:"9f8ebd171479bec0ada837d7ee641dec2f8c6dd1", GitTreeState:"clean", BuildDate:"2018-03-21T15:13:31Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"linux/amd64"}
2019/02/08 11:35:23 process.go:155: Step './hack/e2e-internal/e2e-status.sh' finished in 569.032378ms
2019/02/08 11:35:23 process.go:153: Running: ./cluster/kubectl.sh --match-server-version=false version
2019/02/08 11:35:24 process.go:155: Step './cluster/kubectl.sh --match-server-version=false version' finished in 552.562255ms
2019/02/08 11:35:24 process.go:153: Running: ./hack/ginkgo-e2e.sh
Setting up for KUBERNETES_PROVIDER="skeleton".
Skeleton Provider: prepare-e2e not implemented
/Users/jayv/work/go/src/k8s.io/kubernetes/cluster/../cluster/skeleton/util.sh: line 22: KUBE_MASTER_IP: unbound variable
2019/02/08 11:35:24 process.go:155: Step './hack/ginkgo-e2e.sh' finished in 105.82755ms
2019/02/08 11:35:24 main.go:297: Something went wrong: encountered 1 errors: [error during ./hack/ginkgo-e2e.sh: exit status 1]

Anything else we need to know?:

Environment

  • build on OS x.
  • tag = 1.11.3t

@jayunit100 jayunit100 added the kind/bug label Feb 8, 2019

@jayunit100 jayunit100 changed the title kubetest: A Few UX improvements around build reqiurements, setting apiserver IP addres, skeleton/local mode guidance. kubetest: A Few UX bugs improvements around build reqiurements, setting apiserver IP addres, skeleton/local mode guidance. Feb 8, 2019

@jayunit100

This comment has been minimized.

Copy link
Member Author

jayunit100 commented Feb 8, 2019

/sig testing

@k8s-ci-robot k8s-ci-robot added sig/testing and removed needs-sig labels Feb 8, 2019

@jayunit100

This comment has been minimized.

Copy link
Member Author

jayunit100 commented Feb 8, 2019

FYI, some tricks to get kubetest working for me on my 1.9.6 cluster ... i havent run raw e2e's in a while so, while this probably seems trivial, it wasnt completely obvious, ill try to think of a way to put this into the docs once if i get a chance to look more at the new structure to the test repos.

  • Make sure you build the same version of kube test that your testing against. The master tests will verify the actual kubernetes Master version and make sure it matches the end to end build binary. In my case, I was testing 1.9.6.
  • KUBE_MASTER_IP=pmdb-stage-api.snn1.pf9.io KUBE_MASTER_NAME=pmdb-stage kubetest --test --kubeconfig=/Users/jayunit100/.kube/config --provider=skeleton
@BenTheElder

This comment has been minimized.

Copy link
Member

BenTheElder commented Feb 9, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment