Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add developer documentation on e2e testing. #14112

Merged
merged 1 commit into from
Sep 21, 2015

Conversation

timothysc
Copy link
Member

Add developer documentation for e2e testing, such that Q/E and other folks can leverage.

/cc @eparis @pmorie @jayunit100

@eparis
Copy link
Contributor

eparis commented Sep 17, 2015

@ncdc @smarterclayton

@k8s-bot
Copy link

k8s-bot commented Sep 17, 2015

GCE e2e build/test passed for commit d8253a2c6399d802da66487793a8014a7774ddee.

@k8s-bot
Copy link

k8s-bot commented Sep 17, 2015

GCE e2e build/test passed for commit 8feb9fadc887f4f904c3e912c73c52f5b6a9a9a8.


For a good overview of how we analyze performance data, please read the following [post](http://blog.kubernetes.io/2015/09/kubernetes-performance-measurements-and.html)

Developers who are interested in doing their own performance analysis, we recommend setting up [prometheus](http://prometheus.io/) for data collection, and using [promdash](http://prometheus.io/docs/visualization/promdash/) to visualize the data. There also exists the option of pushing your own metrics in from the tests using a [prom-push-gateway](http://prometheus.io/docs/instrumenting/pushing/). Containers for all of these components can be found [here](https://hub.docker.com/u/prom/).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/Develeopers/For developers

@jayunit100
Copy link
Member

Good tutorial, some minor ideas/comments, also pinged jon cope on it offline, who is a typical audience for this doc. Overall LGTM to me and good to merge as first iteration.

@timothysc
Copy link
Member Author

I'll wait for other comments, then change in bulk.

@k8s-github-robot k8s-github-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 17, 2015
@k8s-github-robot
Copy link

Labelling this PR as size/L

@k8s-github-robot k8s-github-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Sep 17, 2015
@k8s-github-robot
Copy link

Labelling this PR as size/L

@k8s-bot
Copy link

k8s-bot commented Sep 17, 2015

GCE e2e build/test failed for commit acb417f6736c6f310ec56dc9d6f912a584872c69.

@k8s-bot
Copy link

k8s-bot commented Sep 17, 2015

GCE e2e build/test passed for commit f958daf.

@pmorie
Copy link
Member

pmorie commented Sep 17, 2015

@timothysc You should add a link to this from the api-changes doc; there's text about needing to write this doc in that one.

@timothysc
Copy link
Member Author

@pmorie happy to do a follow on PR for links + cleanup.

@timothysc
Copy link
Member Author

@eparis Either the bot is super slow, or label magic has shifted yet again ... ?

@pmorie pmorie added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 21, 2015
@k8s-github-robot
Copy link

Automatic merge from submit-queue

k8s-github-robot pushed a commit that referenced this pull request Sep 21, 2015
Auto commit by PR queue bot
@k8s-github-robot k8s-github-robot merged commit 9f93dce into kubernetes:master Sep 21, 2015
@nikhiljindal
Copy link
Contributor

I just tried this and I am getting the following error:

--- FAIL: TestE2E (0.00s)
    e2e_test.go:132: Error waiting for all pods to be running and ready: error creating client: KubeConfig must be specified to load client config
FAIL

The command I ran is:

./_output/local/bin/linux/amd64/e2e.test --host="127.0.0.1:8080" --provider="local" --ginkgo.v=true -ginkgo.dryRun=true

I have a local cluster up which is fine.
Looking at the code, it seems like SetTextContext (https://github.com/kubernetes/kubernetes/blob/master/test/e2e/util.go#L130) is never called, which is why kubeconfig is empty leading to that error: https://github.com/kubernetes/kubernetes/blob/master/test/e2e/util.go#L824

What am I missing?

@jayunit100
Copy link
Member

ah ! provide the kubeconfig file, it can be empty :)

echo "{}" >> /tmp/dud
--kubeconfig=/tmp/dud

I created #14355 so that this doesn't bother us anymore when doing local testing.

@timothysc
Copy link
Member Author

@nikhiljindal you never specified your --kubeconfig which was denoted in the docs.

@ikehz
Copy link
Contributor

ikehz commented Oct 2, 2015

(cc @ixdy @quinton-hoole)

Just saw this PR; thanks for these docs @timothysc!

It seems, though, to contradict some of the information in development.md. It also doesn't look like it accurately reflects how we run tests in Jenkins. Was there discussion about removing, linking, or changing existing documentation, and how to keep this from drifting with what we do in Jenkins?

@timothysc
Copy link
Member Author

@ihmccreery This is developer oriented so it will certainly be different then the details re: Jenkins. I did that on purpose b/c upstreams scripts etc are prescriptive, while the details here are general to developer running locally.

We also automate our testing very differently as well. At some point we should unify that process, but imho separate doc.

@ikehz
Copy link
Contributor

ikehz commented Oct 2, 2015

Cool, thanks. Any thoughts on combining or removing content in development.md?

@timothysc
Copy link
Member Author

@ihmccreery I think it's fine to keep the content, but just add a link to this one.

@ixdy
Copy link
Member

ixdy commented Oct 6, 2015

Yeah, I missed this PR when it went by.

There is some important setup that's done by hack/ginkgo-e2e.sh; developers should be calling that (or one of the wrappers which calls it) rather than the e2e.test binary directly.

Eventually I'd like to fold much of the functionality from hack/ginkgo-e2e.sh into test/e2e/e2e_test.go, so we don't need this wrapper, but as-is this doc is incorrect.

@ixdy
Copy link
Member

ixdy commented Oct 6, 2015

... all that said, I really appreciate you writing documentation for this process!

I'll try to take a look tomorrow at this doc and give specific feedback on things where our process differs a bit with yours.

@ikehz ikehz mentioned this pull request Dec 11, 2015
xingzhou pushed a commit to xingzhou/kubernetes that referenced this pull request Dec 15, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm "Looks good to me", indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet