Improve functional tests #816
Conversation
tomdee
commented
Sep 18, 2017
tomdee
commented
- Move to using bash_unit
- Test pings in both directions
- Test kubernetes subnet manager
- Test multiple versions of etcd and k8s during a release
tomdee
force-pushed
the
improve-tests
branch
from
51ac2c6
to
24a3268
Compare
Makefile
Outdated
| release-tests: bash_unit | ||
| # Run the functional tests with different etcd versions. Currently these are the latest point releases. | ||
| # This list should be updated during the release process. | ||
| ETCD_IMG="quay.io/coreos/etcd:v3.2.7" ./bash_unit dist/functional-test.sh |
There was a problem hiding this comment.
what if we do a latest tag here instead?
dist/functional-test-k8s.sh
Outdated
| } | ||
|
|
||
| test_vxlan() { | ||
| start_flannel vxlan && sleep 2 |
There was a problem hiding this comment.
how about instead of all these sleep 2, we put the pings in a loop so it retries for 5-10 times to see if pings work, so the tests don't become flakey
There was a problem hiding this comment.
the sleep is actually there to make sure the subnet.env file is written. I'll update the PR to get rid of the sleeps
|
|
||
| pings() { | ||
| # ping in both directions | ||
| assert "docker exec -it --privileged flannel-e2e-test-flannel1 /bin/ping -c 5 $ping_dest2" "Host 1 cannot ping host 2" |
There was a problem hiding this comment.
or we could put this in some sort of retry block, maybe
|
|
||
| test_manifest() { | ||
| # This just tests that the API server accepts the manifest, not that it actually acts on it correctly. | ||
| assert "cat ../Documentation/kube-flannel.yml | docker run -i --rm --net=host gcr.io/google_containers/hyperkube-amd64:v$K8S_VERSION /hyperkube kubectl create -f -" |
There was a problem hiding this comment.
btw, there's a cool project garethr/kubeval that can validate the k8s manifests. Not suggesting we should or shouldn't use it, but just throwing it out there
dist/functional-test.sh
Outdated
| # sleep 10000 | ||
| #fi | ||
| test_vxlan_ping() { | ||
| write_config_etcd vxlan && sleep 1 |
There was a problem hiding this comment.
curious, why is this sleep 1 vs 2 in the other test?
There was a problem hiding this comment.
i'd still advocate for some sort of ping retry instead of sleep
- Move to using bash_unit - Test pings in both directions - Test kubernetes subnet manager - Test multiple versions of etcd and k8s during a release
Improve functional tests
