Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

contrib/script: To add script for kind cluster #13797

Closed
wants to merge 1 commit into from

Conversation

sayboras
Copy link
Member

@sayboras sayboras commented Oct 28, 2020

This commit is to add the script to bring up kind cluster with cilium.
Most of the options and steps are similar as smoke-test in github
action. Hence, it will help to some capacity to debug any failure in
smoke test as well.

Signed-off-by: Tam Mach sayboras@yahoo.com

PS: Sorry, I just remembered that you have similar PR #12527 after
created this, I can close this one as well /cc @christarazi

Testing

I used this one for quite sometimes as part of local development, some sample run can be found as below

NO_BUILD=1 IPv6=1 ./contrib/scripts/kind.sh
$ NO_BUILD=1 IPv6=1 ./contrib/scripts/kind.sh
Deleting cluster "kind" ...
Deleted Networks:
kind

Total reclaimed space: 2.885GB
Creating cluster "kind" ...
 βœ“ Ensuring node image (kindest/node:v1.19.1) πŸ–Ό 
 βœ“ Preparing nodes πŸ“¦ πŸ“¦  
 βœ“ Writing configuration πŸ“œ 
 βœ“ Starting control-plane πŸ•ΉοΈ 
 βœ“ Installing StorageClass πŸ’Ύ 
 βœ“ Joining worker nodes 🚜 
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Not sure what to do next? πŸ˜…  Check out https://kind.sigs.k8s.io/docs/user/quick-start/
Image: "cilium/cilium:latest" with ID "sha256:0c1d192b31a884752703d81126f284ca9e9f415793b5815de230372bf8445fcd" not yet present on node "kind-control-plane", loading...
Image: "cilium/cilium:latest" with ID "sha256:0c1d192b31a884752703d81126f284ca9e9f415793b5815de230372bf8445fcd" not yet present on node "kind-worker", loading...
Image: "cilium/operator-generic:latest" with ID "sha256:6d307105932393dbb92dc9c787cb18a510ed5b53c760a81908df4b4936c391c1" not yet present on node "kind-control-plane", loading...
Image: "cilium/operator-generic:latest" with ID "sha256:6d307105932393dbb92dc9c787cb18a510ed5b53c760a81908df4b4936c391c1" not yet present on node "kind-worker", loading...
Release "cilium" does not exist. Installing it now.
NAME: cilium
LAST DEPLOYED: Thu Oct 29 00:02:08 2020
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.

Your release version is 1.8.90.

For any further help, visit https://docs.cilium.io/en/v1.8/gettinghelp
pod/cilium-8jpsq condition met
pod/cilium-node-init-8fmll condition met
pod/cilium-node-init-r22f2 condition met
pod/cilium-operator-c4855965d-mrlpc condition met
pod/cilium-operator-c4855965d-xtjqc condition met
pod/cilium-r9m4b condition met
pod/coredns-f9fd979d6-nn5pl condition met
pod/coredns-f9fd979d6-sr6dc condition met
pod/etcd-kind-control-plane condition met
pod/kube-apiserver-kind-control-plane condition met
pod/kube-controller-manager-kind-control-plane condition met
pod/kube-proxy-898xb condition met
pod/kube-proxy-9q4x8 condition met
pod/kube-scheduler-kind-control-plane condition met
customresourcedefinition.apiextensions.k8s.io/ciliumnetworkpolicies.cilium.io condition met
deployment.apps/echo-a created
deployment.apps/echo-b created
deployment.apps/echo-b-host created
deployment.apps/pod-to-a created
deployment.apps/pod-to-external-1111 created
deployment.apps/pod-to-a-denied-cnp created
deployment.apps/pod-to-a-allowed-cnp created
deployment.apps/pod-to-external-fqdn-allow-google-cnp created
deployment.apps/pod-to-b-multi-node-clusterip created
deployment.apps/pod-to-b-multi-node-headless created
deployment.apps/host-to-b-multi-node-clusterip created
deployment.apps/host-to-b-multi-node-headless created
deployment.apps/pod-to-b-multi-node-nodeport created
deployment.apps/pod-to-b-intra-node-nodeport created
service/echo-a created
service/echo-b created
service/echo-b-headless created
service/echo-b-host-headless created
ciliumnetworkpolicy.cilium.io/pod-to-a-denied-cnp created
ciliumnetworkpolicy.cilium.io/pod-to-a-allowed-cnp created
ciliumnetworkpolicy.cilium.io/pod-to-external-fqdn-allow-google-cnp created
NO_BUILD=1 ./contrib/scripts/kind.sh
$ξ‚° NO_BUILD=1 ./contrib/scripts/kind.sh    
Deleting cluster "kind" ...
Deleted Networks:
kind

Total reclaimed space: 0B
Creating cluster "kind" ...
 βœ“ Ensuring node image (kindest/node:v1.19.1) πŸ–Ό 
 βœ“ Preparing nodes πŸ“¦ πŸ“¦  
 βœ“ Writing configuration πŸ“œ 
 βœ“ Starting control-plane πŸ•ΉοΈ 
 βœ“ Installing StorageClass πŸ’Ύ 
 βœ“ Joining worker nodes 🚜 
Set kubectl context to "kind-kind"
You can now use your cluster with:

kubectl cluster-info --context kind-kind

Thanks for using kind! 😊
Image: "cilium/cilium:latest" with ID "sha256:0c1d192b31a884752703d81126f284ca9e9f415793b5815de230372bf8445fcd" not yet present on node "kind-worker", loading...
Image: "cilium/cilium:latest" with ID "sha256:0c1d192b31a884752703d81126f284ca9e9f415793b5815de230372bf8445fcd" not yet present on node "kind-control-plane", loading...
Image: "cilium/operator-generic:latest" with ID "sha256:6d307105932393dbb92dc9c787cb18a510ed5b53c760a81908df4b4936c391c1" not yet present on node "kind-worker", loading...
Image: "cilium/operator-generic:latest" with ID "sha256:6d307105932393dbb92dc9c787cb18a510ed5b53c760a81908df4b4936c391c1" not yet present on node "kind-control-plane", loading...
Release "cilium" does not exist. Installing it now.
NAME: cilium
LAST DEPLOYED: Thu Oct 29 00:07:18 2020
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.

Your release version is 1.8.90.

For any further help, visit https://docs.cilium.io/en/v1.8/gettinghelp
pod/cilium-4nmkl condition met
pod/cilium-node-init-cw82x condition met
pod/cilium-node-init-sr25f condition met
pod/cilium-operator-c4855965d-cclsm condition met
pod/cilium-operator-c4855965d-ksg9l condition met
pod/cilium-vq66w condition met
pod/coredns-f9fd979d6-7vjmx condition met
pod/coredns-f9fd979d6-xlfhf condition met
pod/etcd-kind-control-plane condition met
pod/kube-apiserver-kind-control-plane condition met
pod/kube-controller-manager-kind-control-plane condition met
pod/kube-proxy-czxtb condition met
pod/kube-proxy-jmmvx condition met
pod/kube-scheduler-kind-control-plane condition met
customresourcedefinition.apiextensions.k8s.io/ciliumnetworkpolicies.cilium.io condition met
deployment.apps/echo-a created
deployment.apps/echo-b created
deployment.apps/echo-b-host created
deployment.apps/pod-to-a created
deployment.apps/pod-to-external-1111 created
deployment.apps/pod-to-a-denied-cnp created
deployment.apps/pod-to-a-allowed-cnp created
deployment.apps/pod-to-external-fqdn-allow-google-cnp created
deployment.apps/pod-to-b-multi-node-clusterip created
deployment.apps/pod-to-b-multi-node-headless created
deployment.apps/host-to-b-multi-node-clusterip created
deployment.apps/host-to-b-multi-node-headless created
deployment.apps/pod-to-b-multi-node-nodeport created
deployment.apps/pod-to-b-intra-node-nodeport created
service/echo-a created
service/echo-b created
service/echo-b-headless created
service/echo-b-host-headless created
ciliumnetworkpolicy.cilium.io/pod-to-a-denied-cnp created
ciliumnetworkpolicy.cilium.io/pod-to-a-allowed-cnp created
ciliumnetworkpolicy.cilium.io/pod-to-external-fqdn-allow-google-cnp created
NO_BUILD=1 NO_PROVISION=1 ./contrib/scripts/kind.sh
$ξ‚° NO_BUILD=1 NO_PROVISION=1 ./contrib/scripts/kind.sh
Release "cilium" has been upgraded. Happy Helming!
NAME: cilium
LAST DEPLOYED: Thu Oct 29 00:10:51 2020
NAMESPACE: kube-system
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.

Your release version is 1.8.90.

For any further help, visit https://docs.cilium.io/en/v1.8/gettinghelp
pod/cilium-4nmkl condition met
pod/cilium-node-init-cw82x condition met
pod/cilium-node-init-sr25f condition met
pod/cilium-operator-c4855965d-cclsm condition met
pod/cilium-operator-c4855965d-ksg9l condition met
pod/cilium-vq66w condition met
pod/coredns-f9fd979d6-7vjmx condition met
pod/coredns-f9fd979d6-xlfhf condition met
pod/etcd-kind-control-plane condition met
pod/kube-apiserver-kind-control-plane condition met
pod/kube-controller-manager-kind-control-plane condition met
pod/kube-proxy-czxtb condition met
pod/kube-proxy-jmmvx condition met
pod/kube-scheduler-kind-control-plane condition met
customresourcedefinition.apiextensions.k8s.io/ciliumnetworkpolicies.cilium.io condition met
deployment.apps/echo-a configured
deployment.apps/echo-b configured
deployment.apps/echo-b-host configured
deployment.apps/pod-to-a configured
deployment.apps/pod-to-external-1111 configured
deployment.apps/pod-to-a-denied-cnp configured
deployment.apps/pod-to-a-allowed-cnp configured
deployment.apps/pod-to-external-fqdn-allow-google-cnp configured
deployment.apps/pod-to-b-multi-node-clusterip configured
deployment.apps/pod-to-b-multi-node-headless configured
deployment.apps/host-to-b-multi-node-clusterip configured
deployment.apps/host-to-b-multi-node-headless configured
deployment.apps/pod-to-b-multi-node-nodeport configured
deployment.apps/pod-to-b-intra-node-nodeport configured
service/echo-a unchanged
service/echo-b unchanged
service/echo-b-headless unchanged
service/echo-b-host-headless configured
ciliumnetworkpolicy.cilium.io/pod-to-a-denied-cnp unchanged
ciliumnetworkpolicy.cilium.io/pod-to-a-allowed-cnp unchanged
ciliumnetworkpolicy.cilium.io/pod-to-external-fqdn-allow-google-cnp unchanged
contrib/script: To add script for kind cluster

@sayboras sayboras requested review from a team as code owners October 28, 2020 13:02
@maintainer-s-little-helper maintainer-s-little-helper bot added the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Oct 28, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot added this to In progress in 1.10.0 Oct 28, 2020
@sayboras sayboras force-pushed the contrib/kind-script branch 4 times, most recently from ae2b4f6 to b3d4893 Compare October 28, 2020 13:16
@sayboras sayboras added area/misc Impacts miscellaneous areas of the code not otherwise owned by another area. release-note/misc This PR makes changes that have no direct user impact. labels Oct 28, 2020
@maintainer-s-little-helper maintainer-s-little-helper bot removed the dont-merge/needs-release-note-label The author needs to describe the release impact of these changes. label Oct 28, 2020
Copy link
Member

@qmonnet qmonnet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have much experience with kind, but the script looks good overall, thanks!
Please find a few issues to fix and some suggestions below.

One other thing, should the script be mentioned in the documentation maybe?

I leave it to you and Chris to see what version of the script is preferable.

contrib/scripts/kind.sh Show resolved Hide resolved
contrib/scripts/kind.sh Outdated Show resolved Hide resolved
--set operator.prometheus.enabled=true \
--set hubble.enabled=true \
--set hubble.metrics.enabled="{dns,drop,tcp,flow,port-distribution,icmp,http}"
fi
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one suggestion: I would be tempted to have a set of options common to IPv4 and IPv6, and to complete with just the relevant ones in the if... else... fi. It would avoid duplicating a good portion of those options? But I'm fine if you prefer to keep this version, too.

# ./contrib/scripts/kind.sh
# IPv6=1 ./contrib/scripts/kind.sh
# NO_BUILD=1 ./contrib/scripts/kind.sh
# NO_PROVISION=1 ./contrib/scripts/kind.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could check for the number of arguments passed to the script, and print a help message shortly describing those variables if the user tries to pass any argument (which would likely be -h or --help if they haven't opened the script to read it).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point πŸ’―

contrib/scripts/kind.sh Show resolved Hide resolved
@sayboras
Copy link
Member Author

@qmonnet thanks for your review, I have corrected silly mistake from my side. I will wait for Chris input first before addressing your pending comments, hope its fine :)

@qmonnet
Copy link
Member

qmonnet commented Oct 28, 2020

Awesome and sure, no problem!

Copy link
Member

@christarazi christarazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! This is definitely useful to automate for GH Actions.

My script in #12527 was mostly intended for local use. That is why I also included a way to deploy a local registry into the kind cluster for quick iteration while developing because pushing images externally can be slow. Some of my comments are based on this. Do you plan for this script to be used for local dev as well? If so, it'd be nice to have a local registry like my PR adds (you can probably just rip it out from there.)

It would be good to also document this as well.

No need for you to close this PR; this is much further along πŸ˜‰

Comment on lines +17 to +20
# Set NO_BUILD=1 to avoid building docker images again
export 'NO_BUILD'="${NO_BUILD:0}"
# Set NO_PROVISION=1 to use existing kind cluster
export 'NO_PROVISION'="${NO_PROVISION:0}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find it easier to deal with flags if they are in the "positive" tone. Meaning, we can have a BUILD flag that's by default 1 and the user would need to say BUILD=0 ... to disable the building, instead of enabling the "negative".

I know we have precedence for using the "negative" tone in our Vagrantfiles, but maybe we can start to move away from that pattern. What do you think?

if [[ "${NO_PROVISION}" -ne "1" ]]; then
kind delete cluster
# clean up previous kind network
docker system prune -f
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this script is intended to be used locally by users wanting to spin up a local kind cluster, then this line will remove a lot more things than just the previous kind network, e.g. build cache.

Comment on lines +50 to +51
kind load docker-image cilium/cilium:latest
kind load docker-image cilium/operator-generic:latest
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this script is intended for local use, then the user might want to deploy their own dev images.

@sayboras
Copy link
Member Author

Thanks both for your review comments, let me mark this one as draft to reducing unnecessary notifications while I am incorporating your comments. Will mark it ready once done :)

@sayboras sayboras marked this pull request as draft October 29, 2020 08:23
This commit is to add the script to bring up kind cluster with cilium.
Most of the options and steps are similar as smoke-test in github
action. Hence, it will help to some capacity to debug any failure in
smoke test as well.

Signed-off-by: Tam Mach <sayboras@yahoo.com>
@stale
Copy link

stale bot commented Dec 11, 2020

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale. label Dec 11, 2020
@stale
Copy link

stale bot commented Dec 25, 2020

This issue has not seen any activity since it was marked stale. Closing.

@stale stale bot closed this Dec 25, 2020
@joestringer joestringer moved this from Done to In progress in 1.10.0 Feb 12, 2021
@sayboras sayboras deleted the contrib/kind-script branch January 10, 2022 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/misc Impacts miscellaneous areas of the code not otherwise owned by another area. release-note/misc This PR makes changes that have no direct user impact. stale The stale bot thinks this issue is old. Add "pinned" label to prevent this from becoming stale.
Projects
No open projects
1.10.0
In progress
Development

Successfully merging this pull request may close these issues.

None yet

4 participants