Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete entire pods in lifecycle environment #44

Merged
merged 1 commit into from
May 24, 2018
Merged

Conversation

siggy
Copy link
Member

@siggy siggy commented May 24, 2018

The lifecycle environment was testing service discovery via bb-terminus
container exit. This did not play well with k8s's CrashLoopBackoff.

Disable bb-terminus container exit in favor of a redeployer script,
that deletes each bb-terminus pod once per minute.

Fixes #42, relates to #43.

Signed-off-by: Andrew Seigner siggy@buoyant.io

@siggy siggy self-assigned this May 24, 2018
Copy link
Member

@klingerf klingerf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, this fixes the CrashLoopBackoff. 🎉

FWIW, I tested this on kubernetes 1.10, and the redeployer pod failed with:

Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:lifecycle:default" cannot list pods in the namespace "lifecycle"

Once I created a clusterrolebinding for the lifecycle namespace, it succeeded:

kubectl create clusterrolebinding lifecycle-cluster-admin --clusterrole=cluster-admin --serviceaccount=lifecycle:default

Maybe we should add that clusterrolebinding as a separate object in the k8s config?

- name: redeploy
configMap:
name: redeploy
defaultMode: 0744
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whoa cool, TIL

The lifecycle environment was testing service discovery via bb-terminus
container exit. This did not play well with k8s's `CrashLoopBackoff`.

Disable bb-terminus container exit in favor of a `redeployer` script,
that deletes each bb-terminus pod once per minute. Also create
`ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding`, and install
conduit into conduit-lifecycle namespace.

Fixes #42, relates to #43.

Signed-off-by: Andrew Seigner <siggy@buoyant.io>
Copy link
Member

@klingerf klingerf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⭐️ Great, thanks for adding the RBAC stuff!

@siggy siggy merged commit e6719ad into master May 24, 2018
@siggy siggy deleted the siggy/redeployer branch May 24, 2018 21:38
@siggy siggy added the benchmarking Performance testing label Jul 25, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
benchmarking Performance testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants