Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add k8s event to the resource owning the pod(s) being terminated. #155

Open
pims opened this issue Nov 5, 2019 · 6 comments
Labels

Comments

@pims
Copy link
Contributor

@pims pims commented Nov 5, 2019

Hi @linki,

Thoughts on also adding the termination event to the top most owner of a pod in addition to the pod itself?

We're attempting to add visibility for application owners when their pods get terminated – and new pods are starting. The terminated pods kind of disappear from the current view of the deployment, which adds a bit of work to find if any pod has been terminated through ChaosKube.

I know that it's possible to look at events in that namespace to find out what happened, but I believe that adding an event to the deployment/parent of the pod would greatly help in surfacing the actions of ChaosKube.

If that sounds like a useful feature, I'll happily submit a PR.

@linki

This comment has been minimized.

Copy link
Owner

@linki linki commented Nov 7, 2019

Hi @pims,

that sounds very useful to me.

What about duplicated events in this case? When I'm doing kubectl get events -w would the event show up twice (for the Pod and the Deployment)? Does it make sense to drop the event on the Pod? Do you know how other Kubernetes components deal with this?

@pims

This comment has been minimized.

Copy link
Contributor Author

@pims pims commented Nov 7, 2019

I need to do some more research on how other project implement this kind of functionality.
I’d like to avoid duplicate events if possible, but I didn't want to introduce breaking changes.

My guess is that most of the complexity comes from navigating the tree of owners.
I'll work on a PoC and we can continue the design discussion in that PR.

@linki linki added the enhancement label Nov 8, 2019
@linki

This comment has been minimized.

Copy link
Owner

@linki linki commented Nov 8, 2019

Sounds good!

@pims

This comment has been minimized.

Copy link
Contributor Author

@pims pims commented Nov 11, 2019

After prototyping a solution to this, I am no longer convinced this is a good idea.
It drastically extends the permissions granted to Chaoskube and is somewhat convoluted code.

One alternative would be to add annotations on the pod itself that points to any resource that should be notified.

@linki

This comment has been minimized.

Copy link
Owner

@linki linki commented Nov 12, 2019

Do you mean that in order to reach a Pod's Deployment you have to read the owning ReplicaSet to get its owner?

How about adding the event only on the direct owner for now?

@pims

This comment has been minimized.

Copy link
Contributor Author

@pims pims commented Nov 12, 2019

Do you mean that in order to reach a Pod's Deployment you have to read the owning ReplicaSet to get its owner?

Yes, which requires doing a Get on the parent and possibly on the parent's parent.

How about adding the event only on the direct owner for now?

That'd be easier, but in the case of a Deployment, I'm not sure how useful it is to add an event to the ReplicaSet controlling the current deployment.

What do you think?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.