Skip to content
This repository has been archived by the owner on Sep 14, 2020. It is now read-only.

Exceptions for kopf on event for network connectivity delayed / issues #396

Open
2 tasks done
kutysam opened this issue Aug 28, 2020 · 0 comments
Open
2 tasks done
Labels
question Further information is requested

Comments

@kutysam
Copy link

kutysam commented Aug 28, 2020

Question

I have been using kopf.on.event('', 'v1', 'pods') to listen to pods event and then I will do some operations to the events.
However, the issue comes when network is unstable.

One way to replicate it is,

  1. Load kopf.on.event('', 'v1', 'pods'), on a very low traffic cluster
  2. We will see lots of events coming in.
  3. Wait till events noise become minimal (as in no more new events)
  4. Turn off wifi to simulate network connectivity issues, no exception is thrown.
  5. When wifi is turned on, yeap, it resumes.

(If you turn off your wifi when an event is coming in, exception can be seen)

My aim is to do a restart when a network connectivity issue persists after a period of time.
Some possible ways:

  1. Have a health check to probe this connectivity every few seconds.
  2. Monitor for exceptions and do some logic based on them.

I've played around with some backoff etc. but I could not get it to work well.
I've tried upgraded from 0.25 to 0.27 but I don't see any exceptions.

Checklist

Keywords

  1. on event
  2. backoff
@kutysam kutysam added the question Further information is requested label Aug 28, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant