Making it variable would be very much appreciated in our particular use-case.
Running a k8s cluster with various services running as pods in the cluster. Each of these pods has a sidecar Health-Monitor container which simply pings certain dependencies on a regular interval (10s). Should the health-check fail, the dependency is marked as down and potentially the pod can be marked as not-ready.
One of our dependency types is SQL, and the go-sql lib is used.
There are times where we need to bring down our DB (during patching etc)
and during these times we see connections to the DB spike to triple in traffic (makes sense now that we have looked at the code)
this causes the DB instance to be hit hard and load to increase, during a period where load is already high due to an external process
this also means that the HM acts differently according to different scenarios (happy/sad)
Note that we cannot simply block connections to the entire DB instance as it needs to be contactable during this period.
The text was updated successfully, but these errors were encountered: