[bug]: Too old resource version

[lkt82]

Describe the bug

Hi.
After having a operator running for a few days I start getting these errors and the operator stops working.

fail: KubeOps.Operator.Watcher.ResourceWatcher[0]
There was an error while watching the resource "XXX".
k8s.KubernetesException: too old resource version: XXXXXX (XXXXXX)

To reproduce

Watch a custom resource for a few days

Expected behavior

The operator continues to work all the time

Screenshots

No response

Additional Context

Version 8.0.1

[gabrieledemaglie]

Hi,

I'm facing the same problem. If I leave the operator there without performing any operation for a while(the amount of time needed for the error to appear is random) the same error occurs to me.

I tried to study a bit the situation, this is my understanding:

When the operator wants to watch a list of resources it has to perform a LIST operation. It will obtain the current status of the asked resources. The resourceVersion obtained is used to start a WATCH (and saved in the "_lastResourceVersion" property of ResourceWatcher class).

Every time an event related to the watched resources occurs, the "_lastResourceVersion" property is updated.

When the current WATCH operation is too old, a new WATCH must be started: the saved resourceVersion will be used:

• If the operator has refreshed recently the "_lastResourceVersion" property, the WATCH will be started again with no issues.
• If the operator did nothing for a while, the "_lastResourceVersion" property will contain a version that is not accepted anymore from the API Server.

Currently this leads to an infinite loop where the same invalid ResourceVersion is continuously used.
There should be something like a try/catch surrounding the watch() call that set to NULL the "_lastResourceVersion" property in case of the "resource version too old" error.

@buehler am I correct?

Thanks for your work.
Gabriele

[lkt82]

@gabrieledemaglie I think you are correct :)

[robertcoltheart]

I'm facing this issue too, any resolution @buehler?