Refactor the memcached discovery client

[caesarxuchao]

When debugging #68735, I found the cached discovery clients confusing, which in turn makes the DeferredDiscoveryRESTMapper hard to understand. I'll try to summarize the confusion and propose a high-level fix.

We have two implementations of the CacheDiscoveryInterface, one mem-cached, and one disk-cached.

The behavior of the disk-cached discovery client is intuitive,

• Upon cache misses, it contacts the server to get live results and updates the cache.
• Its Invalidate() method invalidates local caches, future requests results in server traffic.
• Its Fresh() method returns true unless pre-existing on-disk cache is used.

The behavior of the mem-cached one is not intuitive,

• Its cache update logic is implemented in the Invalidate() method, and at the end, the Invalidate() function sets valid=true.
• Upon cache misses, it returns an error without further contacting the apiserver. It returns a specific error type if the cache is completely empty. It expects the caller to interpret the error and invoke Invalidate() to populate the cache.
• Its Fresh() method always returns true as long as Invalidate() is called once.

IIUC, mem-cached client is implemented in this way to avoid overloading the apiserver in case the caller keeps causing cache misses with invalid groupVersion. The disk-cached one is used by kubectl, which is intended for human users, so it doesn't have this concern.

I plan to refactor the mem-cached client to:

• Invalidate() just sets the cache as "not-fresh"
• Upon cache misses, if the cache is "not-fresh", the client contacts the server to do a discovery, and updates the cache. Otherwise, it reports "cache miss".
• The client is created with empty cache, and the cache is initially "not-fresh".
• Remove the specific error type for empty cache, it shouldn't happen anymore.

I will add a Refresh() method to the cached discovery client interface. The method will do the discovery, and marks the cache as fresh upon success.

The current comment on the DeferredDiscoveryRESTMapper.Reset() claims the method marks the cached restmapper as invalid, but does not do discovery. However, if it's using the current memcached client as the underlying discovery client, the claim is false. I plan to give the DeferredDiscoveryRESTMapper an Invalidate method and a Refresh method, so user can choose whether the restmapper is lazily initialized.

/sig api-machinery
/assign

[caesarxuchao]

/cc @mbohlool

[mborsz]

We have run into #68735 again in scalability test. Any update here?
/cc @mborsz

[yue9944882]

this will simplify #65527 as well