KAFKA-14887: FinalizedFeatureChangeListener should not shut down when…

… ZK session expires

FinalizedFeatureChangeListener shuts the broker down when it encounters an issue trying to process feature change
events. However, it does not distinguish between issues related to feature changes actually failing and other
exceptions like ZooKeeper session expiration. This introduces the possibility that Zookeeper session expiration
could cause the broker to shutdown, which is not intended. This patch updates the code to distinguish between
these two types of exceptions. In the case of something like a ZK session expiration it logs a warning and continues.
We shutdown the broker only for FeatureCacheUpdateException.

Reviewers: Kamal Chandraprakash <kamal.chandraprakash@gmail.com>, Christo Lolov <christololov@gmail.com>, Colin P. McCabe <cmccabe@apache.org>

core/src/main/scala/kafka/server/FinalizedFeatureChangeListener.scala

@@ -17,7 +17,7 @@
 
 package kafka.server
 
-import kafka.server.metadata.ZkMetadataCache
+import kafka.server.metadata.{FeatureCacheUpdateException, ZkMetadataCache}
 
 import java.util.concurrent.{CountDownLatch, LinkedBlockingQueue, TimeUnit}
 import kafka.utils.Logging
@@ -160,10 +160,12 @@ class FinalizedFeatureChangeListener(private val finalizedFeatureCache: ZkMetada
           // safe to ignore the exception if the thread is being shutdown. We raise the exception
           // here again, because, it is ignored by ShutdownableThread if it is shutting down.
           throw ie
-        case e: Exception => {
-          error("Failed to process feature ZK node change event. The broker will eventually exit.", e)
+        case cacheUpdateException: FeatureCacheUpdateException =>
+          error("Failed to process feature ZK node change event. The broker will eventually exit.", cacheUpdateException)
           throw new FatalExitError(1)
-        }
+        case e: Exception =>
+          // do not exit for exceptions unrelated to cache change processing (e.g. ZK session expiration)
+          warn("Unexpected exception in feature ZK node change event processing; will continue processing.", e)
       }
     }
   }