KAFKA-14275; KRaft Controllers should crash after failing to apply any metadata record (#12709)

[niket-goel]

Make all faults in metadata processing on standby controllers be fatal. This is the same behavior-wise as the active controller. This prevents a standby controller from eventually becoming active with incomplete state.

Reviewers: Colin Patrick McCabe cmccabe@apache.org, Jason Gustafson jason@confluent.io
(cherry picked from commit 98a3dcb)

Conflicts were resolved in the following files around availability of Builders for some test classes:
metadata/src/test/java/org/apache/kafka/controller/QuorumControllerTest.java
metadata/src/test/java/org/apache/kafka/controller/QuorumControllerTestEnv.java

[niket-goel]

Interesting test failures on this. All are in related code, but not if sure if there are flakes or actual failures. Will run some local tests to verify.

Build / JDK 8 and Scala 2.12 / testDescribeQuorumRequestToBrokers() – kafka.server.KRaftClusterTest 
(need to understand - failing assertion around return value)

Build / JDK 17 and Scala 2.13 / testIncrementalAlterConfigs() – kafka.server.KRaftClusterTest 
(Could be infrastructural - failed to startup due to heap space)

Build / JDK 17 and Scala 2.13 / testConfigurationOperations() – org.apache.kafka.controller.QuorumControllerTest
(Could be a flake - renouncing from a higher epoch than expected)

[niket-goel]

All failing tests seem to have failed due to general flakiness.

[niket-goel]

PS submitted #12733 to try to address one of the Flaky tests above.

[hachikuji]

LGTM. Looks like there are some conflicts to resolve.

[hachikuji]

Closing since it looks like @cmccabe already picked this.