KAFKA-20295: bridge pre-KIP-1312 controller decommissioning

[davide-armand]

KIP-1312 can unregister a controller only after metadata.version
reaches 4.4. Old, stopped controller registrations ("ghost
controllers") can advertise only older metadata versions, so feature
validation rejects the metadata.version upgrade needed to remove them
(deadlock situation).

This PR solves that deadlock by bridging the pre-KIP-1312 implementation
(PR #70) with KIP-1312.
Before 4.4-IV2, it marks a stopped controller that is no longer a quorum
voter with __decommissioned_controller. Patched controllers ignore
marked registrations during feature validation, allowing
metadata.version to reach 4.4. Afterwards, running the same operation
writes an UnregisterControllerRecord and deletes the registration
(KIP-1312 behavior).

This retains the useful marker from PR #70 and validation filter , but
changes its integration with Kafka. PR #70 introduced a fork-local
DecommissionController protocol and always retained the marked
registration, while also widening its advertised feature ranges. This
port reuses the upstream KIP-1312 UnregisterController API, preserves
the stored feature ranges, and physically removes the registration once
4.4 is finalized.

Without this PR, the following happens when doing a rolling upgrade for example
from 4.0 to 4.4:

1. Regular rolling upgrade operations (4.4 nodes up, 4.0 nodes down)
2. Unregister 4.0 brokers --> OK
3. Unregister 4.0 controllers using KIP-1312 --> KO
   This fails because 4.0 ghost controllers don't have KIP-1312, so they cannot be unregistered:

[root@davide-armand-test-n1313 ~]# $K_BIN/kafka-cluster.sh unregister-controller --bootstrap-controller "$K_CONTROLLER_BOOTSTRAP" --id 6

Unregistering controller 10006
[2026-08-18 10:00:25,329] ERROR [AdminClient clientId=adminclient-1] Unregister controller request for controller ID 10006
  failed: The current MetadataVersion is too old to support controller unregistration.
(org.apache.kafka.clients.admin.KafkaAdminClient)
  The target cluster does not support the controller unregistration API.

4. Upgrade metadata version --> KO
   This fails because 4.0 ghost controllers could not be unregistered at step 3:

[root@davide-armand-test-n1313 ~]#  $K_BIN/kafka-features.sh --bootstrap-server "$K_BOOTSTRAP" upgrade --release-version 4.4-IV2 --dry-run

Can not upgrade eligible.leader.replicas.version to 1. The update failed for all features since the following feature had an error: Invalid update version 33 for feature metadata.version. Controller 10006 only supports versions 7-25
Can not upgrade group.version to 1. The update failed for all features since the following feature had an error: Invalid update version 33 for feature metadata.version. Controller 10006 only supports versions 7-25
Can not upgrade kraft.version to 1. The update failed for all features since the following feature had an error: Invalid update version 33 for feature metadata.version. Controller 10006 only supports versions 7-25
Can not upgrade metadata.version to 33. The update failed for all features since the following feature had an error: Invalid update version 33 for feature metadata.version. Controller 10006 only supports versions 7-25
Can not upgrade share.version to 2. The update failed for all features since the following feature had an error: Invalid update version 33 for feature metadata.version. Controller 10006 only supports versions 7-25
Can not upgrade streams.version to 1. The update failed for all features since the following feature had an error: Invalid update version 33 for feature metadata.version. Controller 10006 only supports versions 7-25
Can not upgrade transaction.version to 2. The update failed for all features since the following feature had an error: Invalid update version 33 for feature metadata.version. Controller 10006 only supports versions 7-25
7 out of 7 operation(s) failed.

Co-authored-by: Pi Agent (gpt-5.6-terra)