HDDS-15889. Show failed MoveResult breakdown in ozone admin containerbalancer status -v

[sreejasahithi]

What changes were proposed in this pull request?

Improves observability of Container Balancer move failures in ozone admin containerbalancer status -v.

Previously, verbose status only showed aggregate counts (“Failed to move containers”, “Failed to move containers by timeout”) with no breakdown of why moves failed, so users had to dig through SCM logs.

This change tracks per-iteration move failures on the SCM side and surfaces them in the CLI:

• Failure breakdown — counts grouped by reason (e.g. REPLICATION_FAIL_TIME_OUT, PRE_MOVE_CONTAINER_NOT_FOUND, ITERATION_MOVE_TIMEOUT, plus other MoveResult values)
• Failed move details — sample failed moves with container ID and source/target datanode UUIDs (capped at 100 total, 10 per reason; counts are not capped)

What is the link to the Apache JIRA

HDDS-15889

How was this patch tested?

Added test cases.
Tested in docker ozone-balancer cluster:

bash-5.1$ ozone admin containerbalancer start \
  -t 0.1 -d 100 -i 2 \
  --move-replication-timeout-minutes 1 \
  --move-timeout-minutes 16 \
  --balancing-iteration-interval-minutes 1
Container Balancer started successfully.
bash-5.1$

bash-5.1$ ozone admin containerbalancer status --verbose -H
ContainerBalancer is Running.
Started at: 2026-08-10 09:40:32
Balancing duration: 5m 24s

Container Balancer Configuration values:
Key                                                Value
Threshold                                          0.001
Max Datanodes to Involve per Iteration(percent)    100
Max Size to Move per Iteration                     500GB
Max Size Entering Target per Iteration             26GB
Max Size Leaving Source per Iteration              26GB
Number of Iterations                               2
Time Limit for Single Container's Movement         16min
Time Limit for Single Container's Replication      1min
Interval between each Iteration                    1min
Whether to Enable Network Topology                 false
Whether to Trigger Refresh Datanode Usage Info     false
Container IDs to Include in Balancing              None
Container IDs to Exclude from Balancing            None
Datanodes Specified to be Balanced                 None
Datanodes Excluded from Balancing                  None

Current iteration info:
Key                                                Value
Iteration number                                   2
Iteration duration                                 8s
Iteration result                                   -
Size scheduled to move                             1.56 KB
Moved data size                                    0 B
Scheduled to move containers                       2
Already moved containers                           0
Failed to move containers                          0
Failed to move containers by timeout               0
Entered data to nodes                              
d6898173-c204-4363-b841-48a130976c13 <- 800 B
d90d5e66-767e-4e24-9250-21ce9de426a5 <- 800 B
Exited data from nodes                             
b994d7e1-1394-4764-ac80-95ac89269baf -> 800 B
a7aa68a9-47c5-4c23-9a94-e8d0fb1fb3b1 -> 800 B

Completed iteration history:
Key                                                Value
Iteration number                                   1
Iteration duration                                 4m 16s
Iteration result                                   ITERATION_COMPLETED
Size scheduled to move                             1.56 KB
Moved data size                                    0 B
Scheduled to move containers                       2
Already moved containers                           0
Failed to move containers                          0
Failed to move containers by timeout               2
Failed container moves                             
  REPLICATION_FAIL_TIME_OUT                        2
    Source datanodes                               
      b994d7e1-1394-4764-ac80-95ac89269baf         1
      a7aa68a9-47c5-4c23-9a94-e8d0fb1fb3b1         1
    Target datanodes                               
      d90d5e66-767e-4e24-9250-21ce9de426a5         1
      d6898173-c204-4363-b841-48a130976c13         1
Entered data to nodes                              
d6898173-c204-4363-b841-48a130976c13 <- 800 B
d90d5e66-767e-4e24-9250-21ce9de426a5 <- 800 B
Exited data from nodes                             
b994d7e1-1394-4764-ac80-95ac89269baf -> 800 B
a7aa68a9-47c5-4c23-9a94-e8d0fb1fb3b1 -> 800 B


bash-5.1$ 

Green CI : https://github.com/sreejasahithi/ozone/actions/runs/30628368322

[sreejasahithi]

@ashishkumar50 could you please review this PR.

[ashishkumar50]

@sreejasahithi Thanks for working on this, Please update the patch wrt count instead of showing containerids. Here are the initial comments.

[ashishkumar50]

I think Failure breakdown and Failed move details should be merged. Also instead of showing containerids, we should show count of failures for source and target, otherwise it will be too verbose. Issue generally happens at DN level(like slow/unhealthy etc), so count is enough to know which dn is problematic.

[ashishkumar50]

The proto already sorts failure summaries, then the CLI sorts them again which is not required. Remove from server side sort.

[ashishkumar50]

Please update as per dn count suggestion and remove these limits.

[ashishkumar50]

Use enum FailureReason instead of plain string.

[sreejasahithi]

We already use enums at the call sites i.e MoveManager.MoveResult for what MoveManager.move() returns and ContainerBalancerTask.ContainerMoveFailureReason for before/during move setup, or iteration wait timeout.

[ashishkumar50]

Source or target can be null.