HDDS-15306. Expose Disk Balancer CLI in top-level datanode help and improve usability

[Gargi-jais11]

What changes were proposed in this pull request?

1. Make diskBalancer visible on top-level datanode help command which was hidden in this PR HDDS-13649
2. Improve diskBalancer help msg to explain how to terminate stdin input when using - to provide datanode addresses through standard input.
3. All the values in diskbalancer report should be rounded to 2 decimal places.
4. Only the IdealUsage: 0.14747954 | Threshold: 10.0% | ThresholdRange: (0.04747954, 0.24747954)
   Utilization | VolumeDensity should be in percentage form.
5. In diskbalancer report for each dn storage report also add ozoneAvailable field and change TotalCapacity -> OzoneCapacity and UsedSpace -> OzoneUsed.
6. Add more clarity about delta moves in the diskbalancer status notes.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15306

How was this patch tested?

Tested manually:

Before Fix:

// diskbalancer not appear in datanode cli option
Usage: ozone admin datanode [-hV] [--verbose] [COMMAND]
Datanode specific operations
  -h, --help      Show this help message and exit.
  -V, --version   Print version information and exit.
      --verbose   More verbose output. Show the stack trace of the errors.
Commands:
  list          List info of datanodes
  decommission  Decommission a datanode
  maintenance   Put a datanode into Maintenance Mode
  recommission  Return a datanode to service
  status        Show status of datanodes
  usageinfo     List usage information of a datanode by IP address, hostname or
                  UUID.

                Legend (bytes):
                  Filesystem Capacity/Used/Available: raw filesystem stats for
                  the Datanode, aggregated across volumes.
                  Ozone Capacity/Used/Available: Ozone-usable stats after
                  reserved-space adjustment, aggregated across   volumes (see
                  hdds.datanode.dir.du.reserved / hdds.datanode.dir.du.reserved.
                  percent).
                  Reserved: hdds.datanode.dir.du.reserved: configured reserved
                  space for non-Ozone usage.
                  Committed: space pre-allocated for containers.
                  Free Space To Spare: minimum free space to keep before
                  closing containers.
 
 // not much info to use stdin -
 
 Get DiskBalancer status
      [<datanode address>...]
                  Datanode addresses: one or more, separated by spaces. To read
                    from stdin, specify '-' and supply one item per line.Port
                    is optional and defaults to 19864 (CLIENT_RPC port).
                    Examples: 'DN-1', 'DN-1:19864', '192.168.1.10'.
  -h, --help      Show this help message and exit. 
  
// Inconsistency in decimals and percentage for Disk Utilisation or Threshold
// no ozoneAvailable given
  
  dn-1 ~]# ozone admin datanode diskbalancer report --in-service-datanodes --verbose
Report result:
Datanode: dn1 (10.82.246.167:19864)
Aggregate VolumeDataDensity: 0.20786860859385214
IdealUsage: 0.14747954 | Threshold: 10.0% | ThresholdRange: (0.04747954, 0.24747954)
Volume Details:
StorageID                                     StoragePath                                TotalCapacity       UsedSpace   Container Pre-AllocatedSpace   EffectiveUsedSpace     Utilization   VolumeDensity
DS-71f33f87-9212-488c-9ae6-4db48c9a48f1       /hadoop-ozone/datanode/data/hdds               349.27 GB         4.71 GB                       26.82 GB             75.05 GB      0.21487346      0.06739392
DS-ac73c11c-993a-49f9-b97e-ea61145452e2       /hadoop-ozone/datanode/data2/hdds       195.78 GB         1.11 GB                        4.27 GB             16.36 GB      0.08357777      0.06390177
DS-5b27f109-0029-424a-9092-5ae7692b98db       hadoop-ozone/datanode/data3/hdds       195.78 GB         2.32 GB                       18.06 GB             30.86 GB      0.15760217      0.01012263
DS-3350ed2c-326c-4725-9fe2-755e85d17b02       hadoop-ozone/datanode/data4/hdds       195.78 GB         1.13 GB                        4.26 GB             15.86 GB      0.08102925      0.06645028 

After Fix:

// diskbalancer visible in datanode cli option
Usage: ozone admin datanode [-hV] [--verbose] [COMMAND]
Datanode specific operations
  -h, --help      Show this help message and exit.
  -V, --version   Print version information and exit.
      --verbose   More verbose output. Show the stack trace of the errors.
Commands:
  list          List info of datanodes
  decommission  Decommission a datanode
  maintenance   Put a datanode into Maintenance Mode
  recommission  Return a datanode to service
  status        Show status of datanodes
  usageinfo     List usage information of a datanode by IP address, hostname or
                  UUID.

                Legend (bytes):
                  Filesystem Capacity/Used/Available: raw filesystem stats for
                  the Datanode, aggregated across volumes.
                  Ozone Capacity/Used/Available: Ozone-usable stats after
                  reserved-space adjustment, aggregated across   volumes (see
                  hdds.datanode.dir.du.reserved / hdds.datanode.dir.du.reserved.
                  percent).
                  Reserved: hdds.datanode.dir.du.reserved: configured reserved
                  space for non-Ozone usage.
                  Committed: space pre-allocated for containers.
                  Free Space To Spare: minimum free space to keep before
                  closing containers.

  diskbalancer  DiskBalancer specific operations to ensure even disk
                  utilisation. It is disabled by default. To enable it, set
                  hdds.datanode.disk.balancer.enabled as true.
                  
  // improved stdin - info in help msg
  Usage: ozone admin datanode diskbalancer status [-hV] [--in-service-datanodes]
       [--json] [--verbose] [<datanode address>...]
Get DiskBalancer status
      [<datanode address>...]
                  Datanode addresses: one or more on the command line, OR read
                    from stdin with '-'.
                  Stdin usage:
                    ozone admin datanode diskbalancer <subcommand> -
                    Then type one datanode per line and end input:
                      - Linux/macOS: Ctrl-D
                      - Windows:     Ctrl-Z, then Enter
                  Examples:
                    # Piped (recommended for scripts)
                    echo -e "DN-1\nDN-2" | ozone admin datanode diskbalancer
                    status -
                    # From file having list of dns to balance
                    ozone admin datanode diskbalancer report - < datanode-lists.
                    txt
                  Port is optional and defaults to 19864 (CLIENT_RPC port).
                  Address examples: 'DN-1', 'DN-1:19864', '192.168.1.10'.
  -h, --help      Show this help message and exit.
      --in-service-datanodes
                  If set, the client will send DiskBalancer requests to all
                    available DataNodes in the HEALTHY and IN_SERVICE
                    operational state.
      --json      Format output as JSON
  -V, --version   Print version information and exit.
      --verbose   More verbose output. Show the stack trace of the errors.
      
  // round of value to 2 decimal places
  // *IdealUsage | Threshold | ThresholdRange | Utilization | VolumeDensity** should be in percentage form.
  // add **ozoneAvailable field** and change **TotalCapacity -> OzoneCapacity and UsedSpace -> OzoneUsed.**
  // explain delta move for source volume
 bash-5.1$ ozone admin datanode diskbalancer report - < datanode-list
Report result:
Datanode: ozone-datanode-3.ozone_default (172.18.0.8:19864)
Aggregate VolumeDataDensity: 0.18%
IdealUsage: 12.66% | Threshold: 0.01% | ThresholdRange: (12.65%, 12.67%)

Volume Details:

StorageID                                     StoragePath                                OzoneCapacity  OzoneAvailable       OzoneUsed   Container Pre-AllocatedSpace   EffectiveUsedSpace Utilization VolumeDensity
DS-78ca0f8e-6b4f-4639-aa91-ec2389f1c49f       /data/hdds1/hdds                              1006.75 GB       880.15 GB         5.42 GB                            0 B            126.60 GB 12.57%    0.09%
DS-45ac9e0c-b254-4585-b535-60467fde3be3       /data/hdds2/hdds                              1006.75 GB       880.15 GB         5.64 GB                        1.77 GB            128.37 GB 12.75%    0.09%
DS-ee5326f5-5933-4560-8b2c-75dcd8900b1b       /data/hdds3/hdds                              1006.75 GB       880.15 GB         5.53 GB                         907 MB            127.48 GB 12.66%    0.00%
Note:
  - Aggregate VolumeDataDensity: Sum of per-volume density (deviation from ideal); higher means more imbalance.
  - IdealUsage: Target utilization ratio (0-1) when volumes are evenly balanced.
  - ThresholdRange: Acceptable deviation (percent); volumes within IdealUsage +/- Threshold are considered balanced.
  - VolumeDensity: Deviation of a particular volume's utilization from IdealUsage.
  - Utilization: Ratio of actual used space to capacity (0-1) for a particular volume.
  - OzoneCapacity: Ozone volume capacity.
  - OzoneAvailable: Ozone available space.
  - OzoneUsed: Ozone used space.
  - Container Pre-AllocatedSpace: Space reserved for containers not yet written to disk.
  - EffectiveUsedSpace: This is the actual used space of volume which is visible to the diskBalancer : (ozoneCapacity minus ozoneAvailable) + containerPreAllocatedSpace + move delta .
  - move delta: source volume space to be reclaimed after move completion; this value is reflected only when diskBalancer is running else it is 0.

[Copilot]

Pull request overview

This PR improves DiskBalancer CLI discoverability and output usability by exposing the diskbalancer command in top-level datanode help, enhancing stdin-related help text, and changing the diskbalancer report output to show percentages/rounded values and additional per-volume fields.

Changes:

• Unhide ozone admin datanode diskbalancer and improve CLI help text for reading datanode addresses from stdin (-).
• Update diskbalancer report output to show selected ratios as percentages (2 decimals) and rename/add per-volume capacity/usage fields (including ozoneAvailable).
• Update protobuf schema and tests to reflect the new per-volume report fields.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
hadoop-ozone/cli-admin/src/test/java/org/apache/hadoop/hdds/scm/cli/datanode/TestDiskBalancerSubCommands.java	Updates JSON-output assertions and report-proto generation to include the renamed/new per-volume fields.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerReportSubcommand.java	Formats report ratios as percent strings, renames/adds reported columns/JSON keys, and adds explanatory notes.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerCommands.java	Exposes the diskbalancer command in top-level datanode CLI help and updates description text.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DatanodeParameters.java	Improves help text explaining stdin usage termination and examples for -.
hadoop-hdds/interface-client/src/main/proto/hdds.proto	Renames/adds fields in VolumeReportProto for ozone capacity/available/used.
hadoop-hdds/container-service/src/test/java/org/apache/hadoop/ozone/container/diskbalancer/TestDiskBalancerProtocolServer.java	Updates protocol server tests for the new VolumeReportProto fields.
hadoop-hdds/container-service/src/main/java/org/apache/hadoop/ozone/container/diskbalancer/DiskBalancerService.java	Populates the updated VolumeReportProto fields when building volume reports.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Gargi-jais11]

@ChenSammi Please re-review, I have addressed all comments.