docs(cli): document all 106 CLI commands

[rangareddy]

Describe the issue this Pull Request addresses

Closes #15376 (JIRA: HUDI-4689, epic
HUDI-1388), "Add documentation for all CLI commands".

hudi-cli exposes 106 commands across 26 command classes. website/docs/cli.md had two gaps:

1. 20 commands were absent from the page entirely - not documented, and not even listed, so a reader could not
   discover they exist. The whole locks audit family, both timeline commands and both diff commands were among
   them.
2. 56 more appeared only as a one-line entry in the embedded help output, with their options documented
   nowhere on the page.

Summary and Changelog

One file, website/docs/cli.md.

1. The help listing now carries all 106 commands. Entries were inserted at their alphabetical positions
rather than the block being regenerated, so the diff stays reviewable.

2. A new Command reference section documents every command, grouped into nine areas, with each command's
options and their defaults. It also records which entries are aliases rather than distinct commands: refresh,
metadata refresh, commits refresh, cleans refresh and savepoints refresh are five names for one method that
reloads table metadata, and temp query / temp_query, temp delete / temp_delete and
temps show / temps_show are spelling variants of three commands.

3. New worked sections for the commands that had no documentation at all:

Section	Commands
Auditing Storage Locks	the five locks audit commands, the layout under .hoodie/.locks, and what validate reports as an error versus a warning
Inspecting the Timeline	timeline show active / incomplete and the two metadata timeline equivalents, with their output columns
Diffing a File or Partition	diff file, diff partition
Repairing a Table	rename partition, repair deprecated partition, repair show empty commit metadata
Changing the Meta Fields Mode	table set-meta-fields-mode and its two guards

Four more are documented in the sections they belong to: trigger archival under Archived Commits,
show restore / show restores under Savepoint and Restore, kerberos kdestroy under the Kerberos section, and
metadata delete-record-index under Record Index Lookup.

Every command name, option, default and behaviour was read from the @ShellMethod and @ShellOption declarations
on master, not from the help strings, which have drifted.

Things the source read turned up that the help text does not say

• diff file and diff partition ignore --startTs and --endTs unless --includeArchivedTimeline is also
  true. CLIUtils#getTimelineInRange (:46-62) applies the bounds only on the archived branch and returns the
  whole active timeline otherwise, so a narrow range silently does nothing. The help text advertises the defaults
  with no hint of the precondition.
• repair show empty commit metadata returns void and reports through log.warn, so it renders no result
  table; its findings go to the CLI log.
• rename partition physically removes the old partition directory after its delete_partition write
  (SparkMain#renamePartition), while repair deprecated partition leaves the emptied default directory. The two
  are otherwise the same shape.

Verification

cd website && npm run build
  -> [SUCCESS] Generated static files in "build"

docs markdown link-format checks (the ones asf-site.ci.yml runs on website/docs/**)
  -> no absolute /docs/ paths, no links missing .md, no ../ or ./ prefixes

coverage check, parsing @ShellMethod keys out of hudi-cli on master and grepping cli.md
  -> 106 source commands, 0 missing, 0 documented only in the help listing

The extraction behind the command reference was validated against the source as well: the 18 commands recorded as
taking no options were each confirmed to declare none, and the captured option count reconciles exactly with the
404 @ShellOption occurrences in the tree once the two alias methods that double-count are accounted for.

Sections and the reference were confirmed in the rendered build/docs/next/cli/index.html, including that the
escaped angle brackets in the rename partition usage string render as literal < and >.

Impact

Documentation only; no code or config changes. The page is the current (next) docs version, which tracks
master, where these commands live. Versioned copies under website/versioned_docs/ are deliberately untouched,
since several of these commands do not exist in those releases.

Risk Level

none

Documentation Update

This is the documentation update.

Contributor's checklist

• Read through contributor's guide
• Enough context is provided in the sections above
• Adequate tests were added if applicable
• CI passes on my PR

[voonhous]

Is this a 2 part PR or? The description says part of. If this is a 2 part PR, please make it clear in the title this is Part 1 of N, where N should be clearly defined.

[rangareddy]

Hi @voonhous I have update the PR description and documentation changes.

[hudi-agent]

⚠️ 🤖 This review was generated by an AI agent and may contain mistakes. Please verify any suggestions before applying.

Thanks for the docs update! This is a thorough and impressively accurate expansion of cli.md — I cross-checked the command names, options, defaults, and behavioral notes (diff range semantics, lock-audit paths and verdicts, delete-record-index backup path, set-meta-fields-mode guards, trigger archival, etc.) against the @ShellMethod/@ShellOption declarations on master and they line up. The only concern is a handful of reference entries that print an unresolved Java constant as the --sparkMaster default instead of its value. Please route to a Hudi committer/PMC member (e.g. @yihua) for a final pass before merge.

[hudi-agent]

🤖 This entry lists the --sparkMaster default as the raw Java constant SparkUtil.DEFAULT_SPARK_MASTER rather than its value. That constant resolves to yarn (SparkUtil.DEFAULT_SPARK_MASTER = "yarn"), so it would help to substitute the concrete value here — the way the other Spark commands in this reference already do (e.g. compaction/marker show local). The same unresolved constant appears on the clustering schedule / clustering scheduleAndExecute entries (lines 1177, 1179) and on metadata create / metadata init / metadata list-partitions (lines 1246, 1252, 1256). It might also be worth a one-line note that this default is yarn, since that differs from the local default used by the compaction and other Spark commands and could surprise a reader running these on a non-YARN setup.

_{⚠️ AI-generated; verify before applying. React 👍/👎 to flag quality.}

[voonhous]

minor: Not blocking, but the same trap is wider than these six entries. SparkUtil.getDefaultConf (SparkUtil.java:89-95) treats a blank master as absent, so the twelve --sparkMaster options rendered bare in this reference (defaultValue = "") and the "empty by default" wording at line 998 also resolve to yarn. The legend at lines 1070-1071 has no state for the 119 options rendered without a value. Could we add a legend clause for unset defaults and one sentence that an unset --sparkMaster means yarn?

[voonhous]

Site build, MDX compile and the asf-site link checks pass on this head, and the reference matches the master annotations on 105 of 106 option surfaces. Findings inline.

[voonhous]

major: This transcript cannot be produced from a fresh session: deleteRecordIndex passes new HoodieSparkEngineContext(jsc) (MetadataCommand.java:156) and jsc is only set by metadata create / init / list-partitions (:460), so connect followed by this command NPEs in jsc.sc(). metadata delete (:143) uses HoodieLocalEngineContext, and deleteMetadataTablePartition never reads the context. Could we note that prerequisite here, and follow up with a one-line master fix to HoodieLocalEngineContext so the output shown is reachable?

[voonhous]

major: The site's default docs version is 1.2.0, not next (versions.json; docusaurus.config.js:563-565 maps 1.2.0 to path ""), so hudi.apache.org/docs/cli will not show any of this. On release-1.2.0 every command documented here exists with the same options except table set-meta-fields-mode, and the older spelling there is commits show_infights. Could we mirror the change into versioned_docs/version-1.2.0/cli.md, minus the Meta Fields Mode section and reference entry?

[voonhous]

major: --partition_path is optional in source (MetadataCommand.java:398-399, defaultValue = ""), and line 770 of this page already says so for a global RLI; (required) here contradicts both. The description is also cut at the first string literal, dropping the global-vs-partitioned rule. Could we take this?

Suggested change

	- **`metadata lookup-record-index`** Print Record index information for a record_key.
	<br />Options: `--record_key` (required), `--partition_path` (required)
	- **`metadata lookup-record-index`** Print Record index information for a record_key. For global RLI, only record key is required. For partitioned RLI, both record key and partition path are required.
	<br />Options: `--record_key` (required), `--partition_path`

[voonhous]

major: hdfsparquetimport no longer exists: HDFSParquetImportCommand was deleted in aeebfcf (#12695), and the new Command reference correctly omits it, so the listing now contradicts the reference. Could we drop the line while this block is being reconciled?

Suggested change

* hdfsparquetimport - Imports Parquet table to a hoodie table

[voonhous]

major: The listing still misses four commands that exist on master, so it covers 102 rather than all 106: commits show_inflights (after line 254), kerberos kinit (here, next to its new counterpart), metadata lookup-record-index (after line 292) and table change-table-type (before line 332). Could we add them, starting with:

Suggested change

	* kerberos kdestroy - Destroy Kerberos authentication
	* kerberos kdestroy - Destroy Kerberos authentication
	* kerberos kinit - Perform Kerberos authentication

[voonhous]

nit: Feel free to ignore, but line 317 just above is a pre-existing copy of the show archived commits line: the source string is "Read commits from archived files and show file group details" (ArchivedCommitsCommand.java:99). Could we fix it while the block is open?

[voonhous]

nit: Feel free to ignore. The code logs out the login user, not the current user, and only when it holds Kerberos credentials (KerberosAuthenticationCommand.java:70-76). Could we say "logs the login user out of the keytab"?

[voonhous]

nit: Feel free to ignore. Rollbacks never appear in commits show regardless of state, since it filters to getCommitsTimeline() (CommitsCommand.java:73; commit, deltacommit, replacecommit and clustering only). Could we drop "that never completed" so the reason reads as action rather than state?

[voonhous]

nit: Feel free to ignore. null is also printed when the backup rename fails or returns false (HoodieTableMetadataUtil.java:1774-1781); in that case the partition is neither backed up nor deleted, yet the success line still prints. Could we widen this sentence to "or if the rename fails"?

[voonhous]

nit: Feel free to ignore. This block and the metadata delete-record-index one at line 774 use shell, while the other fourteen new transcript blocks use java. Could we pick one for the new blocks?