feat: Add metadata record_index lookup command to Hudi CLI#17940

Merged

nsivabalan merged 6 commits intoapache:masterfrom

suryaprasanna:surya-dev-06

Feb 11, 2026

Contributor

suryaprasanna commented Jan 19, 2026

Describe the issue this Pull Request addresses

This PR adds a new CLI command to query record index information from the metadata table, allowing users to look up the global file location (partition path, file ID, and instant time) for a given record key.

Summary and Changelog

Users can now use the metadata lookup-record-index command in Hudi CLI to fetch record index entry details including partitionPath, fileId, and instantTimestamp for a given record key.

Changes:

Added getRecordIndexInfo() method in MetadataCommand.java to implement the new CLI command
Added testGetRecordIndexInfo() test case in TestMetadataCommand.java to verify the new functionality
Command validates that metadata table and record index partition are enabled before lookup

Impact

This is a new CLI command with no impact on existing functionality. It provides users with a convenient way to debug and inspect record index entries through the Hudi CLI.

Risk Level

Low - This is a new read-only CLI command that does not modify any data or change existing behavior. It only adds new functionality for querying metadata table.

Documentation Update

Documentation should be updated to include the new metadata lookup-record-index CLI command with usage examples.

Contributor's checklist

Read through contributor's guide
Enough context is provided in the sections above
Adequate tests were added if applicable

github-actions bot added the size:M label

nsivabalan reviewed

View reviewed changes

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Outdated

                 }
+                // @ShellOption(value = "--backup", help = "Backup the metadata table before delete", defaultValue = "true", arity = 1) final boolean backup
+                @ShellMethod(key = "metadata lookup-record-index", value = "Print Record index information for a record_key")

Contributor

nsivabalan Jan 28, 2026

should we support a list of record keys. comma separated.
Also, we have partitioned RLI support in latest 1.x.
We might need to take in partition path as input in such case.

even if we don't have bandwidth to support non global variant, lets check for RLI index definition and if its non global variant, we should throw exception for now.

For users who are looking to read records from MDT global RLI index, it should succeed.

Contributor Author

suryaprasanna Feb 1, 2026

Yes, we should support partitioned RLI as well. Added the necessary code for it.

suryaprasanna force-pushed the surya-dev-06 branch from f386786 to 3901452 Compare

January 30, 2026 19:02

nsivabalan reviewed

View reviewed changes

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Outdated

                   return HoodiePrintHelper.print(header, new HashMap<>(), "", false, Integer.MAX_VALUE, false, rows);
                 }
+                // @ShellOption(value = "--backup", help = "Backup the metadata table before delete", defaultValue = "true", arity = 1) final boolean backup

Contributor

nsivabalan Feb 1, 2026

why commented out. can we remove

Contributor Author

suryaprasanna Feb 1, 2026

Removing it.

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Outdated

                 }
+                // @ShellOption(value = "--backup", help = "Backup the metadata table before delete", defaultValue = "true", arity = 1) final boolean backup
+                @ShellMethod(key = "metadata lookup-record-index", value = "Print Record index information for a record_key")

Contributor

nsivabalan Feb 1, 2026

lets fix the documentation to call out, either a record key for global RLI or partition path and record key in case of partitioned RLI

Contributor Author

suryaprasanna Feb 1, 2026

Yes, fixed the documentation.

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Show resolved Hide resolved

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Outdated

+                public String getRecordIndexInfo(
+                    @ShellOption(value = "--record_key", help = "Record key entry whose info will be fetched")
+                    final String recordKey,
+                    @ShellOption(value = "--partition_path_for_non_global_rli", help = " Partition path needs to be provided for non Global or partition level Record index",

Contributor

nsivabalan Feb 1, 2026

we can just name this --partition_path.
in the documentation/desc of the option, we can call out the purpose.

Contributor Author

suryaprasanna Feb 1, 2026

I agree, making the change.

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Outdated

+                      metaReader.readRecordIndexLocationsWithKeys(HoodieListData.eager(Collections.singletonList(recordKey)), dataTablePartition);
+                  List<Pair<String, HoodieRecordGlobalLocation>> recordLocationKeyPair = recordKeyToGlobalLocationMap.collectAsList();
+                  if (recordLocationKeyPair.isEmpty()) {
+                    return "[INFO] Record key " + recordKey + " not found in Record Index";

Contributor

nsivabalan Feb 1, 2026

we should print the partition path as well incase of partitioned RLI

Contributor Author

suryaprasanna Feb 1, 2026

Adding it.

hudi-common/src/main/java/org/apache/hudi/common/config/HoodieMetadataConfig.java Show resolved Hide resolved

suryaprasanna force-pushed the surya-dev-06 branch from 3901452 to 6516a66 Compare

February 1, 2026 23:31

nsivabalan reviewed

View reviewed changes

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Outdated

+                      "[ERROR] Record index partition is not enabled/initialized\n\n");
+                  // Check if RLI is partitioned and validate partition_path is provided
+                  if (config.isRecordLevelIndexEnabled() && !config.isGlobalRecordLevelIndexEnabled()) {

Contributor

nsivabalan Feb 3, 2026

sorry. for hudi cli, we may not have all writer properties.
so we cant really rely on config.isGlobalRecordLevelIndexEnabled() or we can't rely on HoodieMetadataConfig only.

thats why I suggested to look into index definition. which the reader will have access to.

Contributor Author

suryaprasanna Feb 9, 2026

Yes, I am thinking of using indexMetadata to find out if the RLI is partitioned. I think if we can write that information we should be able to test it.

hudi-cli/src/main/java/org/apache/hudi/cli/commands/MetadataCommand.java Show resolved Hide resolved

hudi-cli/src/test/java/org/apache/hudi/cli/commands/TestMetadataCommand.java Outdated

+                }
+                @Test
+                public void testGetRecordIndexInfoForNonGlobalRLI() throws Exception {

Contributor

nsivabalan Feb 3, 2026

lets name the method testGetRecordIndexInfoForPartitionedRLI

hudi-cli/src/test/java/org/apache/hudi/cli/commands/TestMetadataCommand.java

+                    assertTrue(metaClient.getTableConfig().isMetadataPartitionAvailable(org.apache.hudi.metadata.MetadataPartitionType.RECORD_INDEX));
+                    // Validate entries in the Global RLI.
+                    validateRecordIndexOutput(recordKey, Option.empty(), newCommitTime, DEFAULT_FIRST_PARTITION_PATH);

Contributor

nsivabalan Feb 3, 2026

can we also try looking up a non existant record key.

hudi-cli/src/test/java/org/apache/hudi/cli/commands/TestMetadataCommand.java Show resolved Hide resolved

suryaprasanna force-pushed the surya-dev-06 branch from 6516a66 to d25c821 Compare

February 9, 2026 22:04

nsivabalan approved these changes

View reviewed changes

suryaprasanna added 6 commits

February 10, 2026 22:57


          Include metadata record_index command in Hudi-cli to fetch global fil…

9169d36

…e location

Summary: Using this command metadata table's record_index entry details like partitionPath, fileId and instantTimestamp can be fetched.


          Fix failed tests

31196d8


          Fix tests cases and add a method to enable RLI at partition level

9f407fb


          Fix failed unit test

f1a6177


          Address review comments

490a51a


          Address review comments

c197d8a

suryaprasanna force-pushed the surya-dev-06 branch from d25c821 to c197d8a Compare

February 10, 2026 22:57

Collaborator

hudi-bot commented Feb 11, 2026

CI report:

c197d8a Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

@hudi-bot run azure re-run the last Azure build

nsivabalan merged commit 3244013 into apache:master

72 checks passed

suryaprasanna deleted the surya-dev-06 branch

February 11, 2026 00:41

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M