HBASE-30161 Add paginated, single-RPC RegionLocator.getRegionLocations(startKey, limit) API for bulk meta-cache warmup#8237
Open
sanjeet006py wants to merge 3 commits into
Conversation
added 3 commits
May 14, 2026 22:21
…s(startKey, limit) API for bulk meta-cache warmup
… and add default-throws Review comments on apache#8236 by haridsv: - Adding an abstract method to RegionLocator / AsyncTableRegionLocator breaks external implementers. Convert the new method to a default that throws UnsupportedOperationException, with javadoc instructing callers to fall back to getAllRegionLocations(). - Reusing the getRegionLocations name overloads a method with completely different semantic (row -> all replicas of containing region). Rename to getRegionLocationsPage(byte[] startKey, int limit) to make the pagination/range intent explicit. - SnapshotRegionLocator no longer needs an override; it inherits the default-throws and callers fall back to getAllRegionLocations(). Tests renamed to call getRegionLocationsPage; semantic and assertions unchanged.
Address point apache#4 of the canvas/code review: the paged meta scan was making ceil(limit / hbase.meta.scanner.caching) ScannerNext RPCs whenever the limit exceeded the configured caching, contradicting the "at most one RPC per invocation" javadoc on RegionLocator.getRegionLocationsPage / AsyncTableRegionLocator.getRegionLocationsPage. Plumb a private isPagedScan flag through ClientMetaTableAccessor.scanMeta and getMetaScan. When true, size both setLimit and setCaching to the caller-supplied rowUpperLimit so the slice returns in a single ScannerNext RPC regardless of the configured caching default. The unbounded scanMeta overload (used by getTableHRegionLocations(metaTable, tableName) for getAllRegionLocations) keeps its existing behavior. Add TestRegionLocatorPagedScanRpcCount: starts a mini-cluster with hbase.meta.scanner.caching=2, wraps the meta AsyncTable to count AdvancedScanResultConsumer.onNext invocations, and asserts: - limit <= caching -> 1 ScannerNext (baseline) - limit > caching -> 1 ScannerNext (the regression check) - unbounded scan -> ceil(N / caching) ScannerNext (proves the flag does not leak into the existing path) Verified the test catches the regression: reverting the getMetaScan change makes testSingleRpcWhenLimitExceedsCaching fail with onNext=3.
sanjeet006py
pushed a commit
to sanjeet006py/hbase
that referenced
this pull request
May 15, 2026
…scans Mirrors the master-PR test (TestRegionLocatorPagedScanRpcCount on apache#8237). On the sync path we proxy: Connection.getTable(META_TABLE_NAME) -> Table.getScanner(Scan) -> ResultScanner.close() so we can flip Scan.setScanMetricsEnabled(true) on the way in and read ScanMetrics.countOfRPCcalls before the underlying scanner is closed. The RPC count is the natural sync analogue of the AdvancedScanResultConsumer.onNext invocation count we used on master. Cluster runs with hbase.meta.scanner.caching = 2 against a table of 5 user regions, asserting: - limit <= caching: 1 ScannerNext RPC (baseline) - limit > caching: 1 ScannerNext RPC (regression check; without the isPagedScan fix this would be ceil(5/2) = 3) - unbounded scan: ceil(NUM_REGIONS / caching) = 3 ScannerNext RPCs (proves the isPagedScan flag does not leak into the non-paged callers) Verified the test catches the regression: reverting the isPagedScan branch in MetaTableAccessor.getMetaScan makes testSingleRpcWhenLimitExceedsCaching fail with countOfRPCcalls = 3.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JIRA: HBASE-30161
Generated-by: Claude Opus 4.7