HBASE-30051 Avoid Unnecessary Protobuf Conversions in HMaster Admin API Calls

[srinireddy2020]

Admin APIs invoked on the HMaster are routed internally back to the same HMaster instance, leading to unnecessary protobuf serialization and deserialization. This adds significant overhead, especially in environments with a large number of tables.

[Apache9]

Do you have any performance numbers about the overhead?

[srinireddy2020]

Do you have any performance numbers about the overhead?

Verified performance test in HBase Mini cluster with 350 tables. Observed 97.5% improvement for this changes.

Method: admin.listTableDescriptors(true)
Iterations: 100
Avg Time: 10311891.82 nanos
Min Time: 6341330 nanos
Max Time: 119611397 nanos
Median Time: 7631084 nanos
P95 Time: 16353979 nanos
P99 Time: 119611397 nanos

Method: master.listTableDescriptorsByNamespace(null)
Iterations: 100
Avg Time: 253733.7 nanos
Min Time: 197507 nanos
Max Time: 511075 nanos
Median Time: 224771 nanos
P95 Time: 444529 nanos
P99 Time: 511075 nanos

[Apache9]

I mean the performance change in real workload.

I see that you have changed two places, what is the performance numbers for these two places before and after this PR?

[srinireddy2020]

I mean the performance change in real workload.

I see that you have changed two places, what is the performance numbers for these two places before and after this PR?

I validated the performance test on a real HBase cluster with 400 tables. Here are the observations:

For rsGroup changes observed 42.5% improvement.

For RSGroup (with out fix)
Iterations: 100
Avg Time: 41175178.39 nanos
Min Time: 29244928 nanos
Max Time: 81204085 nanos
Median Time: 37750862 nanos
P95 Time: 59508779 nanos
P99 Time: 69041148 nanos

For RSGroup (with fix)
Iterations: 100
Avg Time: 23654449.31 nanos
Min Time: 16609079 nanos
Max Time: 36157928 nanos
Median Time: 22623794 nanos
P95 Time: 32494615 nanos
P99 Time: 35838576 nanos

For SnapshotQuotaObserverChore observed 40.6% improvement

For SnapshotQuota (with out fix)
Iterations: 100
Avg Time: 43582296.45 nanos
Min Time: 31566942 nanos
Max Time: 82250476 nanos
Median Time: 38645752 nanos
P95 Time: 62457758 nanos
P99 Time: 70146689 nanos

For SnapshotQuota (with fix)
Iterations: 100
Avg Time: 25858876.45 nanos
Min Time: 17589768 nanos
Max Time: 38564598 nanos
Median Time: 23498268 nanos
P95 Time: 33386064 nanos
P99 Time: 37569430 nanos