Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results by somandal · Pull Request #15050 · apache/pinot

somandal · 2025-02-13T01:27:20Z

This PR adds a summary option to the TableRebalance API which is meant to be used with dryRun. If summary is set to true a summary of the dry-run is returned rather than the full dry-run. This summary gives some stats about changes that will occur during the rebalance, such as:

Total number of segments to be moved
Unique and total number of segments
Replication factor
Number of servers
Server map to segment add/remove/unchanged information
Average segment size
Number of servers getting segments added
Total data size being moved (calculated based on average segment size - based on TableSize)
An estimate of how long the move can take (based on estimates to download and process each segment) - we should decide on a good throughput estimate as part of this PR review (left a TODO where the code is relevant)

Today the dry-run output can be very large, and can be difficult to make sense of in terms of the changes occurring. It can also be difficult to display the full output. For now summary is not appended to dryRun without summary enabled, but this can be added if it makes sense to add a summary even for the usual result return.

Note: This PR will have conflicts with #15029 and will need to be rebased once that is merged.

Sample JSON summary [NOTE - please see later comments for updated summary based on review comments]:

{
  "totalSegmentsToBeMoved" : 6,
  "numServersGettingNewSegments" : 1,
  "estimatedAverageSegmentSizeInBytes" : 1690546,
  "totalEstimatedDataToBeMovedInBytes" : 10143276,
  "totalEstimatedTimeToMoveDataInSecs" : 0.09673381805419921,
  "numServers" : {
    "_existingValue" : 1,
    "_newValue" : 2
  },
  "replicationFactor" : {
    "_existingValue" : 1,
    "_newValue" : 1
  },
  "numUniqueSegments" : {
    "_existingValue" : 12,
    "_newValue" : 12
  },
  "numTotalSegments" : {
    "_existingValue" : 12,
    "_newValue" : 12
  },
  "serverSegmentChangeInfo" : {
    "Server_localhost_22004" : {
      "_totalNewSegments" : 6,
      "_totalExistingSegments" : 0,
      "_segmentsAdded" : 6,
      "_segmentsDeleted" : 0,
      "_segmentsUnchanged" : 0
    },
    "Server_localhost_22001" : {
      "_totalNewSegments" : 6,
      "_totalExistingSegments" : 12,
      "_segmentsAdded" : 0,
      "_segmentsDeleted" : 6,
      "_segmentsUnchanged" : 6
    }
  }
}

cc @Jackie-Jiang @klsince @deepthi912 @npawar

codecov-commenter · 2025-02-13T02:11:32Z

Codecov Report

Attention: Patch coverage is 87.68473% with 25 lines in your changes missing coverage. Please review.

Project coverage is 63.42%. Comparing base (59551e4) to head (6a04c66).
Report is 1740 commits behind head on master.

Files with missing lines	Patch %	Lines
...r/helix/core/rebalance/RebalanceSummaryResult.java	80.64%	12 Missing ⚠️
...ntroller/helix/core/rebalance/TableRebalancer.java	92.62%	3 Missing and 6 partials ⚠️
...oller/api/resources/PinotTableRestletResource.java	33.33%	2 Missing ⚠️
...ntroller/helix/core/rebalance/RebalanceConfig.java	83.33%	1 Missing ⚠️
...x/core/rebalance/tenant/TenantRebalanceResult.java	0.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##             master   #15050      +/-   ##
============================================
+ Coverage     61.75%   63.42%   +1.66%     
- Complexity      207     1481    +1274     
============================================
  Files          2436     2745     +309     
  Lines        133233   154271   +21038     
  Branches      20636    23803    +3167     
============================================
+ Hits          82274    97839   +15565     
- Misses        44911    49041    +4130     
- Partials       6048     7391    +1343

Flag	Coverage Δ
custom-integration1	`100.00% <ø> (+99.99%)`	⬆️
integration	`100.00% <ø> (+99.99%)`	⬆️
integration1	`100.00% <ø> (+99.99%)`	⬆️
integration2	`0.00% <ø> (ø)`
java-11	`63.40% <87.68%> (+1.69%)`	⬆️
java-21	`63.31% <87.68%> (+1.69%)`	⬆️
skip-bytebuffers-false	`63.41% <87.68%> (+1.67%)`	⬆️
skip-bytebuffers-true	`63.28% <87.68%> (+35.55%)`	⬆️
temurin	`63.42% <87.68%> (+1.66%)`	⬆️
unittests	`63.41% <87.68%> (+1.66%)`	⬆️
unittests1	`56.03% <ø> (+9.14%)`	⬆️
unittests2	`33.95% <87.68%> (+6.22%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

npawar · 2025-02-16T04:12:14Z

This PR adds a summary option to the TableRebalance API which is meant to be used with dryRun. If summary is set to true a summary of the dry-run is returned rather than the full dry-run. This summary gives some stats about changes that will occur during the rebalance, such as:

Total number of segments to be moved

Unique and total number of segments

Replication factor

Number of servers

Server map to segment add/remove/unchanged information

Average segment size

Number of servers getting segments added

Total data size being moved (calculated based on average segment size - based on TableSize)

An estimate of how long the move can take (based on estimates to download and process each segment) - we should decide on a good throughput estimate as part of this PR review (left a TODO where the code is relevant)

Today the dry-run output can be very large, and can be difficult to make sense of in terms of the changes occurring. It can also be difficult to display the full output. For now summary is not appended to dryRun without summary enabled, but this can be added if it makes sense to add a summary even for the usual result return.

Note: This PR will have conflicts with #15029 and will need to be rebased once that is merged.

Sample JSON summary:
{
  "totalSegmentsToBeMoved" : 6,
  "numServersGettingNewSegments" : 1,
  "estimatedAverageSegmentSizeInBytes" : 1690546,
  "totalEstimatedDataToBeMovedInBytes" : 10143276,
  "totalEstimatedTimeToMoveDataInSecs" : 0.09673381805419921,
  "numServers" : {
    "_existingValue" : 1,
    "_newValue" : 2
  },
  "replicationFactor" : {
    "_existingValue" : 1,
    "_newValue" : 1
  },
  "numUniqueSegments" : {
    "_existingValue" : 12,
    "_newValue" : 12
  },
  "numTotalSegments" : {
    "_existingValue" : 12,
    "_newValue" : 12
  },
  "serverSegmentChangeInfo" : {
    "Server_localhost_22004" : {
      "_totalNewSegments" : 6,
      "_totalExistingSegments" : 0,
      "_segmentsAdded" : 6,
      "_segmentsDeleted" : 0,
      "_segmentsUnchanged" : 0
    },
    "Server_localhost_22001" : {
      "_totalNewSegments" : 6,
      "_totalExistingSegments" : 12,
      "_segmentsAdded" : 0,
      "_segmentsDeleted" : 6,
      "_segmentsUnchanged" : 6
    }
  }
}
cc @Jackie-Jiang @klsince @deepthi912 @npawar

A few comments on the summary. Feel free to just take those that make sense for this first iteration:

along with numServers, it might be useful to see the list there of existing and new, so operator can confirm that their tagging / untagging is effective (so a servers added / removed / unchanged ?)
how about showing what tenant tag we're operating with? so summarize the tags from tenants, completed, tier, pools
what does num unique segments mean? same with numTotalSegments, didn't follow what existing/new value means in context of rebalance. Perhaps having a description field within the sections will help.
in the server to stats map for 22001, if it started with 12 and 6 are moving, shouldn't totalNewSegments be 0?
this payload will get pretty extensive over time. wondering if we should take time to do some more top level categorization - segments related info, servers related info, generic info

…mmary of the dry-run results

somandal · 2025-02-18T15:23:05Z

Updated summary:

Adding a server:

{
  "serverInfo" : {
    "numServersGettingNewSegments" : 1,
    "numServers" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 2
    },
    "serverSegmentChangeInfo" : {
      "Server_localhost_22004" : {
        "serverStatus" : "ADDED",
        "totalSegmentsAfterRebalance" : 6,
        "totalSegmentsBeforeRebalance" : 0,
        "segmentsAdded" : 6,
        "segmentsDeleted" : 0,
        "segmentsUnchanged" : 0,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      },
      "Server_localhost_22001" : {
        "serverStatus" : "UNCHANGED",
        "totalSegmentsAfterRebalance" : 6,
        "totalSegmentsBeforeRebalance" : 12,
        "segmentsAdded" : 0,
        "segmentsDeleted" : 6,
        "segmentsUnchanged" : 6,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      }
    }
  },
  "segmentInfo" : {
    "totalSegmentsToBeMoved" : 6,
    "estimatedAverageSegmentSizeInBytes" : 1690546,
    "totalEstimatedDataToBeMovedInBytes" : 10143276,
    "totalEstimatedTimeToMoveDataInSecs" : 0.09673381805419921,
    "replicationFactor" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 1
    },
    "numSegmentsInSingleReplica" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    },
    "numSegmentsAcrossAllReplicas" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    }
  }
}

Removing a server:

{
  "serverInfo" : {
    "numServersGettingNewSegments" : 1,
    "numServers" : {
      "valueBeforeRebalance" : 2,
      "expectedValueAfterRebalance" : 1
    },
    "serverSegmentChangeInfo" : {
      "Server_localhost_22004" : {
        "serverStatus" : "REMOVED",
        "totalSegmentsAfterRebalance" : 0,
        "totalSegmentsBeforeRebalance" : 6,
        "segmentsAdded" : 0,
        "segmentsDeleted" : 6,
        "segmentsUnchanged" : 0,
        "tagList" : [ ]
      },
      "Server_localhost_22001" : {
        "serverStatus" : "UNCHANGED",
        "totalSegmentsAfterRebalance" : 12,
        "totalSegmentsBeforeRebalance" : 6,
        "segmentsAdded" : 6,
        "segmentsDeleted" : 0,
        "segmentsUnchanged" : 6,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      }
    }
  },
  "segmentInfo" : {
    "totalSegmentsToBeMoved" : 6,
    "estimatedAverageSegmentSizeInBytes" : 1690546,
    "totalEstimatedDataToBeMovedInBytes" : 10143276,
    "totalEstimatedTimeToMoveDataInSecs" : 0.09673381805419921,
    "replicationFactor" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 1
    },
    "numSegmentsInSingleReplica" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    },
    "numSegmentsAcrossAllReplicas" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    }
  }
}

A few comments on the summary. Feel free to just take those that make sense for this first iteration:

along with numServers, it might be useful to see the list there of existing and new, so operator can confirm that their tagging / untagging is effective (so a servers added / removed / unchanged ?)

how about showing what tenant tag we're operating with? so summarize the tags from tenants, completed, tier, pools

what does num unique segments mean? same with numTotalSegments, didn't follow what existing/new value means in context of rebalance. Perhaps having a description field within the sections will help.

in the server to stats map for 22001, if it started with 12 and 6 are moving, shouldn't totalNewSegments be 0?

this payload will get pretty extensive over time. wondering if we should take time to do some more top level categorization - segments related info, servers related info, generic info

@npawar responses regarding the above are below:

done, added a server status for each server
I've added the tenant tags for each server - is this what you meant? Or do you think just dumping the list of all tenant tags across servers is good enough? Or did you want finer categorization based on whether it's OFFLINE/CONSUMING/COMPLETED/TIER etc? For more categorization, we can add it as part of a future change?
by unique segments -> i meant number of segments per replica. I've updated the name of the field in the new summary. Total segments was the total number of segments across all replicas. Existing was the value before rebalance, and the value after expected rebalance. I've updated the fields in the summary. Let me know if that makes more sense or if we still need a description field
here totalNewSegments was total segments hosted after the rebalance completes. So 22001 was losing 6 of the existing 12 segments. I've updated the field to better explain this.
I've attempted a categorization into server vs. segments. Please take a look and let me know what you think

klsince · 2025-02-18T22:28:05Z

+      long totalDataPerServerInBytes = totalEstimatedDataToBeMovedInBytes / numServersGettingDataAdded;
+      // TODO: Pick a good threshold to calculate estimated time to rebalance. For now assume 100 MB/s data download
+      //       + process rate
+      estimatedTimeToRebalanceInSec = ((double) totalDataPerServerInBytes) / (100.0D * 1024.0D * 1024.0D);


I think maybe we skip this field, as it's not easy to estimate this accurately because the cluster setup can vary a lot. The summary contains the total amount of segments and data bytes a server may get now, so the user can estimate the duration for their envs accordingly.

yeah, this is a tricky field for sure. I've removed it for now, but added a TODO for adding it back once we have a better understanding of a good estimate. This field was an ask for this summary PR.

klsince · 2025-02-18T22:32:17Z

+    RebalanceSummaryResult summaryResult = null;
+    if (summary) {
+      summaryResult = calculateDryRunSummary(currentAssignment, targetAssignment, tableNameWithType, rebalanceJobId);
+    }


how about calculating the summary after this whole if-block, so we save the cost from calculating it when the table is already balanced?

if (segmentAssignmentUnchanged) { .. }

I think it might still be useful to see the summary in case the table being balanced is not expected? wdyt?
I haven't moved it yet, but let me know if you still think it's not worth calculating the summary for this scenario and I can move it down and adjust the tests.

make sense. Perhaps comment this for this if-branch so that others know the purpose and keep it here.

npawar · 2025-02-18T23:56:18Z

Updated summary:

Adding a server:
{
  "serverInfo" : {
    "numServersGettingNewSegments" : 1,
    "numServers" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 2
    },
    "serverSegmentChangeInfo" : {
      "Server_localhost_22004" : {
        "serverStatus" : "ADDED",
        "totalSegmentsAfterRebalance" : 6,
        "totalSegmentsBeforeRebalance" : 0,
        "segmentsAdded" : 6,
        "segmentsDeleted" : 0,
        "segmentsUnchanged" : 0,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      },
      "Server_localhost_22001" : {
        "serverStatus" : "UNCHANGED",
        "totalSegmentsAfterRebalance" : 6,
        "totalSegmentsBeforeRebalance" : 12,
        "segmentsAdded" : 0,
        "segmentsDeleted" : 6,
        "segmentsUnchanged" : 6,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      }
    }
  },
  "segmentInfo" : {
    "totalSegmentsToBeMoved" : 6,
    "estimatedAverageSegmentSizeInBytes" : 1690546,
    "totalEstimatedDataToBeMovedInBytes" : 10143276,
    "totalEstimatedTimeToMoveDataInSecs" : 0.09673381805419921,
    "replicationFactor" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 1
    },
    "numSegmentsInSingleReplica" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    },
    "numSegmentsAcrossAllReplicas" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    }
  }
}
Removing a server:
{
  "serverInfo" : {
    "numServersGettingNewSegments" : 1,
    "numServers" : {
      "valueBeforeRebalance" : 2,
      "expectedValueAfterRebalance" : 1
    },
    "serverSegmentChangeInfo" : {
      "Server_localhost_22004" : {
        "serverStatus" : "REMOVED",
        "totalSegmentsAfterRebalance" : 0,
        "totalSegmentsBeforeRebalance" : 6,
        "segmentsAdded" : 0,
        "segmentsDeleted" : 6,
        "segmentsUnchanged" : 0,
        "tagList" : [ ]
      },
      "Server_localhost_22001" : {
        "serverStatus" : "UNCHANGED",
        "totalSegmentsAfterRebalance" : 12,
        "totalSegmentsBeforeRebalance" : 6,
        "segmentsAdded" : 6,
        "segmentsDeleted" : 0,
        "segmentsUnchanged" : 6,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      }
    }
  },
  "segmentInfo" : {
    "totalSegmentsToBeMoved" : 6,
    "estimatedAverageSegmentSizeInBytes" : 1690546,
    "totalEstimatedDataToBeMovedInBytes" : 10143276,
    "totalEstimatedTimeToMoveDataInSecs" : 0.09673381805419921,
    "replicationFactor" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 1
    },
    "numSegmentsInSingleReplica" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    },
    "numSegmentsAcrossAllReplicas" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    }
  }
}
A few comments on the summary. Feel free to just take those that make sense for this first iteration:

along with numServers, it might be useful to see the list there of existing and new, so operator can confirm that their tagging / untagging is effective (so a servers added / removed / unchanged ?)

how about showing what tenant tag we're operating with? so summarize the tags from tenants, completed, tier, pools

what does num unique segments mean? same with numTotalSegments, didn't follow what existing/new value means in context of rebalance. Perhaps having a description field within the sections will help.

in the server to stats map for 22001, if it started with 12 and 6 are moving, shouldn't totalNewSegments be 0?

this payload will get pretty extensive over time. wondering if we should take time to do some more top level categorization - segments related info, servers related info, generic info

@npawar responses regarding the above are below:

done, added a server status for each server

I've added the tenant tags for each server - is this what you meant? Or do you think just dumping the list of all tenant tags across servers is good enough? Or did you want finer categorization based on whether it's OFFLINE/CONSUMING/COMPLETED/TIER etc? For more categorization, we can add it as part of a future change?

by unique segments -> i meant number of segments per replica. I've updated the name of the field in the new summary. Total segments was the total number of segments across all replicas. Existing was the value before rebalance, and the value after expected rebalance. I've updated the fields in the summary. Let me know if that makes more sense or if we still need a description field

here totalNewSegments was total segments hosted after the rebalance completes. So 22001 was losing 6 of the existing 12 segments. I've updated the field to better explain this.

I've attempted a categorization into server vs. segments. Please take a look and let me know what you think

points 3,4,5 look great.
point 2 - sure we can take this up later
For 1, the tag in each server's section is helpful. What I was suggesting though, is that we provide the list right on top (Essentially 4 lists called serversGettingNewSegments, serverAdded, serverRemoved, serverUnchanged) as when we'll get to clusters with 50-100 servers, parsing through 50-100 individual json blurbs per server might get tricky.

… and servers getting new segments

somandal · 2025-02-19T00:30:13Z

points 3,4,5 look great.
point 2 - sure we can take this up later
For 1, the tag in each server's section is helpful. What I was suggesting though, is that we provide the list right on top (Essentially 4 lists called serversGettingNewSegments, serverAdded, serverRemoved, serverUnchanged) as when we'll get to clusters with 50-100 servers, parsing through 50-100 individual json blurbs per server might get tricky.

Got it, updated the PR to add your suggestion for 1.

Sample summary:

Add server:

{
  "serverInfo" : {
    "numServersGettingNewSegments" : 1,
    "numServers" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 2
    },
    "serversAdded" : [ "Server_localhost_22004" ],
    "serversRemoved" : [ ],
    "serversUnchanged" : [ "Server_localhost_22001" ],
    "serversGettingNewSegments" : [ "Server_localhost_22004" ],
    "serverSegmentChangeInfo" : {
      "Server_localhost_22004" : {
        "serverStatus" : "ADDED",
        "totalSegmentsAfterRebalance" : 6,
        "totalSegmentsBeforeRebalance" : 0,
        "segmentsAdded" : 6,
        "segmentsDeleted" : 0,
        "segmentsUnchanged" : 0,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      },
      "Server_localhost_22001" : {
        "serverStatus" : "UNCHANGED",
        "totalSegmentsAfterRebalance" : 6,
        "totalSegmentsBeforeRebalance" : 12,
        "segmentsAdded" : 0,
        "segmentsDeleted" : 6,
        "segmentsUnchanged" : 6,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      }
    }
  },
  "segmentInfo" : {
    "totalSegmentsToBeMoved" : 6,
    "maxSegmentsAddedToASingleServer" : 6,
    "estimatedAverageSegmentSizeInBytes" : 1690546,
    "totalEstimatedDataToBeMovedInBytes" : 10143276,
    "replicationFactor" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 1
    },
    "numSegmentsInSingleReplica" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    },
    "numSegmentsAcrossAllReplicas" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    }
  }
}

Remove server:

{
  "serverInfo" : {
    "numServersGettingNewSegments" : 1,
    "numServers" : {
      "valueBeforeRebalance" : 2,
      "expectedValueAfterRebalance" : 1
    },
    "serversAdded" : [ ],
    "serversRemoved" : [ "Server_localhost_22004" ],
    "serversUnchanged" : [ "Server_localhost_22001" ],
    "serversGettingNewSegments" : [ "Server_localhost_22001" ],
    "serverSegmentChangeInfo" : {
      "Server_localhost_22004" : {
        "serverStatus" : "REMOVED",
        "totalSegmentsAfterRebalance" : 0,
        "totalSegmentsBeforeRebalance" : 6,
        "segmentsAdded" : 0,
        "segmentsDeleted" : 6,
        "segmentsUnchanged" : 0,
        "tagList" : [ ]
      },
      "Server_localhost_22001" : {
        "serverStatus" : "UNCHANGED",
        "totalSegmentsAfterRebalance" : 12,
        "totalSegmentsBeforeRebalance" : 6,
        "segmentsAdded" : 6,
        "segmentsDeleted" : 0,
        "segmentsUnchanged" : 6,
        "tagList" : [ "DefaultTenant_OFFLINE", "DefaultTenant_REALTIME" ]
      }
    }
  },
  "segmentInfo" : {
    "totalSegmentsToBeMoved" : 6,
    "maxSegmentsAddedToASingleServer" : 6,
    "estimatedAverageSegmentSizeInBytes" : 1690546,
    "totalEstimatedDataToBeMovedInBytes" : 10143276,
    "replicationFactor" : {
      "valueBeforeRebalance" : 1,
      "expectedValueAfterRebalance" : 1
    },
    "numSegmentsInSingleReplica" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    },
    "numSegmentsAcrossAllReplicas" : {
      "valueBeforeRebalance" : 12,
      "expectedValueAfterRebalance" : 12
    }
  }
}

Also note that I've removed the time estimates for now as recommended by @klsince as it is hard to come up with a good estimate (different environments may have different speeds). We can perhaps come up with some heuristic based on some real rebalances if that makes sense in the future.

klsince · 2025-02-19T18:50:20Z

+    RebalanceSummaryResult summaryResult = null;
+    if (summary) {
+      summaryResult = calculateDryRunSummary(currentAssignment, targetAssignment, tableNameWithType, rebalanceJobId);
+    }


make sense. Perhaps comment this for this if-branch so that others know the purpose and keep it here.

…mmary of the dry-run results (apache#15050) * Add a dry-run summary mode for TableRebalance which only returns a summary of the dry-run results * Address review - add server level lists of added, removed, unchanged, and servers getting new segments

somandal force-pushed the rebalance-phase-1-dry-run-summary branch from 2449cc2 to aa3fa91 Compare February 14, 2025 21:17

somandal added 3 commits February 18, 2025 07:16

Add a dry-run summary mode for TableRebalance which only returns a su…

986950d

…mmary of the dry-run results

Add table size option

6085faa

Address review comments to improve summary

35ba42c

somandal force-pushed the rebalance-phase-1-dry-run-summary branch from 447752e to 35ba42c Compare February 18, 2025 15:16

Trigger Build

beb5c9b

klsince reviewed Feb 18, 2025

View reviewed changes

Address review comments

3ce1cce

Address review - add server level lists of added, removed, unchanged,…

20060fa

… and servers getting new segments

somandal requested a review from klsince February 19, 2025 00:30

klsince approved these changes Feb 19, 2025

View reviewed changes

Address review comments

6a04c66

somandal requested a review from klsince February 19, 2025 19:13

klsince approved these changes Feb 19, 2025

View reviewed changes

klsince merged commit 31e94d3 into apache:master Feb 19, 2025

somandal deleted the rebalance-phase-1-dry-run-summary branch February 19, 2025 21:38

yashmayya added the segment-rebalance Related to segment rebalancing across servers label Mar 12, 2025

Uh oh!

Conversation

somandal commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov-commenter commented Feb 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

npawar commented Feb 16, 2025

Uh oh!

somandal commented Feb 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

klsince Feb 18, 2025

Choose a reason for hiding this comment

Uh oh!

somandal Feb 18, 2025

Choose a reason for hiding this comment

Uh oh!

klsince Feb 18, 2025

Choose a reason for hiding this comment

Uh oh!

somandal Feb 18, 2025

Choose a reason for hiding this comment

Uh oh!

klsince Feb 19, 2025

Choose a reason for hiding this comment

Uh oh!

somandal Feb 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

npawar commented Feb 18, 2025

Uh oh!

somandal commented Feb 19, 2025

Uh oh!

Uh oh!

klsince Feb 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

somandal commented Feb 13, 2025 •

edited

Loading

codecov-commenter commented Feb 13, 2025 •

edited

Loading

somandal commented Feb 18, 2025 •

edited

Loading