Account GroupSizeInUnits in TGroupFitter #19337
Conversation
|
🟢 |
rosik
changed the title
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
rosik
force-pushed
the
size-in-units-mvp-4
branch
from
685cc2f to
f952e1f
Compare
rosik
force-pushed
the
size-in-units-mvp-4
branch
2 times, most recently
from
9378599 to
a55b92b
Compare
There was a problem hiding this comment.
Pull Request Overview
This PR introduces new distributed storage parameters—GroupSizeInUnits and SlotSizeInUnits—in multiple components of the system. Key changes include:
- Updates to proto, whiteboard, and VDisk state reporting to propagate the new unit sizes.
- Adjustments to group allocation and sanitization functions to accept and apply the new GroupSizeInUnits parameter.
- Updates to unit tests and PDisk components ensuring that group size accounting is integrated throughout.
Reviewed Changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| ydb/core/protos/node_whiteboard.proto | Added new fields SlotSizeInUnits and GroupSizeInUnits to PVdisk and BSGroup messages for whiteboard reporting. |
| ydb/core/node_whiteboard/node_whiteboard.h | Updated TEvVDiskStateUpdate constructor to include groupSizeInUnits in whiteboard updates. |
| ydb/core/mind/bscontroller/* | Modified AllocateGroup, FindMisplacedVDisks, and related APIs to include the new groupSizeInUnits parameter. |
| ydb/core/mind/bscontroller/group_mapper_ut.cpp | Updated test cases to pass appropriate groupSizeInUnits values. |
| ydb/core/blobstorage/* and ydb/core/blobstorage/pdisk/* | Propagated the new parameters through PDisk parameter initialization, mock state, and reporting. |
| ydb/core/blobstorage/nodewarden/* | Updated VDisk state reporting to include GroupSizeInUnits. |
| ydb/core/blobstorage/distconf_generate.cpp | Adjusted group allocation calls to pass an explicit groupSizeInUnits value. |
Comments suppressed due to low confidence (1)
ydb/core/mind/bscontroller/group_mapper.h:102
- Consider updating the function documentation and comments to clearly explain the purpose and expected values for the new 'groupSizeInUnits' parameter in the API.
bool AllocateGroup(ui32 groupId, TGroupDefinition& group, TGroupMapper::TGroupConstraintsDefinition& constraints,
rosik
added
the
rebase-and-check
github-actions
bot
removed
the
rebase-and-check
rosik
force-pushed
the
size-in-units-mvp-4
branch
from
a55b92b to
95e3182
Compare
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
|
⚪ Test history | Ya make output | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat
⚪ Test history | Ya make output | Test bloat | Test bloat | Test bloat
🟢
*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation |
Besides counters this PR includes 2 other minor changes:
- Fix squeezed tablet counters in Developer UI html
- Remove redundant OwnerWeight from YardInitResult
During VDisk initialization PDisk replies to Skeleton with TYardInitResult event containing TPDiskParams and Skeleton saves it in its context. In #19337 it was extended with SlotSizeInUnits and OwnerWeight field. Now OwnerWeight is removed from there for two reasons. It's actually isn't used anywhere but in html rendering. And, what is more important, OwnerWeight can be changed with ChangeGroupSizeInUnits command, but PDiskParams in skeleton context is a constant field.
Besides counters this PR includes 2 other minor changes:
- Fix squeezed tablet counters in Developer UI html
- Remove redundant OwnerWeight from YardInitResult
During VDisk initialization PDisk replies to Skeleton with TYardInitResult event containing TPDiskParams and Skeleton saves it in its context. In ydb-platform#19337 it was extended with SlotSizeInUnits and OwnerWeight field. Now OwnerWeight is removed from there for two reasons. It's actually isn't used anywhere but in html rendering. And, what is more important, OwnerWeight can be changed with ChangeGroupSizeInUnits command, but PDiskParams in skeleton context is a constant field.
Besides counters this PR includes 2 other minor changes:
- Fix squeezed tablet counters in Developer UI html
- Remove redundant OwnerWeight from YardInitResult
During VDisk initialization PDisk replies to Skeleton with TYardInitResult event containing TPDiskParams and Skeleton saves it in its context. In ydb-platform#19337 it was extended with SlotSizeInUnits and OwnerWeight field. Now OwnerWeight is removed from there for two reasons. It's actually isn't used anywhere but in html rendering. And, what is more important, OwnerWeight can be changed with ChangeGroupSizeInUnits command, but PDiskParams in skeleton context is a constant field.
Changelog entry
GroupSizeInUnitsproperty of storage groups andSlotSizeInUnitsof PDisks. Docs: User documentation for pdisks of different sizes #19364Changelog category
Description for reviewers
This PR includes the following changes:
TGroupFitteraccounts GroupSizeInUnits when creating new groups or sanitizing existing onesVDisk.PDiskParamsnow reports OwnerWeigt and SlotSizeInUnitsNodeWhiteboardnow reports corresponding new properties of pdisk, vdisk and bsgroup, it's testedTGroupMapperTestis updated to account new NumSlots calculation logics and cover new cases