Skip to content

Conversation

@rosik
Copy link
Collaborator

@rosik rosik commented Nov 6, 2025

Changelog category

  • Not for changelog (changelog entry is not required)

Description for reviewers

Add capacity metrics to whiteboard according to RFC 164_capacity_metrics.md

@github-actions
Copy link

github-actions bot commented Nov 6, 2025

🟢 2025-11-06 17:34:32 UTC The validation of the Pull Request description is successful.

@rosik rosik force-pushed the capacity-metrics-2 branch 3 times, most recently from 935da10 to 0ba7668 Compare November 6, 2025 18:12
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 6, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 6, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 6, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 6, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 6, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 6, 2025
@rosik rosik force-pushed the capacity-metrics-2 branch from 0ba7668 to e5c30ee Compare November 6, 2025 18:15
@github-actions
Copy link

github-actions bot commented Nov 6, 2025

2025-11-06 18:17:15 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4f804e2 has started.
2025-11-06 18:17:32 UTC Artifacts will be uploaded here
2025-11-06 18:19:50 UTC ya make is running...
🟡 2025-11-06 20:10:34 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41619 38800 0 2 2791 26

2025-11-06 20:10:58 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-11-06 20:19:53 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
53 (only retried tests) 39 0 0 0 14

🟢 2025-11-06 20:20:02 UTC Build successful.
🟢 2025-11-06 20:20:21 UTC ydbd size 2.3 GiB changed* by +14.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0daa817 merge: 4f804e2 diff diff %
ydbd size 2 431 199 208 Bytes 2 431 214 256 Bytes +14.7 KiB +0.001%
ydbd stripped size 516 413 392 Bytes 516 417 296 Bytes +3.8 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 6, 2025

2025-11-06 18:17:17 UTC Pre-commit check linux-x86_64-release-asan for 4f804e2 has started.
2025-11-06 18:17:34 UTC Artifacts will be uploaded here
2025-11-06 18:19:50 UTC ya make is running...
🟡 2025-11-06 20:47:43 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
18152 17748 0 158 214 32

🟢 2025-11-06 20:47:58 UTC Build successful.
🟢 2025-11-06 20:48:24 UTC ydbd size 3.8 GiB changed* by +28.4 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0daa817 merge: 4f804e2 diff diff %
ydbd size 4 071 937 656 Bytes 4 071 966 712 Bytes +28.4 KiB +0.001%
ydbd stripped size 1 511 446 344 Bytes 1 511 460 104 Bytes +13.4 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@rosik rosik marked this pull request as ready for review November 6, 2025 18:37
@rosik rosik requested review from a team as code owners November 6, 2025 18:37
@rosik rosik requested review from Enjection and Copilot November 6, 2025 18:37
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces additional disk space metrics to provide more granular visibility into PDisk and VDisk capacity utilization. The changes rename existing fields to better reflect their purpose and add new metrics to distinguish between different types of space measurements.

  • Renamed Occupancy to NormalizedOccupancy and SpaceColor to CapacityAlertLevel to clarify their semantics
  • Added three new metrics: VDiskSlotUtilization, VDiskFairPartFillPercent, and PDiskFillPercent to provide different perspectives on disk space usage
  • Updated tests to validate the new metrics with comprehensive assertions

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
ydb/core/protos/node_whiteboard.proto Added new disk space metric fields to PDisk and VDisk state info messages
ydb/core/protos/blobstorage_disk.proto Renamed existing fields and added new metric fields to VDisk and PDisk metrics messages
ydb/core/mind/hive/hive_ut.cpp Updated test to use renamed NormalizedOccupancy field
ydb/core/mind/bscontroller/bsc.cpp Updated references to use renamed fields NormalizedOccupancy and CapacityAlertLevel
ydb/core/blobstorage/vdisk/skeleton/skeleton_oos_tracker.cpp Updated to use renamed NormalizedOccupancy field
ydb/core/blobstorage/pdisk/mock/pdisk_mock.cpp Updated mock to use renamed field
ydb/core/blobstorage/pdisk/blobstorage_pdisk_ut.cpp Extended tests with comprehensive validation of all new metrics, fixed comment typos
ydb/core/blobstorage/pdisk/blobstorage_pdisk_quota_record.h Added const qualifier to ColorFlagLimit method
ydb/core/blobstorage/pdisk/blobstorage_pdisk_keeper.h Extended GetSpaceStatusFlags signature to support new metric output parameters
ydb/core/blobstorage/pdisk/blobstorage_pdisk_impl.h Extended GetStatusFlags signature to support new metric output parameters
ydb/core/blobstorage/pdisk/blobstorage_pdisk_impl.cpp Implemented calculation and reporting of new metrics in WhiteboardReport and CheckSpace
ydb/core/blobstorage/pdisk/blobstorage_pdisk_color_limits.h Added const qualifier to GetOccupancyForColor method
ydb/core/blobstorage/pdisk/blobstorage_pdisk_chunk_tracker.h Implemented calculation logic for new metrics, added const qualifiers, added GetOccupancyForColor helper
ydb/core/blobstorage/pdisk/blobstorage_pdisk.h Added new metric fields to TEvCheckSpaceResult event structure
ydb/core/blobstorage/nodewarden/blobstorage_node_warden_ut.cpp Added assertions to validate PDiskFillPercent field in tests

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@rosik rosik force-pushed the capacity-metrics-2 branch 2 times, most recently from 46f5950 to afe61a8 Compare November 7, 2025 11:57
@rosik rosik force-pushed the capacity-metrics-2 branch 2 times, most recently from 8f18eec to 9b72331 Compare November 7, 2025 14:00
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 7, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 7, 2025
@rosik rosik force-pushed the capacity-metrics-2 branch from 13d8ebe to 044c232 Compare November 10, 2025 08:42
@github-actions
Copy link

github-actions bot commented Nov 10, 2025

2025-11-10 08:43:23 UTC Pre-commit check linux-x86_64-release-asan for 91da091 has started.
2025-11-10 08:43:27 UTC Artifacts will be uploaded here
2025-11-10 08:44:54 UTC ya make is running...
🟡 2025-11-10 11:22:31 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
18205 17736 0 175 267 27

🟢 2025-11-10 11:22:46 UTC Build successful.
🟢 2025-11-10 11:23:15 UTC ydbd size 3.8 GiB changed* by +34.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 56f5415 merge: 91da091 diff diff %
ydbd size 4 080 343 792 Bytes 4 080 378 840 Bytes +34.2 KiB +0.001%
ydbd stripped size 1 514 662 856 Bytes 1 514 677 000 Bytes +13.8 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 10, 2025

2025-11-10 08:44:10 UTC Pre-commit check linux-x86_64-relwithdebinfo for 91da091 has started.
2025-11-10 08:44:27 UTC Artifacts will be uploaded here
2025-11-10 08:46:45 UTC ya make is running...
🟡 2025-11-10 10:54:59 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
41689 38852 0 1 2807 29

2025-11-10 10:55:19 UTC ya make is running... (failed tests rerun, try 2)
🟢 2025-11-10 11:04:01 UTC Tests successful.

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
79 (only retried tests) 62 0 0 0 17

🟢 2025-11-10 11:04:10 UTC Build successful.
🟢 2025-11-10 11:04:36 UTC ydbd size 2.3 GiB changed* by +14.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 56f5415 merge: 91da091 diff diff %
ydbd size 2 437 268 264 Bytes 2 437 283 552 Bytes +14.9 KiB +0.001%
ydbd stripped size 518 550 160 Bytes 518 553 424 Bytes +3.2 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 10, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 10, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 10, 2025
@ydb-platform ydb-platform deleted a comment from github-actions bot Nov 10, 2025
alexvru
alexvru previously approved these changes Nov 11, 2025
adameat
adameat previously approved these changes Nov 11, 2025
Enjection
Enjection previously approved these changes Nov 11, 2025
@rosik rosik dismissed stale reviews from Enjection, adameat, and alexvru via e204220 November 11, 2025 11:24
@rosik rosik force-pushed the capacity-metrics-2 branch from 044c232 to e204220 Compare November 11, 2025 11:24
@github-actions
Copy link

github-actions bot commented Nov 11, 2025

2025-11-11 11:26:04 UTC Pre-commit check linux-x86_64-release-asan for 166a3d8 has started.
2025-11-11 11:26:20 UTC Artifacts will be uploaded here
2025-11-11 11:28:32 UTC ya make is running...
🟡 2025-11-11 14:13:12 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
17309 16688 0 242 346 33

🟢 2025-11-11 14:13:21 UTC Build successful.
🟢 2025-11-11 14:13:51 UTC ydbd size 3.8 GiB changed* by +34.3 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 531f527 merge: 166a3d8 diff diff %
ydbd size 4 081 237 048 Bytes 4 081 272 160 Bytes +34.3 KiB +0.001%
ydbd stripped size 1 515 018 120 Bytes 1 515 032 328 Bytes +13.9 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions
Copy link

github-actions bot commented Nov 11, 2025

2025-11-11 19:00:51 UTC Pre-commit check linux-x86_64-relwithdebinfo for 166a3d8 has started.
2025-11-11 19:00:57 UTC Artifacts will be uploaded here
2025-11-11 19:02:17 UTC ya make is running...
🟡 2025-11-11 21:12:36 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
40669 37840 0 3 2794 32

2025-11-11 21:13:11 UTC ya make is running... (failed tests rerun, try 2)
🟡 2025-11-11 22:08:30 UTC Some tests failed, follow the links below. Going to retry failed tests...

Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13852 (only retried tests) 11045 0 1 2788 18

2025-11-11 22:08:55 UTC ya make is running... (failed tests rerun, try 3)
🔴 2025-11-11 22:57:28 UTC Some tests failed, follow the links below.

Ya make output | Test bloat | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
13170 (only retried tests) 10365 0 1 2788 16

🟢 2025-11-11 22:57:36 UTC Build successful.
🟢 2025-11-11 22:57:54 UTC ydbd size 2.3 GiB changed* by +18.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 531f527 merge: 166a3d8 diff diff %
ydbd size 2 437 870 168 Bytes 2 437 889 480 Bytes +18.9 KiB +0.001%
ydbd stripped size 518 656 528 Bytes 518 663 824 Bytes +7.1 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@Enjection Enjection self-requested a review November 11, 2025 12:31
@PavelEfarinov PavelEfarinov merged commit bb953da into ydb-platform:main Nov 12, 2025
13 of 17 checks passed
@rosik rosik deleted the capacity-metrics-2 branch November 12, 2025 14:22
@rosik rosik self-assigned this Nov 12, 2025
qyryq pushed a commit to qyryq/ydb that referenced this pull request Nov 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants