HDDS-15323. Add storage capacity planning administrator guide.#436
Open
jojochuang wants to merge 2 commits into
Open
HDDS-15323. Add storage capacity planning administrator guide.#436jojochuang wants to merge 2 commits into
jojochuang wants to merge 2 commits into
Conversation
Document datanode reserved space, full-disk thresholds, and inspection workflows. Link from the Datanodes architecture page. Co-authored-by: Cursor <cursoragent@cursor.com>
jojochuang
requested review from
ChenSammi,
Gargi-jais11,
ashishkumar50 and
aswinshakil
|
|
||
| - **Purpose**: Protects the Datanode from OS-level disk-full errors. | ||
| - **Configuration**: | ||
| - `hdds.datanode.volume.min.free.space`: A fixed value (e.g., `20GB`). |
Contributor
There was a problem hiding this comment.
@jojochuang Should we also mention about volume min free space hard limit?
- Soft limit (min.free.space.percent, default 2%): used for what is reported to SCM in heartbeats as freeSpaceToSpare. This governs SCM placement decisions.
- Hard limit (min.free.space.hard.limit.percent, default 1.5%): used for local write rejection on the Datanode.
Contributor
Author
There was a problem hiding this comment.
Ah yes. I missed these recent additions.
| To see the status of individual disks on a specific Datanode: | ||
|
|
||
| ```bash | ||
| ozone admin datanode usageinfo --uuid <datanode-uuid> |
Contributor
There was a problem hiding this comment.
uuid is deprecated.
Suggested change
| ozone admin datanode usageinfo --uuid <datanode-uuid> | |
| ozone admin datanode usageinfo --node-id <datanode-uuid> |
|
|
||
| ### 3. Detailed Volume Info | ||
|
|
||
| To see the status of individual disks on a specific Datanode: |
Contributor
There was a problem hiding this comment.
This is a bit misleading as datanode usageInfo it does not give individual disks info on specific Datanode.
Rather it should be,
Suggested change
| To see the status of individual disks on a specific Datanode: | |
| To see the aggregated capacity, usage, and remaining space for a specific Datanode: |
| | `hdds.datanode.volume.min.free.space.percent` | `0.02` (2%) | Percentage-based minimum free space. | | ||
| | `hdds.datanode.dir.du.reserved` | (unset) | Fixed bytes reserved for non-Ozone use. | | ||
| | `ozone.scm.container.size` | `5GB` | The target size for containers. | | ||
| | `hdds.datanode.storage.utilization.critical.threshold` | `0.95` (95%) | Threshold at which SCM logs a "Critical" space warning. | |
Contributor
There was a problem hiding this comment.
We also mention about warning utilisation threshold :
<property>
<name>hdds.datanode.storage.utilization.warning.threshold</name>
<value>0.75</value>
<tag>OZONE, SCM, MANAGEMENT</tag>
<description>
If a datanode overall storage utilization exceeds more than this
value, a warning will be logged while processing the nodeReport in SCM.
</description>
</property>
Document soft/hard min free space limits, warning utilization threshold, --node-id CLI flag, and clarify per-datanode usageinfo scope. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
HDDS-15323. Add storage capacity planning administrator guide.
Please describe your PR in detail:
What is the link to the Apache Jira?
https://issues.apache.org/jira/browse/HDDS-15323
How was this patch tested?