feat(server): add threads count and disk space to sysinfo stats#2917
Open
seokjin0414 wants to merge 5 commits intoapache:masterfrom
Open
feat(server): add threads count and disk space to sysinfo stats#2917seokjin0414 wants to merge 5 commits intoapache:masterfrom
seokjin0414 wants to merge 5 commits intoapache:masterfrom
Conversation
Add threads_count, free_disk_space, and total_disk_space fields to the Stats struct. Collect thread count via sysinfo Process::tasks() API and disk space via sysinfo Disks API with longest-prefix mount point matching. Update binary protocol serialization/deserialization with backward-compatible decoding, CLI table/list output, sysinfo printer log format, and integration tests including message size verification. Closes apache#2732 Signed-off-by: seokjin0414 <sars21@hanmail.net> Signed-off-by: shin <sars21@hanmail.net>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #2917 +/- ##
============================================
- Coverage 70.10% 70.09% -0.02%
Complexity 776 776
============================================
Files 1028 1028
Lines 85279 85324 +45
Branches 62655 62711 +56
============================================
+ Hits 59786 59808 +22
- Misses 22966 22980 +14
- Partials 2527 2536 +9
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
hubcio
reviewed
Mar 11, 2026
Comment on lines
+123
to
+135
| let data_path = std::path::Path::new(&self.config.system.path); | ||
| let data_path = | ||
| std::fs::canonicalize(data_path).unwrap_or_else(|_| data_path.to_path_buf()); | ||
| let disks = sysinfo::Disks::new_with_refreshed_list(); | ||
| let mut best_mount_len = 0usize; | ||
| for disk in disks.list() { | ||
| let mount = disk.mount_point(); | ||
| if data_path.starts_with(mount) && mount.as_os_str().len() > best_mount_len { | ||
| best_mount_len = mount.as_os_str().len(); | ||
| stats.free_disk_space = disk.available_space().into(); | ||
| stats.total_disk_space = disk.total_space().into(); | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
how much time does this take when there are 10k++ files in local_data directory of iggy? this is blocking code, we dont want to introduce latency spikes on shard 0
Contributor
Author
There was a problem hiding this comment.
Replaced sysinfo::Disks with fs2::available_space() / fs2::total_space() in a5f7f70.
fs2 uses a single statvfs syscall on the given path — it does not scan directory contents, so file count (10k++) has zero impact.
Measured overhead is ~10μs regardless of directory size.
fs2 is already a dependency of the server crate (used in logger.rs for the same purpose).
Use numeric IDs from server responses instead of string identifiers for send_messages call to avoid partition_not_found race condition in multi-shard architecture. Signed-off-by: shin <sars21@hanmail.net>
…rics Replace sysinfo::Disks::new_with_refreshed_list() with fs2::available_space() and fs2::total_space() for significantly lower overhead. fs2 performs a single statvfs syscall per metric (~10μs) vs sysinfo scanning all mount points (~0.5ms). fs2 is already used in the server crate (logger.rs). Signed-off-by: shin <sars21@hanmail.net>
Use Partitioning::default() instead of partition_id(1) to avoid partition_not_found errors in multi-shard CI environments where partition metadata may not have propagated to all shards yet. Signed-off-by: shin <sars21@hanmail.net>
Signed-off-by: shin <sars21@hanmail.net>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2732.
threads_count,free_disk_space,total_disk_spacefields to Stats structProcess::tasks()API (Linux only, 0 on macOS)DisksAPI with longest-prefix mount point matchingIggyUsage(messages_size_bytes),Disk(free/total),Threads(conditional) to sysinfo log outputDesign
process.tasks().map(|t| t.len()).unwrap_or(0)— uses existingrefresh_processes()call, no additional sysinfo refresh needed. Conditional log output (skipped when 0, matching OpenFDs pattern)Disks::new_with_refreshed_list()per call (no caching) — longest prefix match againstcanonicalize(config.system.path)to find the correct mount pointmessages_size_bytesfield (iggy metadata only, no sysinfo) — added to sysinfo log outputcache_metrics, decoded withcurrent_position + N <= payload.len()guards for backward compatibility