Skip to content

Add per-process and per-call statistics for time and size metrics#54

Merged
izzet merged 4 commits intollnl:developfrom
izzet:feature/two-track-statistics
Mar 20, 2026
Merged

Add per-process and per-call statistics for time and size metrics#54
izzet merged 4 commits intollnl:developfrom
izzet:feature/two-track-statistics

Conversation

@izzet
Copy link
Collaborator

@izzet izzet commented Mar 20, 2026

This pull request introduces enhancements to the statistics and metrics computation pipeline, focusing on clearer separation between per-process and per-call metrics, and improving the calculation and naming of derived statistics. The main changes include new utility functions for column renaming and per-call statistics derivation, updates to aggregation logic, and expanded metric calculations for both process-level and call-level views.

Statistics and column handling improvements:

  • Added build_view_rename_map to standardize column names for view-level statistics, ensuring per-process and per-call metrics are clearly distinguished and fixing naming inconsistencies.
  • Introduced derive_call_stats, which computes per-call mean and standard deviation from sum-of-squares columns and cleans up intermediary columns.
  • Updated aggregation logic in _compute_view and related functions to handle new per-call and per-process metric suffixes, ensuring correct aggregation functions are applied for each metric type. [1] [2] [3]

Metric calculation enhancements:

  • Refactored metric suffixes and calculations in set_view_metrics to separately compute process-level (*_proc_*) and call-level (*_call_*) metrics, including new fractional metrics for both categories. [1] [2]
  • Expanded the calculation of fractional and percentile metrics to cover both process-based and call-based statistics, improving the granularity of performance analysis.

Consistency and naming updates:

  • Updated references and logic throughout the analyzer to use the new process-level metric suffixes (*_proc_*), replacing previous ambiguous suffixes and ensuring consistent metric naming in all views and additional metric calculations. [1] [2]

These changes collectively improve the clarity, accuracy, and flexibility of metric computation in the analysis pipeline.

izzet added 3 commits March 20, 2026 09:42
…etrics

  Separate view-level statistics into two families with unambiguous names:
  - time_proc_*/size_proc_* for inter-process distribution (renamed from existing)
  - time_call_*/size_call_* for individual I/O call properties (new)

  This enables diagnosing *why* a bottleneck is slow (load imbalance vs
  call jitter vs uniformly slow) rather than just identifying *what* is slow.
@izzet izzet self-assigned this Mar 20, 2026
@izzet izzet added the enhancement New feature or request label Mar 20, 2026
@izzet izzet merged commit 6044e49 into llnl:develop Mar 20, 2026
4 checks passed
@izzet izzet deleted the feature/two-track-statistics branch March 20, 2026 23:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants