Skip to content

Add configurable size_layers and size_derived_metrics for layer-specific size metric handling#47

Merged
izzet merged 3 commits intollnl:developfrom
izzet:feat/support-size-derived-metrics
Mar 5, 2026
Merged

Add configurable size_layers and size_derived_metrics for layer-specific size metric handling#47
izzet merged 3 commits intollnl:developfrom
izzet:feat/support-size-derived-metrics

Conversation

@izzet
Copy link
Collaborator

@izzet izzet commented Mar 4, 2026

This pull request introduces a more flexible system for handling "size" metrics and layers in the analyzer configuration and processing logic. The main improvements are the ability to specify which layers and metrics should be treated as "size" related, and to propagate these settings throughout the analyzer pipeline. This enhances configurability for different storage backends and analysis presets.

Configuration enhancements:

  • Added size_derived_metrics and size_layers fields to the AnalyzerPresetConfig dataclass, with appropriate defaults for POSIX and DLIO presets, allowing each preset to specify which layers and metrics are considered "size"-related. [1] [2] [3]
  • Introduced DERIVED_POSIX_SIZE_METRICS constant to centralize the list of metrics considered as size-related for POSIX.

Analyzer logic updates:

  • Modified the set_layer_metrics method in analyzer.py to accept a size_derived_metrics argument, and use it to determine which metrics should be treated as size-related, improving flexibility and reducing hardcoding.
  • Updated the _compute_main_view method to use the new configuration fields, ensuring that size columns and metrics are handled correctly based on the current layer and preset.

Bug fix / data extraction improvement:

  • Fixed the logic for extracting image_size in io_function to only assign the size field when the operation name does not include "open", ensuring more accurate data attribution.

@izzet izzet requested a review from Copilot March 4, 2026 23:52
@izzet izzet self-assigned this Mar 4, 2026
@izzet izzet added the enhancement New feature or request label Mar 4, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes “size” metric handling configurable per analyzer preset and propagates that configuration through the analyzer pipeline, while also improving image_size extraction in IO event parsing.

Changes:

  • Added preset config fields size_layers and size_derived_metrics (with POSIX/DLIO defaults) and centralized POSIX size-derived metric names.
  • Updated analyzer logic to keep/drop size columns by configured layers and to treat only configured derived metrics as “size” metrics.
  • Adjusted IO parsing to populate size from image_size only for non-open operations.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
python/dftracer/analyzer/dftracer.py Updates image_sizesize attribution logic during IO event parsing.
python/dftracer/analyzer/config.py Introduces configurable size layer/metric preset fields and a POSIX size metrics constant.
python/dftracer/analyzer/analyzer.py Threads new size config into main view computation and derived metric column creation.
Comments suppressed due to low confidence (1)

python/dftracer/analyzer/config.py:1

  • These fields are typed as Optional[...] but default to empty containers via default_factory, so they’ll never be None unless explicitly passed as None. This makes downstream code more complex (extra or {} / or []) and makes the API ambiguous. Prefer either (a) making them non-optional (Dict[...] / List[...]) with the current defaults, or (b) defaulting to None without default_factory if None is a meaningful “unset” state.
import dataclasses as dc

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

izzet and others added 2 commits March 4, 2026 18:21
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Collaborator

@rayandrew rayandrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@rayandrew
Copy link
Collaborator

@izzet let me know if i want to merge this first or u still need to test using Karim's new traces

@izzet
Copy link
Collaborator Author

izzet commented Mar 5, 2026

I am merging this @rayandrew thanks

@izzet izzet merged commit 3cc85b5 into llnl:develop Mar 5, 2026
4 checks passed
@izzet izzet deleted the feat/support-size-derived-metrics branch March 5, 2026 02:29
@izzet izzet restored the feat/support-size-derived-metrics branch March 5, 2026 17:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants