Skip to content

Conversation

PatKamin
Copy link
Contributor

@PatKamin PatKamin commented Oct 7, 2025

Generate Compute Benchmarks scenarios' metadata automatically based on Benchmark.explicit_group.

@PatKamin PatKamin requested a review from a team as a code owner October 7, 2025 07:37
@lslusarczyk lslusarczyk requested a review from Copilot October 7, 2025 07:44
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the metadata generation for Compute Benchmarks by introducing an automated metadata generator. The changes centralize metadata creation logic and ensure consistency between benchmark group membership and metadata definitions.

  • Introduces a new ComputeMetadataGenerator class to automatically generate metadata based on Benchmark.explicit_group
  • Updates type annotations for optional fields in BenchmarkMetadata to use union syntax
  • Replaces manual metadata creation with automated generation in the Compute suite

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
devops/scripts/benchmarks/utils/result.py Updates type annotations for range fields to use explicit union syntax
devops/scripts/benchmarks/benches/compute_metadata.py Adds new metadata generator class with centralized group metadata definitions
devops/scripts/benchmarks/benches/compute.py Refactors to use automated metadata generation instead of manual creation

Copy link
Contributor

@lslusarczyk lslusarczyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would love to see some test that executes this code, while we do a change. To not rely on manual checks. But UTs, if any, we can add in other PR.

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

Copy link
Contributor

@lslusarczyk lslusarczyk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you, good job

PatKamin and others added 4 commits October 7, 2025 12:17
Co-authored-by: Łukasz Ślusarczyk <lukasz.slusarczyk@intel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@PatKamin
Copy link
Contributor Author

PatKamin commented Oct 7, 2025

@intel/llvm-gatekeepers, please merge. Failing e2e test is unrelated to this benchmarks framework change. This change should fix missing charts issue in the dashboard. Benchmark run CI job passed.

@lslusarczyk
Copy link
Contributor

@intel/llvm-gatekeepers , could you please merge this? Changes are OK, CI passes and this is an important fix that makes benchmarks plots working again.

@sarnex sarnex merged commit 5c53e77 into intel:sycl Oct 7, 2025
46 of 47 checks passed
@PatKamin PatKamin deleted the metadata-generator branch October 7, 2025 16:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants