Skip to content

Conversation

@vkarak
Copy link
Contributor

@vkarak vkarak commented May 22, 2023

This PR provides the following:

  1. The performance logs for paramterised tests using the filelog handler are combined in a single file that is named after the test's base name (i.e. the class name). This is very useful as a parameterised test is essentially the same test with differing parameters.
  2. A new special log format specifier %(check_#ALL)s is added. If this is found in the log format, it will be replaced by all the loggable %(check_<attr>)s.
  3. A new ignore_keys configuration parameter is added to the filelog log handler that allows users to ignore certain test attributes during logging. The builtin configuration already puts several attributes in that list in order to save space.
  4. The default delimiter for the filelog output is reverted to | as the , may conflict with the logged env_vars and extra_resources. Users using the %(check_#ALL)s specifier should pay attention to the delimiter used in their format.

Todos

  • Update docs
  • Update the configuration schema

Closes #2591.

Also:

- New special log format specifier that dumps all the loggable test attributes
@vkarak vkarak added this to the ReFrame 4.3 milestone May 22, 2023
@vkarak vkarak requested review from ekouts, teojgo and victorusu May 22, 2023 22:22
@vkarak vkarak self-assigned this May 22, 2023
@vkarak vkarak marked this pull request as draft May 22, 2023 22:22
@vkarak vkarak changed the title [feat] Combine logs for parameterised tests [feat] Combine perflogs for parameterised tests May 22, 2023
@codecov
Copy link

codecov bot commented May 22, 2023

Codecov Report

Patch coverage: 91.42% and no project coverage change.

Comparison is base (a128328) 86.97% compared to head (3be4031) 86.98%.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop    #2882   +/-   ##
========================================
  Coverage    86.97%   86.98%           
========================================
  Files           60       60           
  Lines        11596    11627   +31     
========================================
+ Hits         10086    10114   +28     
- Misses        1510     1513    +3     
Impacted Files Coverage Δ
reframe/core/settings.py 100.00% <ø> (ø)
reframe/core/logging.py 80.80% <91.42%> (+0.52%) ⬆️

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@vkarak vkarak marked this pull request as ready for review May 23, 2023 21:52
Copy link
Contributor

@ekouts ekouts left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm, just some small comments.

vkarak and others added 2 commits June 13, 2023 01:13
@vkarak vkarak requested a review from ekouts June 13, 2023 19:38
@vkarak vkarak merged commit fa813d0 into reframe-hpc:develop Jun 14, 2023
@vkarak vkarak deleted the feat/combine-param-test-logs branch June 14, 2023 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

RFE: new parameter for perflog file name

2 participants