ReFrame 4.8.2

Release Notes

Enhancements

• Improve programming error messages (#3489)

Bug fixes

• Properly cleanup build jobs on abort (#3512)
• Fix race condition in local scheduler during timeout and properly reap done processes (#3511)
• Make -n option syntax consistent across test filtering and results storage queries (#3508)
• Return Job.jobid as string always (#3493)
• Include (again) partition name in test failure info (#3487)

Other

• Bump requests package version (#3501)

ReFrame 4.8.1

Release Notes

Enhancements

• Use the --use-pep517 option in bootstrap.sh (#3478)
• Use ClusterShell to manipulate node lists instead of our own code (#3470)
• Show up front if result storage is enabled (#3472)
• Fix typo in versionadded annotation (#3473)

Bug fixes

• Use always | to join unique values in column aggregations except for --table-format=pretty (#3471)
• Unset all SBATCH_* environment variables before submitting a job (#3474)
• Use display_name when printing the test's name in FAILURE INFO (#3468)

ReFrame 4.8.0

Release Notes

New features

• Support list assignment from JSON using the -S option (#3449)
• Support discrete testcase filters when comparing performance (#3438)
• Do not emit the -lselect option if num_tasks=None (#3452)
• Allow changing the default mount location of stagedir inside containers (#3442)
• Support selecting A/B column variants in performance comparisons (#3431)
• Support reference values of zero (#3434)
• Allow scoping session-based DB queries by time (#3403)
• Store test sub-result per performance variable (#3414)
• Add support for configure directory in CMake builds (#3395)
• Enable YAML configuration files (#3370)
• Introduce %(hostname)s log format placeholder (#3346)

Enhancements

• Document the requirement of file lock support for enabling results storage (#3457)
• Add a note about results storage and analytics in README (#3453)
• Use correct OMP_PROC_BIND arguments in the tutorial examples (#3435)
• List parameterized tests in their instantiation order (#3440)
• Add warning about combining build_locally=0 and scheduler backends that cannot reliably retrieve the job's exit code (#3426)

Bug fixes

• Correctly handle verbatim scheduler options passed in the CLI in the remote auto-detection script (#3456)
• Treat correctly failed_stage=None in failure stats report (#3454)
• Fix constraint evaluation for Slurm features starting with numbers (#3448)
• Fix lock order when storing reports to the database (#3450)
• Update special parameter character in test name (#3441)
• Disallow command options from execution modes (#3432)
• Fix calculation of lower/upper reference bounds when their absolute value reaches zero (#3433)
• Insert new line between stdout/stderr listings in failures (#3429)
• Disable number parsing when in --list-stored-sessions (#3425)

Other

• Add GH action to test EOL python versions (#3417)
• Enabled slurmdbd in Docker compose setup (#3391)
• Use Python 3.13 to publish to PyPI (#3406)
• Update YAML unit test config (#3405)
• Replace symlinks with file copies in Docker compose config for Github workflows (#3396)
• Remove stale Dockerfiles (#3387)
• Update Slurm cluster images to ubuntu:22.04 (#3383)

ReFrame 4.7.4

Release Notes

Enhancements

• Fix tutorial on --performance-report default behaviour (#3389)
• Update copyright year in docs (#3384)
• Fix default value for results storage in documentation (#3368)

Bug fixes

• Fix bug in update of pipeline progress statistics (#3400)
• Do not copy or git clone sourcesdir if --dont-restage is passed (#3393, #3398)
• Properly expand environment variables in execution modes (#3390)
• Include test info when skipping tests (#3388)
• Fix assignment of default configuration parameters across systems (#3377)
• Fix ESRCH error due to race condition when spawning fast futures (#3379, #3381)

Other

• Fix wheel file pattern in the wheel validation Github action (#3401)

ReFrame 4.7.3

Release Notes

Enhancements

• Report all failed performance variables in FAILURE INFO (#3358)

Bug fixes

• Fix use of dump_pipeline_progress and RFM_DUMP_PIPELINE_PROGRESS options (#3365)
• Add missing topology_prefix option in the configuration schema (#3364)
• Check response status in the httpjson log handler and retry in case of TOO_MANY_REQUESTS (#3356)
• Treat a node in PLANNED Slurm state as available (#3361)

ReFrame 4.7.2

Release Notes

Enhancements

• Optimize JSON decoding for sessions when applying filtering criteria (#3343)

Bug fixes

• Fix treatment of Slurm constraints in job.options and system partition's access parameter (#3352)

Other

• Update GH actions for Python 3.7 after the move of runners to Ubuntu 24.04 (#3350)

ReFrame 4.7.1

Release Notes

Enhancements

• Prefix special (generated) parameters with . instead of $ (#3330)
• Update formatting of list of talks in the documentation (#3321)
• Remove tip for disabling storage results from documentation (#3319)

Bug fixes

• Exit successfully if --duration limit is reached (#3328)
• Treat correctly Slurm constrains with hyphens when filtering nodes (#3327)
• Fix nodelist abbreviation when last node number is a multiple of 10 (#3339)
• Do not ignore --filter-expr when using --describe-stored-testcases (#3329)
• Fix error in fixture resolution when multiple fixtures of the same class are instantiated with different variables (#3326)
• Fix relative imports in deeply nested tests (#3337)

ReFrame 4.7.0

Release Notes

New features

• Make directory prefix for topology files configurable (#3317)

• Allow partition/environment extras to be used also as feature constraints in valid_systems and valid_prog_environs (#3316)

• Show the last N lines in test failures (#3291)

• New configuration parameter for specifying commands to install reframe remotely for topology detection (#3281)

• Add a check_fail_phase log formatting specifier (#3292)

• Allow users define custom log handlers and attach them to the framework (#3274)

• Add new --retries-threshold option (#3262)

• Introduce permanent result storage and support performance results comparisons between runs (#3227)

  • Add a configuration option to enable/disable results storage (#3295)
  • Support session annotations with a new --session-extras option (#3266)
  • Expose SQLite connection timeout as configuration parameter (#3253)
  • Disable results storage by default (#3313)
  • Update --table-format arguments (#3307)
  • Use builtin json package for decoding testcase data (#3296)
  • Various enhancements to the results DB feature (#3283)
  • Allow setting the file mode of the SQLite database (#3260)
  • Change default values for --performance-report and --list-stored-sessions options (#3258)
  • Fix query error when aggregation groups and presentation columns are disjoint (#3294)
  • Fix DST skew in CMPSPEC period calculations (#3288)
  • Optimize testcase queries over time period (#3256)
  • Ignore records with unset performance when grouping test cases (#3247)

• Add a perflog entry also when sanity fails (#3189)

• Skip by default flexible tests when not enough nodes found (#3173)

For the new feature of permanent results storage and performance analytics, please have a look at the tutorial and the manpage for details.

Enhancements

• Report the undefined parameters if an instantiation of an abstract test or fixture is attempted (#3293)
• Warn instead of failing when a filter expression is invalid for test (#3310)
• Issue a more compact message when JSON schema validation fails (#3314)
• Do not issue warning when skipping tests during the init phase (#3286)
• Reorganize the structure of the manpage (#3267)
• Improve performance of module backend when detecting module conflicts (#3263)
• Improve naming of automatically generated tests (#3265)

Bug fixes

• Add job options from the command-line to processor auto-detection remote job (#3318)
• Handle None cpu model name returned from archspec (#3199)
• Constraint the lxml version on aarch64 (#3193)

Test library

• Fix compilation of gpu burn benchmark with hip (#3232)
• Added MetalWalls scalability test for mw (#3233)

Other

• Enable Python 3.13 in CI + expand Github test workflows (#3309)
• Use local copies of get-pip.py for bootstrapping (#3287)
• Run scheduler unittests with GitHub Actions (#3268)
• Do not upload to Codecov through a GH action (#3188)

ReFrame 4.6.4

Release Notes

Enhancements

• Print test file in the failure info (#3282)

Bug fixes

• Allow RESERVED nodes for Slurm "avail" state (#3290)
• Let Slurm decide partition in flexible node allocation if no default partition is defined (#3280)

ReFrame 4.6.3

Release Notes

Enhancements

• Add information on how to cite ReFrame (#3259)
• Print selected system in [ReFrame Setup] information (#3255)
• Update disclaimer on the support of EOL Python versions (#3238)

Bug fixes

• Fix typo in the --distribute docs (#3239)