[None][fix] Fix GPQA Diamond filter_type mismatch in disagg accuracy …

[yingguo-trt]

…test

The disagg YAML declared filter_type: flexible-extract, but the lm_eval config tests/integration/lm_eval_configs/gpqa_diamond_local.yaml only defines a strict-match filter. As a result, the accuracy checker could not find the expected row and marked the case as a "Benchmark run error" in aggregated JUnit, even though the underlying run succeeded.

GPQA is a multiple-choice task where strict-match is the correct filter; flexible-extract is semantically wrong here.

Summary by CodeRabbit

• Tests
  • Updated accuracy validation testing to use stricter matching criteria for improved evaluation accuracy.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

📝 Walkthrough

Walkthrough

A single YAML configuration file was modified to change the dataset accuracy validation filter type from flexible-extract to strict-match for the GPQA Diamond local dataset in a performance test script.

Changes

Cohort / File(s)	Summary
Performance Test Configuration tests/scripts/perf/disaggregated/wideep_accuracy-deepseek-r1-fp4_gpqa_diamond_1k1k_ctx2_gen1_dep16_bs128_eplb288_mtp3_ccb-NIXL.yaml	Updated filter_type for gpqa_diamond_local accuracy metadata from flexible-extract to strict-match.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	PR description explains the issue, cause, and solution clearly. However, required sections like 'Test Coverage' and 'Description' headers are not filled in, though the core information is provided.	Provide explicit details under the 'Description' and 'Test Coverage' sections. Clarify how this fix was validated and whether existing tests cover the accuracy checking behavior.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title follows the required format and accurately describes the fix: it identifies the issue (GPQA Diamond filter_type mismatch) and specifies the location (disagg accuracy config).
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[yingguo-trt]

/bot skip --comment "Not cover in CI pipelines"

[fredricz-20070104]

Approved. This is a bug.

[tensorrt-cicd]

PR_Github #44407 [ skip ] triggered by Bot. Commit: c98f1c0 Link to invocation

[tensorrt-cicd]

PR_Github #44407 [ skip ] completed with state SUCCESS. Commit: c98f1c0
Skipping testing for commit c98f1c0

Link to invocation