[None][fix] Narrow bare except clause and use identity check for None

[edenfunf]

Background / Motivation

examples/scaffolding/run_majority_vote_aime24.py contains two code quality issues that can cause unexpected runtime behavior:

1. Bare except: clause (line 112) catches all exceptions including KeyboardInterrupt and SystemExit, making it impossible to cleanly terminate the process with Ctrl+C during long AIME24 evaluations.
2. == None comparison (line 79) violates PEP 8. The == operator can be overridden by __eq__, potentially leading to incorrect behavior with custom objects.

Change Summary

• except: -> except (ValueError, TypeError): — these are the only exceptions int() can raise, so this precisely catches the intended error cases while allowing system exceptions to propagate normally.
• == None -> is None — uses identity comparison as recommended by PEP 8.

Impact

• Performance: No impact.
• Behavior: The evaluation script now allows clean process interruption (Ctrl+C). All intended error handling behavior is preserved.
• API: No API changes — this is an example script fix.

Related Issue

Fixes #12040

Summary by CodeRabbit

• Refactor
  • Enhanced exception handling in example scripts to provide more specific error messages during answer parsing, improving error reporting clarity.

[coderabbitai]

📝 Walkthrough

Walkthrough

This pull request applies two Python code quality fixes to a scaffolding example file: replacing an improper == None comparison with is None per PEP 8, and narrowing a bare except: clause to catch only specific exceptions (ValueError, TypeError) that can be raised by int() conversion.

Changes

Cohort / File(s)	Summary
Python Code Quality Fixes examples/scaffolding/run_majority_vote_aime24.py	Replaced improper None comparison (== None → is None) and narrowed bare exception handler from all exceptions to specific ones (ValueError, TypeError) for cleaner error handling.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~5 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description check	✅ Passed	The description clearly explains the background, specific changes, and impact, though it doesn't follow the exact template structure.
Linked Issues check	✅ Passed	The PR fully addresses both requirements from issue #12040: narrowing the except clause to (ValueError, TypeError) and replacing == None with is None.
Out of Scope Changes check	✅ Passed	All changes are narrowly scoped to the two specific issues identified in #12040; no unrelated modifications are present.
Title check	✅ Passed	The title accurately summarizes the main changes: narrowing a bare except clause and using identity comparison for None checks, which directly matches the changeset.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

Tip

Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs).
Share your feedback on Discord.

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

examples/scaffolding/run_majority_vote_aime24.py (1)

78-90: ⚠️ Potential issue | 🔴 Critical

--concurrency without --static_with_benchmark still crashes here.

If the caller passes --concurrency alone, this branch runs but task_collection_types is only assigned inside if args.static_with_benchmark:. Line 89 then uses it unconditionally, so the script fails with UnboundLocalError before any requests execute.

Example fix

     if args.static_with_benchmark or args.concurrency:
         if args.concurrency is None:
             args.concurrency = 1
+        task_collection_types = {}
 
         if args.static_with_benchmark:
             task_collection_types = {"token_counter": GenerationTokenCounter}

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@examples/scaffolding/run_majority_vote_aime24.py` around lines 78 - 90, The
branch handling concurrency/static benchmarking sets task_collection_types only
when args.static_with_benchmark is true, causing an UnboundLocalError when
--concurrency is provided alone; ensure task_collection_types is always defined
before it is passed to async_scaffolding_benchmark by initializing it (e.g., to
None or an empty dict) before the if args.static_with_benchmark block, keep the
existing assignment to GenerationTokenCounter inside that block, and then call
async_scaffolding_benchmark(llm, task_collection_types, requests, ...) using the
guaranteed variable; adjust only the setup around args.concurrency,
task_collection_types, and the requests list (ScaffoldingBenchRequest/prompts)
so the code no longer errors when --concurrency is used without
--static_with_benchmark.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Outside diff comments:
In `@examples/scaffolding/run_majority_vote_aime24.py`:
- Around line 78-90: The branch handling concurrency/static benchmarking sets
task_collection_types only when args.static_with_benchmark is true, causing an
UnboundLocalError when --concurrency is provided alone; ensure
task_collection_types is always defined before it is passed to
async_scaffolding_benchmark by initializing it (e.g., to None or an empty dict)
before the if args.static_with_benchmark block, keep the existing assignment to
GenerationTokenCounter inside that block, and then call
async_scaffolding_benchmark(llm, task_collection_types, requests, ...) using the
guaranteed variable; adjust only the setup around args.concurrency,
task_collection_types, and the requests list (ScaffoldingBenchRequest/prompts)
so the code no longer errors when --concurrency is used without
--static_with_benchmark.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: af76fd52-5794-467d-9036-45a4f5af4451

📥 Commits

Reviewing files that changed from the base of the PR and between 9fe24db and 5e9a769.

📒 Files selected for processing (1)

• examples/scaffolding/run_majority_vote_aime24.py

[pengbowang-nv]

/bot run

[tensorrt-cicd]

PR_Github #38367 [ run ] triggered by Bot. Commit: 3153f6a Link to invocation

[tensorrt-cicd]

PR_Github #38367 [ run ] completed with state SUCCESS. Commit: 3153f6a
/LLM/main/L0_MergeRequest_PR pipeline #29735 completed with status: 'FAILURE'

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

[edenfunf]

Hi, I’ve updated the PR title to follow the required format.
Could someone from the NVIDIA team please trigger the CI again? Thanks!

[pengbowang-nv]

Hi @edenfunf , I'll take care of CI failures if it's due to some unstable machine, and will ping you if there is real issue related to the PR. So don't worry about these failure messages sent by bot.

[pengbowang-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #38434 [ run ] triggered by Bot. Commit: dbe42cf Link to invocation