Closed
Conversation
f2d508b to
9939539
Compare
Owner
|
Closing: the agent added 3 new test cases to offense.rb but none have |
auto-merge was automatically disabled
March 22, 2026 23:05
Pull request was closed
6
added a commit
that referenced
this pull request
Mar 23, 2026
Adds an integration test that catches "dead" fixture code — Ruby code added to offense.rb without ^ annotations, which passes tests vacuously because the cop doesn't detect it and no annotation expects it to. Splits offense fixtures into regions at blank lines following top-level `end` lines. Each region with ≥5 source lines must contain at least one annotation. This catches the pattern from PR #135 where a minimax agent added 3 unannotated describe blocks that weren't actually detected. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Automated fix fix for
RSpec/MultipleSubjectscorpus conformance.Details
RSpec/MultipleSubjectsResult
Changes
Agent conversation excerpt
Let me start by reading the current implementation and test fixtures to understand the issue better.
I see the issue. The offense.rb file already has some test cases, but the FN cases from the diagnosis have
subjectinsideifbranches, which are NOT in the test file yet. Let me first verify the current test status and then understand the problem better.I see. The test expects 6 offenses but only detects 3. The issue is that the current implementation only looks at direct children of the example group's body, but the FN cases have
subjectdeclarations nested insideifbranches.Let me update the offense.rb fixture with proper FN cases that match the diagnosis:
The test passed, but I expected it to fail. Let me verify what offenses are actually being detected:
Confirmed - nitrocop detects 0 offenses when it should detect 2. Let me verify RuboCop behavior:
Let me check if RuboCop RSpec is properly installed:
Let me look at the RuboCop RSpec ExampleGroup helper:
Let me look at the
Conceptclass to understand thesubject?andscope_change?matchers:Now I understand the issue. RuboCop's implementation uses a recursive search that finds subjects inside
ifbranches but stops at nested example groups. The current nitrocop implementation only looks at direct children of the example group's body.Let me implement the fix:
Let me check the Prism Node API to find the correct method for iterating children:
Let me look at how other cops handle iterating over different node types:
Let me look at how other cops handle recursive traversal of different node types:
Let me look at how
visit_if_nodeis implemented in one of these cops:Let me check how to iterate over all children in Prism:
Let me look at how
each_child_nodeis used in the codebase:Task prompt (3988 tokens)
Fix RSpec/MultipleSubjects — 0 FP, 3 FN
Instructions
You are fixing ONE cop in nitrocop, a Rust Ruby linter that uses Prism for parsing.
Current state: 229 matches, 0 false positives, 3 false negatives.
Focus on: FN (RuboCop flags code nitrocop misses).
Workflow
behavior on BOTH the specific FP case AND the general pattern:
tests/fixtures/cops/rspec/multiple_subjects/offense.rbwith^annotationtests/fixtures/cops/rspec/multiple_subjects/no_offense.rbcargo test --lib -- cop::rspec::multiple_subjectssrc/cop/rspec/multiple_subjects.rscargo test --lib -- cop::rspec::multiple_subjects///doc comment on the cop struct documenting what you found and fixedFixture Format
Mark offenses with
^markers on the line AFTER the offending source line:The
^characters must align with the offending columns. The message format isRSpec/MultipleSubjects: <message text>.If your test passes immediately
If you add a test case and it passes without code changes, the corpus mismatch is
caused by config/context differences, not a detection bug.
Do NOT loop trying to make the test fail. Instead:
src/config/or the cop's config handling, not detection logica
///comment on the cop struct and commitCRITICAL: Avoid regressions in the opposite direction
When fixing FPs, your change MUST NOT suppress legitimate detections. When fixing FNs,
your change MUST NOT flag code that RuboCop accepts. A fix that eliminates a few issues
in one direction but introduces hundreds in the other is a catastrophic regression.
Before exempting a category of patterns, verify with RuboCop that the general case
is still an offense:
If RuboCop flags the general pattern but not your specific case, the difference is in
a narrow context (e.g., enclosing structure, receiver type, argument count) — your fix
must target that specific context, not the broad category.
Rule of thumb: if your fix adds an early
returnorcontinuethat skips a wholenode type, operator class, or naming pattern, it's probably too broad. Prefer adding a
condition that matches the SPECIFIC differentiating context.
Rules
src/cop/rspec/multiple_subjects.rsandtests/fixtures/cops/rspec/multiple_subjects/cargo test --lib -- cop::rspec::multiple_subjectsto verify your fix (do NOT run the full test suite)git stashPre-diagnostic Results
Diagnosis Summary
Each example was tested by running nitrocop on the extracted source in isolation
with
--force-default-configto determine if the issue is a code bug or config issue.Note: source context is truncated and may not parse perfectly. If a diagnosis
seems wrong (e.g., your test passes immediately for a 'CODE BUG'), treat it as
a config/context issue instead.
FN #1:
doorkeeper-gem__doorkeeper-openid_connect__225e319: spec/lib/oauth/password_access_token_request_spec.rb:7NOT DETECTED — CODE BUG
The cop fails to detect this pattern. Fix the detection logic.
Enclosing structure: if branch (line:
if Gem.loaded_specs['doorkeeper'].version >= Gem::Version.create('5.5.1'))The offense is inside this structure — the cop may need
to handle this context to detect the pattern.
Message:
Do not set more than one subject per example groupReady-made test snippet (add to offense.rb, adjust
^count):Full source context:
FN #2:
madeintandem__hstore_accessor__b527f0f: spec/hstore_accessor_spec.rb:299NOT DETECTED — CODE BUG
The cop fails to detect this pattern. Fix the detection logic.
Enclosing structure: if branch (line:
if ::ActiveRecord::VERSION::STRING.to_f >= 4.2)The offense is inside this structure — the cop may need
to handle this context to detect the pattern.
Message:
Do not set more than one subject per example groupReady-made test snippet (add to offense.rb, adjust
^count):Full source context:
FN #3:
ruby-grape__grape-active_model_serializers__c985c86: spec/grape-active_model_serializers/endpoint_extension_spec.rb:5NOT DETECTED — CODE BUG
The cop fails to detect this pattern. Fix the detection logic.
Enclosing structure: if branch (line:
if Grape::Util.const_defined?('InheritableSetting'))The offense is inside this structure — the cop may need
to handle this context to detect the pattern.
Message:
Do not set more than one subject per example groupReady-made test snippet (add to offense.rb, adjust
^count):Full source context:
Current Rust Implementation
src/cop/rspec/multiple_subjects.rsRuboCop Ruby Implementation (ground truth)
vendor/rubocop-rspec/lib/rubocop/cop/rspec/multiple_subjects.rbRuboCop Test Excerpts
vendor/rubocop-rspec/spec/rubocop/cop/rspec/multiple_subjects_spec.rbCurrent Fixture: offense.rb
tests/fixtures/cops/rspec/multiple_subjects/offense.rbCurrent Fixture: no_offense.rb
tests/fixtures/cops/rspec/multiple_subjects/no_offense.rb