Skip to content

thread safety: reporters and docs#175

Merged
pftg merged 5 commits intomasterfrom
emdash/discuss-thread-safe-ruby-74u
Apr 12, 2026
Merged

thread safety: reporters and docs#175
pftg merged 5 commits intomasterfrom
emdash/discuss-thread-safe-ruby-74u

Conversation

@pftg
Copy link
Copy Markdown
Collaborator

@pftg pftg commented Apr 12, 2026

summary:

  • add mutex-protected reporter notification snapshot in snap_diff
  • add html reporter internal mutex and test coverage
  • update thread safety doc for snap_diff

tests:

  • bundle exec ruby -Itest test/unit/reporters/html_reporter_test.rb test/unit/reporters_mutex_test.rb

Summary by Sourcery

Ensure thread-safe reporter handling and document thread safety guarantees for snap_diff.

Enhancements:

  • Guard HTML reporter internal state with a mutex to safely handle concurrent record and finalize calls.
  • Capture a mutex-protected snapshot of reporters before notification to avoid concurrent modification issues.
  • Add a thread safety guide describing snap_diff behavior under Rails parallel tests and recommended usage.

Tests:

  • Add unit tests verifying that reporter notification and HTML reporter operations are synchronized via mutexes.

@sourcery-ai
Copy link
Copy Markdown
Contributor

sourcery-ai Bot commented Apr 12, 2026

Reviewer's Guide

Makes reporter notification and HTML reporter operations thread-safe by introducing mutex-protected snapshots and internal locks, adds tests to guard synchronization behavior, and documents snap_diff thread-safety guarantees.

Sequence diagram for thread-safe reporter notification and HTML reporter operations

sequenceDiagram
  actor TestThread
  participant ScreenshotAssertion
  participant ReportersMutex
  participant HtmlReporter

  TestThread->>ScreenshotAssertion: notify_reporters(assertions)
  ScreenshotAssertion->>ScreenshotAssertion: validate assertions not nil/empty
  ScreenshotAssertion->>ReportersMutex: synchronize
  activate ReportersMutex
  ScreenshotAssertion->>ScreenshotAssertion: reporters_snapshot = reporters.dup
  ReportersMutex-->>ScreenshotAssertion: release lock
  deactivate ReportersMutex
  ScreenshotAssertion->>ScreenshotAssertion: return if reporters_snapshot.empty?

  loop for each reporter in reporters_snapshot
    ScreenshotAssertion->>HtmlReporter: record(assertions)
    activate HtmlReporter
    HtmlReporter->>HtmlReporter: return if @finalized
    HtmlReporter->>HtmlReporter: local failures = []
    HtmlReporter->>HtmlReporter: local total = 0
    HtmlReporter->>HtmlReporter: iterate assertions, build failures and total
    HtmlReporter->>HtmlReporter: synchronize on @mutex
    activate HtmlReporter
    HtmlReporter->>HtmlReporter: return if @finalized
    HtmlReporter->>HtmlReporter: @total += local total
    HtmlReporter->>HtmlReporter: @failures.concat(local failures)
    HtmlReporter-->>ScreenshotAssertion: release @mutex
    deactivate HtmlReporter
    deactivate HtmlReporter
  end

  TestThread->>HtmlReporter: finalize
  activate HtmlReporter
  HtmlReporter->>HtmlReporter: synchronize on @mutex
  activate HtmlReporter
  HtmlReporter->>HtmlReporter: return if @finalized
  HtmlReporter->>HtmlReporter: @finalized = true
  HtmlReporter->>HtmlReporter: return if failures.empty?
  HtmlReporter->>HtmlReporter: write_report
  HtmlReporter-->>TestThread: output_path
  deactivate HtmlReporter
  deactivate HtmlReporter
Loading

Updated class diagram for ScreenshotAssertion and HtmlReporter thread safety

classDiagram
  class ScreenshotAssertion {
    +Array reporters
    +Mutex reporters_mutex
    +reporters()
    +reporters_mutex()
    -notify_reporters(assertions)
    +screenshot_namer()
    +verify()
  }

  class HtmlReporter {
    -String output_path
    -Boolean embed_images
    -Array failures
    -Integer total
    -Boolean finalized
    -Mutex mutex
    +initialize(output_path, embed_images)
    +record(assertions)
    +finalize()
    +passed()
    -failure_entry_for(name, compare)
    -write_report()
  }

  ScreenshotAssertion "1" o-- "*" HtmlReporter : reporters
  ScreenshotAssertion --> "1" Mutex : reporters_mutex
  HtmlReporter --> "1" Mutex : mutex
Loading

File-Level Changes

Change Details Files
Make HTML reporter record/finalize operations internally thread-safe using a mutex and local accumulation before synchronization.
  • Introduce an internal @Mutex in the HTML reporter to guard access to @failures, @ToTal, and @Finalized
  • Short-circuit record when reporter has been finalized
  • Accumulate per-call failures and total outside the lock, then update shared state inside a synchronized block
  • Wrap finalize logic in a synchronized block while preserving early returns when already finalized or when there are no failures
lib/capybara_screenshot_diff/reporters/html.rb
Add synchronization-focused tests for HTML reporter and reporter notification snapshot behavior.
  • Add a unit test that injects a fake mutex into HTML reporter to assert record/finalize use synchronize
  • Add a test suite that verifies reporters notification uses a mutex-protected snapshot during reset, using a fake mutex and a dummy reporter
  • Ensure global reporters and reporters_mutex state are saved and restored in test setup/teardown
test/unit/reporters/html_reporter_test.rb
test/unit/reporters_mutex_test.rb
Protect reporter notification with a mutex-backed snapshot instead of iterating directly on the shared reporters collection.
  • Introduce a reporters_mutex with lazy initialization to guard access to the reporters collection
  • Change notify_reporters to take a mutex-protected duplicate of reporters and iterate over that snapshot
  • Simplify early-return condition to work on the snapshot rather than the shared collection
lib/capybara_screenshot_diff/screenshot_assertion.rb
Document snap_diff thread-safety model and guidance for running under threaded parallel tests.
  • Add THREAD_SAFETY.md describing per-thread registries, reporter notification snapshots, HTML reporter locking, screenshot naming isolation, and configuration expectations
  • Explain lifecycle of parallel test execution and provide usage examples and best practices for configuration and registry usage
docs/THREAD_SAFETY.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 12, 2026

Warning

Rate limit exceeded

@pftg has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 8 minutes and 43 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 8 minutes and 43 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9d7086df-1766-4e79-9178-ee20ac2f1309

📥 Commits

Reviewing files that changed from the base of the PR and between b4f043c and 18c9641.

📒 Files selected for processing (5)
  • docs/thread_safety.md
  • lib/capybara_screenshot_diff/reporters/html.rb
  • lib/capybara_screenshot_diff/screenshot_assertion.rb
  • test/unit/reporters/html_reporter_test.rb
  • test/unit/reporters_mutex_test.rb
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch emdash/discuss-thread-safe-ruby-74u

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="test/unit/reporters_mutex_test.rb" line_range="18-10" />
<code_context>
+      CapybaraScreenshotDiff.instance_variable_set(:@reporters_mutex, nil)
+    end
+
+    test "reporters notification uses mutex snapshot" do
+      fake_mutex = Class.new do
+        attr_reader :synchronize_calls
+
+        def initialize
+          @synchronize_calls = 0
+        end
+
+        def synchronize
</code_context>
<issue_to_address>
**suggestion (testing):** Broaden coverage to assert that the reporters snapshot is actually used

This test only verifies that `reporters_mutex.synchronize` is invoked; it doesn’t confirm that iteration uses the snapshot (`reporters.dup`), which is the core thread-safety behavior.

Please add a test that configures a reporter whose `record` mutates `CapybaraScreenshotDiff.reporters` (e.g., clearing it or appending another reporter) and asserts that:
- The iteration is unaffected by this mutation (only the original reporter(s) receive `record`), and
- No error is raised during notification.

That would explicitly validate that the loop operates on a snapshot rather than the live array.

Suggested implementation:

```ruby
      CapybaraScreenshotDiff.instance_variable_set(:@reporters_mutex, nil)
    end

    test "reporters notification uses mutex snapshot" do
      fake_mutex = Class.new do
        attr_reader :synchronize_calls

        def initialize
          @synchronize_calls = 0
        end

        def synchronize
          @synchronize_calls += 1
          yield
        end
      end.new

      CapybaraScreenshotDiff.instance_variable_set(:@reporters_mutex, fake_mutex)

      reporter = Class.new do
        def record(_assertions); end
      end.new

      CapybaraScreenshotDiff.reporters << reporter

      # exercise notification, ensuring it goes through the mutex
      CapybaraScreenshotDiff.notify_reporters([])

      assert_equal 1, fake_mutex.synchronize_calls
    end

    test "reporters notification iterates over reporters snapshot" do
      received = []

      mutating_reporter = Class.new do
        define_method :record do |assertions|
          received << [:original, assertions]

          # Mutate the reporters collection during notification to ensure
          # that iteration is done on a snapshot (reporters.dup), not on
          # the live array.
          CapybaraScreenshotDiff.reporters.clear
          CapybaraScreenshotDiff.reporters << Class.new {
            define_method(:record) { |a| received << [:added, a] }
          }.new
        end
      end.new

      CapybaraScreenshotDiff.reporters << mutating_reporter

      assertions = [:some, :assertions]

      # This should not raise even though reporters are mutated while iterating
      CapybaraScreenshotDiff.notify_reporters(assertions)

      # Only the originally-registered reporter should have been invoked
      assert_equal [[:original, assertions]], received
    end

```

If the notification method on `CapybaraScreenshotDiff` is named differently (e.g., `notify` or `notify_assertions` rather than `notify_reporters`), update both test cases to call the correct method. Ensure that any existing tests in this file use the same API so the naming is consistent.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread test/unit/reporters_mutex_test.rb
pftg and others added 2 commits April 12, 2026 20:28
- Eagerly initialize reporters_mutex to prevent race on lazy init
- Move @Finalized flag after write_report so finalize can retry on failure
- Use mutex-protected snapshot in at_exit hook for consistency
- Fix THREAD_SAFETY.md: use proper Ruby class names, document eager mutex
- Add real multi-threaded concurrency test (10 threads x 5 assertions)
- Add finalize retry-after-failure test

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@pftg pftg merged commit c831286 into master Apr 12, 2026
8 checks passed
@pftg pftg deleted the emdash/discuss-thread-safe-ruby-74u branch April 12, 2026 18:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant