fix: Flaky StopEvent test

[toby-coleman]

Summary

Fixes flaky StopEvent test when run locally on Mac.

Changes

• Allow ±1 tolerance on the test.

[gemini-code-assist]

Hello @toby-coleman, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

Summary of Changes

As gemini-code-assist, I've reviewed this pull request. The primary goal is to address a flaky StopEvent integration test, specifically when it fails intermittently, particularly noted when run locally on macOS. The fix involves adjusting the assertions within the test to allow for a small tolerance, accounting for potential timing variations that can cause the strict equality checks to fail.

Highlights

• Fix Flaky Test: The main purpose of this PR is to resolve flakiness in the StopEvent integration test (test_process_stop_event.py) by making the assertions more robust against minor timing variations.
• Add Assertion Tolerance: Strict equality assertions (==) for component step counts and input/output values are replaced with pytest.approx with an absolute tolerance of 1. This allows the test to pass even if the final counts are off by one due to timing.

Changelog

• tests/integration/test_process_stop_event.py
  • Replaced the strict equality assertion for comp_a.step_count (line 107) with pytest.approx(..., abs=1) to allow for a tolerance of +/- 1.
  • Replaced the strict equality assertion for c.step_count within the loop for B components (line 115) with pytest.approx(..., abs=1).
  • Replaced the strict equality assertion for comp_a.out_1 (line 119) with pytest.approx(..., abs=1).
  • Replaced the strict equality assertion for c.in_1 within the loop for B components (line 123) with pytest.approx(..., abs=1).
  • Added a comment explaining the tolerance addition (line 106).

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

---

Test fails, then it's green,
A flaky bug, rarely seen.
Add tolerance small,
Now tests stand up tall,
Consistent, no more chagrin.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Copilot]

Pull Request Overview

This PR fixes the flaky StopEvent test by relaxing assertion tolerances to allow for a ±1 variance, improving its stability on Mac systems.

• Adjusts several assertions using pytest.approx to allow a tolerance of ±1.
• Ensures that both step counts and output/input values account for slight deviations during asynchronous operations.

[gemini-code-assist]

Code Review

This PR addresses a flaky test by introducing a small, fixed tolerance using pytest.approx. This is an excellent approach for handling minor variations in test outcomes that can occur due to timing sensitivities, especially in asynchronous systems or across different environments like the Mac environment mentioned in the PR description. The change is clear, targeted, and utilizes pytest features appropriately. Well done on making the test suite more robust!

Summary of Findings

• Test Flakiness Resolution: The PR effectively addresses test flakiness by using pytest.approx with a small absolute tolerance of 1. This is a standard and appropriate method for making tests more robust against minor, non-deterministic variations in asynchronous or time-sensitive operations.
• Code Commenting (Minor Point): The added comment # Allow a tolerance of +/- 1 on line 106 of tests/integration/test_process_stop_event.py is understandable. While pytest.approx(..., abs=1) is largely self-explanatory, the comment doesn't harm. Its selective application to only the first modified assertion, and not the subsequent ones that also use pytest.approx, is a minor stylistic observation. (Not added as a review comment due to severity settings restricting comments to medium, high, or critical.)

Merge Readiness

The changes in this pull request are well-implemented and directly address the issue of test flakiness. The use of pytest.approx is appropriate and enhances the reliability of the test. I believe the code is in good shape and ready to be merged. As per standard practice, please ensure any other required reviews and CI checks pass before merging. I am not authorized to approve the pull request.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

📢 Thoughts on this report? Let us know!