Skip to content

fix(tests): increase test timeouts to prevent intermittent CI failures#56

Merged
leogdion merged 1 commit into48-demo-applicationfrom
SundialKit-fix-tests
Nov 4, 2025
Merged

fix(tests): increase test timeouts to prevent intermittent CI failures#56
leogdion merged 1 commit into48-demo-applicationfrom
SundialKit-fix-tests

Conversation

@leogdion
Copy link
Copy Markdown
Member

@leogdion leogdion commented Nov 4, 2025

Increase hardcoded sleep durations and improve waitUntil helper to address intermittent test failures on slow CI runners (exit code 65).

Changes:

  • NetworkMonitorTests: 50ms→200ms, 100ms→300ms, 200ms→500ms (13 occurrences)
  • ConnectivityManagerActivationTests: 200ms→500ms (3 occurrences)
  • ObserverRegistryTests: 50ms→200ms (2 occurrences)
  • waitUntil helper: polling 10ms→50ms, added debug logging for timeouts

All 51 tests pass locally with increased timeouts providing better resilience to actor scheduling delays and dispatch queue latency on CI environments.

🤖 Generated with Claude Code


Perform an AI-assisted review on CodePeer.com

Increase hardcoded sleep durations and improve waitUntil helper to address
intermittent test failures on slow CI runners (exit code 65).

Changes:
- NetworkMonitorTests: 50ms→200ms, 100ms→300ms, 200ms→500ms (13 occurrences)
- ConnectivityManagerActivationTests: 200ms→500ms (3 occurrences)
- ObserverRegistryTests: 50ms→200ms (2 occurrences)
- waitUntil helper: polling 10ms→50ms, added debug logging for timeouts

All 51 tests pass locally with increased timeouts providing better resilience
to actor scheduling delays and dispatch queue latency on CI environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Nov 4, 2025

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch SundialKit-fix-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@codecov
Copy link
Copy Markdown

codecov bot commented Nov 4, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 45.84%. Comparing base (d2a041e) to head (bec8546).
⚠️ Report is 2 commits behind head on 48-demo-application.

Additional details and impacted files
@@                 Coverage Diff                  @@
##           48-demo-application      #56   +/-   ##
====================================================
  Coverage                45.84%   45.84%           
====================================================
  Files                       30       30           
  Lines                      506      506           
====================================================
  Hits                       232      232           
  Misses                     274      274           
Flag Coverage Δ
spm 45.84% <ø> (ø)
swift-6.1-jammy 41.86% <ø> (ø)
swift-6.1-jammy-nightly 41.86% <ø> (ø)
swift-6.1-noble 41.86% <ø> (ø)
swift-6.1-noble-nightly 41.86% <ø> (ø)
swift-6.2-jammy 41.86% <ø> (ø)
swift-6.2-jammy-nightly 41.86% <ø> (ø)
swift-6.2-noble 41.86% <ø> (ø)
swift-6.2-noble-nightly 41.86% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@claude
Copy link
Copy Markdown

claude bot commented Nov 4, 2025

Code Review: Fix Test Timeouts for CI Stability

Summary

This PR addresses intermittent test failures (exit code 65) on CI runners by increasing test timeout durations and improving the waitUntil helper with better diagnostics. The changes are focused and appropriate for the stated goal.


✅ Strengths

  1. Targeted Fix: Addresses a real CI stability problem without changing test logic or production code
  2. Consistent Scaling: Timeout increases follow a consistent pattern (50ms→200ms, 100ms→300ms, 200ms→500ms)
  3. Improved Diagnostics: The waitUntil helper now tracks elapsed time, attempt count, and provides detailed timeout messages
  4. Polling Optimization: Increased poll interval (10ms→50ms) reduces CPU usage

🔍 Observations & Potential Improvements

1. Magic Numbers & Test Maintainability

The hardcoded sleep values are scattered across 68 lines in 4 files. Consider centralizing timeout constants in a TestTimeouts enum for better maintainability and self-documentation.

2. Platform-Specific Considerations

The CI matrix tests on multiple platforms including watchOS simulators which are typically slower. Monitor if these timeouts are sufficient across all platforms.

3. waitUntil Helper Enhancement

The improved waitUntil is excellent. Consider adding file/line tracking parameters (#file, #line) for even better debugging in CI logs.

4. Alternative Pattern

For actor state propagation, consider using waitUntil with condition checking instead of fixed sleeps where applicable. This is more deterministic and can be faster.


🛡️ Test Coverage

✅ All 51 tests pass locally
✅ No test logic changes
✅ Improved diagnostics for future issues

🔒 Security

✅ No security implications (test-only changes)

⚡ Performance

Minor increase in test execution time (~3-5s total). Acceptable tradeoff for reliability.


📋 Recommendations

Must Have (Before Merge):
✅ All changes are appropriate and ready to merge as-is

Should Have (Follow-up PR):

  1. Extract timeout constants to centralized TestTimeouts enum
  2. Consider platform-specific timeouts if watchOS shows instability
  3. Add file/line tracking to waitUntil helper

Nice to Have (Future):

  1. Replace more fixed sleeps with waitUntil pattern
  2. Add CI environment detection for auto-adjusting timeouts
  3. Track test performance metrics

✅ Final Verdict

Approved with suggestions for follow-up improvements.

This PR successfully addresses the immediate CI stability issue. Timeout increases are reasonable and well-documented. The improved waitUntil helper adds valuable diagnostics.

Merge Confidence: High ✅

@leogdion leogdion merged commit c444dc5 into 48-demo-application Nov 4, 2025
27 checks passed
@leogdion leogdion deleted the SundialKit-fix-tests branch November 6, 2025 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant