Skip to content


Subversion checkout URL

You can clone with
Download ZIP
Browse files

Epically failed attempt to fix spurious test failures on a Mac

  • Loading branch information...
commit 5d8e6e5c567f3d92db816a8cd9652fad4a167357 1 parent 4bc10c8
@andreyvit andreyvit authored
@@ -96,7 +96,11 @@ To run the tests, use:
rake test
-This is expected to work on all 3 platforms, and shouldn't give any failures. (Although Mac version sometimes gives spurious failures on the monitoring test because the startup delay is not reliable.)
+This is expected to work on all platforms and shouldn't give any failures, EXCEPT that on a Mac spurious test failures occur in about 1–3 of 20 test runs, and we are unable to get rid of them.
+You can use `./testloop` script to run the tests multiple times in a row to check for unreliable behaviors.
+To give a more context on Mac test failures, two constants that have an effect on them are `STARTUP_DELAY` in `lib/em-dir-watcher/platform/mac.rb` and `UNIT_DELAY` in `tests/test_monitor.rb`. We've settled on a sweet spot of `0.5`/`0.5`, which gives a 5%–15% failure rate. Increasing them to `1.0`/`1.0` still results in the same failure rate. Decreasing them to `0.2`/`0.2` results in 30% failed test runs.
Help Wanted aka TODO
6 lib/em-dir-watcher/platform/mac.rb
@@ -7,6 +7,8 @@ module Mac
class Watcher
attr_accessor :handler, :active
def initialize path, inclusions, exclusions
@@ -27,7 +29,9 @@ def initialize path, inclusions, exclusions
yield path, refresh_subtree
- @invoker.additional_delay = 0.1
+ # Mac OS X seems to require this delay till it really starts listening for file system changes.
+ # See README for explaination of the effect.
+ @invoker.additional_delay = STARTUP_DELAY
def when_ready_to_use &ready_to_use_handler
4 test/test_monitor.rb
@@ -2,7 +2,9 @@
class TestMonitor < Test::Unit::TestCase
- UNIT_DELAY = 0.2
+ # A sufficiently reliable maximum file system change reporting lag, in seconds.
+ # See README for explaination of its effect.
+ UNIT_DELAY = 0.5
def setup
FileUtils.rm_rf TEST_DIR
Please sign in to comment.
Something went wrong with that request. Please try again.