Skip to content
This repository

New adapter for Windows and safer specs for Travis #56

Merged
merged 16 commits into from over 1 year ago

7 participants

Maher Sallam Don't Add Me To Your Organization a.k.a The Travis Bot Michael Kessler Rémy Coutable Thibaud Guillaume-Gentil Coveralls
Maher Sallam
Collaborator

Hello everyone,

For the last couple of weeks I've been working on a new gem called WDM to fix the performance issue Listen had on Windows. I've benchmarked it against FChange and the results were encouraging for me to keep developing it. WDM has some disadvantages too, so here is the quick run down:

Advantages of WDM:

  • It's 65175% faster than FChange in reporting changes.
  • It can report more changes.
  • It simplified the specs because it can detect changes recursively (changes in sub-directories).
  • It reports the path of the change and the type of the change.

Disadvantages:

  • It only works on ruby > 1.9.2.
  • It's an extension, so it needs Devkit on Windows to be installed. But then again, FChange relies on FFI and that's also an extension.
  • It's an MRI extension, so it won't run on JRuby or the other implementation for the moment.
  • It can't be included as a dependency in the gemspec of Listen as it will blow up while compiling (I still can't find a way in rubygems to specify that a gem can only be compiled on Windows).

WDM has been stable enough for the past 3 days and didn't crash once while I was using it.

As you may know, Listen has had a performance issue on Windows. Sometimes, it didn't even report changes. So I decided to implement the windows adapter in Listen with WDM and see how it works. It worked quite good and all the specs pass.

I also took the opportunity to make an end to the "russian roulette" we were playing with travis (thanks @netzpirat for this great description :) ). Adapters can now report changes on demand instead of using the latency, which allows us to wait until all tests has run and then check the results. The only disadvantage of this is that tests now require the developer to specify how many change he expects to get.

@guard/listen I'm wondering what you think about these changes. I would say WDM is in the alpha phase right now, so do you think it's a good decision to replace FChange with WDM any time soon? Also, what do you think about the fix for the travis problem?

added some commits July 22, 2012
Maher Sallam Replace FChange with WDM da06b26
Maher Sallam Stop relying on luck when testing adapters
Adapters can now be set to not report changes until explicitly commanded
to. This allows waiting for the expected amount of changes in tests.
2501fc8
Maher Sallam Add the ability to skip tests 7dc2c26
Maher Sallam Don't silent exceptions inside threads d23621d
Maher Sallam Lower tests latency
This latency has another meaning in tests: It's the interval between
checking if the expected amount of changes is reached. So lowering it
translates into faster tests!
aad355e
Maher Sallam Add manual reporting of changes to the Linux adapter b1e2607
Maher Sallam Add manual reporting of changes to the Darwin adapter 38eec03
Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 38eec03 into e985b25).

Michael Kessler
Owner

Oh wow, great work @Maher4Ever

It only works on ruby > 1.9.2.

Ruby 1.8.7 EOL is in 10 months, so it's not a big disadvantage.

It's an MRI extension, so it won't run on JRuby or the other implementation for the moment.

Oh, I think I can help with this and add Java 7 WatchService API support to listen and JRuby is a first class citizen again on all plattforms. I anyway planned to give some JRuby love to Guard, since it's a confession of failure currently.

It can't be included as a dependency in the gemspec of Listen as it will blow up while compiling (I still can't find a way in rubygems to specify that a gem can only be compiled on Windows).

I do not like these dependencies either and I'm in favor of #54

WDM has been stable enough for the past 3 days and didn't crash once while I was using it.

MERGE!

do you think it's a good decision to replace FChange with WDM any time soon?

Sounds like it is and the ultimate reason to do so is that it's maintained by you.

Also, what do you think about the fix for the travis problem?

I'd not care to specify the expected change count, since we do similar with mocks, but I prefer to have solid specs :P

Great work Maher.

Rémy Coutable
Owner
rymai commented July 26, 2012

Awesome @Maher4Ever!!! Thanks for taking care of Windows users! :)

Thibaud Guillaume-Gentil
Owner

Nice work, all green for me :)

Maher Sallam
Collaborator

Glad you all liked it :). I'll merge this after fixing #54

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 2b7e352 into e985b25).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged cbb2d50 into e985b25).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged d7ca568 into e985b25).

Maher Sallam Fix specs on rubinius
Use a patched version of rb-inotify until a new version is released.
ee7167d
Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged ee7167d into e985b25).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 4955b8c into e985b25).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request fails (merged 91e7eb1 into e985b25).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged a62e0b1 into e985b25).

Don't Add Me To Your Organization a.k.a The Travis Bot

This pull request passes (merged ed6a39a into e985b25).

Thibaud Guillaume-Gentil
Owner

Ready to be merged? :)

Maher Sallam
Collaborator

Yes it is :). Could you just confirm it works one more time (because of the new changes to WDM) so we don't end up segfaulting on user's machines?

Thibaud Guillaume-Gentil
Owner

I'll try to give it a try this evening, thanks!

Thibaud Guillaume-Gentil
Owner

@Maher4Ever all specs are green on OS X! Congrats!

Maher Sallam
Collaborator

@thibaudgg Awesome! Thanks for testing :)

Maher Sallam Maher4Ever merged commit c37d9aa into from August 17, 2012
Maher Sallam Maher4Ever closed this August 17, 2012

Hi, how is the new dependency manager supposed to work with guard? I just realized that on Ubuntu, the rb-inotify, rb-fchange, rb-fsevent gems are no longer included, and a warning will be shown:

> [Listen warning]:
  Missing dependency 'rb-inotify' (version '~> 0.8.8')!
  Please add the following to your Gemfile to satisfy the dependency:
    gem 'rb-inotify', '~> 0.8.8'

  For a better performance, it's recommended that you satisfy the missing dependency.
  Listen will be polling changes. Learn more at https://github.com/guard/listen#polling-fallback.

Note that this is not immediately noticeable for those who updated from older versions of guard, since those gems may still be present in Gemfile.lock

Just add

gem 'rb-inotify', '~> 0.8.8'

before gem 'guard' and you'll be good to go.

Coveralls

Coverage Status

Changes Unknown when pulling ed6a39a on wdm into ** on master**.

Coveralls

Coverage Status

Changes Unknown when pulling ed6a39a on wdm into ** on master**.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
This page is out of date. Refresh to see the latest.
5  .travis.yml
@@ -11,7 +11,10 @@ rvm:
11 11
   - rbx-18mode
12 12
   - rbx-19mode
13 13
 bundler_args: --without=development
14  
-env: TEST_LATENCY=1.0
  14
+matrix:
  15
+  allow_failures:
  16
+    - rvm: ruby-head
  17
+    - rvm: jruby-head
15 18
 notifications:
16 19
   recipients:
17 20
     - thibaud@thibaud.me
8  Gemfile
... ...
@@ -1,9 +1,15 @@
  1
+require 'rbconfig'
  2
+
1 3
 source :rubygems
2 4
 
3 5
 gemspec
4 6
 
5 7
 gem 'rake'
6 8
 
  9
+gem 'rb-fsevent', '~> 0.9.1' if RbConfig::CONFIG['target_os'] =~ /darwin(1.+)?$/i
  10
+gem 'rb-inotify', '~> 0.8.8', :github => 'mbj/rb-inotify' if RbConfig::CONFIG['target_os'] =~ /linux/i
  11
+gem 'wdm',        '~> 0.0.3' if RbConfig::CONFIG['target_os'] =~ /mswin|mingw/i
  12
+
7 13
 group :development do
8 14
   platform :ruby do
9 15
     gem 'coolline'
@@ -20,4 +26,4 @@ end
20 26
 
21 27
 group :test do
22 28
   gem 'rspec'
23  
-end
  29
+end
1  README.md
Source Rendered
@@ -251,7 +251,6 @@ want to force the use of the polling adapter, either use the `:force_polling` op
251 251
 while initializing the listener or call the `force_polling` method on your listener
252 252
 before starting it.
253 253
 
254  
-<a name="fallback"/>
255 254
 ## Polling fallback
256 255
 
257 256
 When a OS-specific adapter doesn't work the Listen gem automatically falls back to the polling adapter.
11  lib/listen.rb
... ...
@@ -1,10 +1,11 @@
1 1
 module Listen
2 2
 
3  
-  autoload :Turnstile,       'listen/turnstile'
4  
-  autoload :Listener,        'listen/listener'
5  
-  autoload :MultiListener,   'listen/multi_listener'
6  
-  autoload :DirectoryRecord, 'listen/directory_record'
7  
-  autoload :Adapter,         'listen/adapter'
  3
+  autoload :Turnstile,         'listen/turnstile'
  4
+  autoload :Listener,          'listen/listener'
  5
+  autoload :MultiListener,     'listen/multi_listener'
  6
+  autoload :DirectoryRecord,   'listen/directory_record'
  7
+  autoload :DependencyManager, 'listen/dependency_manager'
  8
+  autoload :Adapter,           'listen/adapter'
8 9
 
9 10
   module Adapters
10 11
     autoload :Darwin,  'listen/adapters/darwin'
99  lib/listen/adapter.rb
@@ -10,8 +10,15 @@ class Adapter
10 10
     # The default delay between checking for changes.
11 11
     DEFAULT_LATENCY = 0.25
12 12
 
  13
+    # The default warning message when there is a missing dependency.
  14
+    MISSING_DEPENDENCY_MESSAGE = <<-EOS.gsub(/^\s*/, '')
  15
+      For a better performance, it's recommended that you satisfy the missing dependency.
  16
+    EOS
  17
+
13 18
     # The default warning message when falling back to polling adapter.
14  
-    POLLING_FALLBACK_MESSAGE = "WARNING: Listen has fallen back to polling, learn more at https://github.com/guard/listen#fallback."
  19
+    POLLING_FALLBACK_MESSAGE = <<-EOS.gsub(/^\s*/, '')
  20
+      Listen will be polling changes. Learn more at https://github.com/guard/listen#polling-fallback.
  21
+    EOS
15 22
 
16 23
     # Selects the appropriate adapter implementation for the
17 24
     # current OS and initializes it.
@@ -31,18 +38,26 @@ class Adapter
31 38
     def self.select_and_initialize(directories, options = {}, &callback)
32 39
       return Adapters::Polling.new(directories, options, &callback) if options.delete(:force_polling)
33 40
 
34  
-      if Adapters::Darwin.usable_and_works?(directories, options)
35  
-        Adapters::Darwin.new(directories, options, &callback)
36  
-      elsif Adapters::Linux.usable_and_works?(directories, options)
37  
-        Adapters::Linux.new(directories, options, &callback)
38  
-      elsif Adapters::Windows.usable_and_works?(directories, options)
39  
-        Adapters::Windows.new(directories, options, &callback)
40  
-      else
41  
-        unless options[:polling_fallback_message] == false
42  
-          Kernel.warn(options[:polling_fallback_message] || POLLING_FALLBACK_MESSAGE)
  41
+      warning = ''
  42
+
  43
+      begin
  44
+        if Adapters::Darwin.usable_and_works?(directories, options)
  45
+          return Adapters::Darwin.new(directories, options, &callback)
  46
+        elsif Adapters::Linux.usable_and_works?(directories, options)
  47
+          return Adapters::Linux.new(directories, options, &callback)
  48
+        elsif Adapters::Windows.usable_and_works?(directories, options)
  49
+          return Adapters::Windows.new(directories, options, &callback)
43 50
         end
44  
-        Adapters::Polling.new(directories, options, &callback)
  51
+      rescue DependencyManager::Error => e
  52
+        warning += e.message + "\n" + MISSING_DEPENDENCY_MESSAGE
  53
+      end
  54
+
  55
+      unless options[:polling_fallback_message] == false
  56
+        warning += options[:polling_fallback_message] || POLLING_FALLBACK_MESSAGE
  57
+        Kernel.warn "[Listen warning]:\n" + warning.gsub(/^(.*)/, '  \1')
45 58
       end
  59
+
  60
+      Adapters::Polling.new(directories, options, &callback)
46 61
     end
47 62
 
48 63
     # Initializes the adapter.
@@ -50,6 +65,7 @@ def self.select_and_initialize(directories, options = {}, &callback)
50 65
     # @param [String, Array<String>] directories the directories to watch
51 66
     # @param [Hash] options the adapter options
52 67
     # @option options [Float] latency the delay between checking for changes in seconds
  68
+    # @option options [Boolean] report_changes whether or not to automatically report changes (run the callback)
53 69
     #
54 70
     # @yield [changed_dirs, options] callback Callback called when a change happens
55 71
     # @yieldparam [Array<String>] changed_dirs the changed directories
@@ -60,12 +76,13 @@ def self.select_and_initialize(directories, options = {}, &callback)
60 76
     def initialize(directories, options = {}, &callback)
61 77
       @directories  = Array(directories)
62 78
       @callback     = callback
63  
-      @latency    ||= DEFAULT_LATENCY
64  
-      @latency      = options[:latency] if options[:latency]
65 79
       @paused       = false
66 80
       @mutex        = Mutex.new
67 81
       @changed_dirs = Set.new
68 82
       @turnstile    = Turnstile.new
  83
+      @latency    ||= DEFAULT_LATENCY
  84
+      @latency      = options[:latency] if options[:latency]
  85
+      @report_changes = options[:report_changes].nil? ? true : options[:report_changes]
69 86
     end
70 87
 
71 88
     # Starts the adapter.
@@ -92,12 +109,37 @@ def started?
92 109
     end
93 110
 
94 111
     # Blocks the main thread until the poll thread
95  
-    # calls the callback.
  112
+    # runs the callback.
96 113
     #
97 114
     def wait_for_callback
98 115
       @turnstile.wait unless @paused
99 116
     end
100 117
 
  118
+    # Blocks the main thread until N changes are
  119
+    # detected.
  120
+    #
  121
+    def wait_for_changes(goal = 0)
  122
+      changes = 0
  123
+
  124
+      loop do
  125
+        @mutex.synchronize { changes = @changed_dirs.size }
  126
+
  127
+        return if @paused || @stop
  128
+        return if changes >= goal
  129
+
  130
+        sleep(@latency)
  131
+      end
  132
+    end
  133
+
  134
+    # Checks if the adapter is usable on the current OS.
  135
+    #
  136
+    # @return [Boolean] whether usable or not
  137
+    #
  138
+    def self.usable?
  139
+      load_depenencies
  140
+      dependencies_loaded?
  141
+    end
  142
+
101 143
     # Checks if the adapter is usable and works on the current OS.
102 144
     #
103 145
     # @param [String, Array<String>] directories the directories to watch
@@ -140,26 +182,29 @@ def self.works?(directory, options = {})
140 182
       adapter.stop if adapter && adapter.started?
141 183
     end
142 184
 
  185
+    # Runs the callback and passes it the changes if there are any.
  186
+    #
  187
+    def report_changes
  188
+      changed_dirs = nil
  189
+
  190
+      @mutex.synchronize do
  191
+        return if @changed_dirs.empty?
  192
+        changed_dirs = @changed_dirs.to_a
  193
+        @changed_dirs.clear
  194
+      end
  195
+
  196
+      @callback.call(changed_dirs, {})
  197
+    end
  198
+
143 199
     private
144 200
 
145 201
     # Polls changed directories and reports them back
146 202
     # when there are changes.
147 203
     #
148  
-    # @option [Boolean] recursive whether or not to pass the recursive option to the callback
149  
-    #
150  
-    def poll_changed_dirs(recursive = false)
  204
+    def poll_changed_dirs
151 205
       until @stop
152 206
         sleep(@latency)
153  
-        next if @changed_dirs.empty?
154  
-
155  
-        changed_dirs = []
156  
-
157  
-        @mutex.synchronize do
158  
-          changed_dirs = @changed_dirs.to_a
159  
-          @changed_dirs.clear
160  
-        end
161  
-
162  
-        @callback.call(changed_dirs, recursive ? {:recursive => recursive} : {})
  207
+        report_changes
163 208
         @turnstile.signal
164 209
       end
165 210
     end
21  lib/listen/adapters/darwin.rb
@@ -4,6 +4,10 @@ module Adapters
4 4
     # Adapter implementation for Mac OS X `FSEvents`.
5 5
     #
6 6
     class Darwin < Adapter
  7
+      extend DependencyManager
  8
+
  9
+      # Declare the adapter's dependencies
  10
+      dependency 'rb-fsevent', '~> 0.9.1'
7 11
 
8 12
       LAST_SEPARATOR_REGEX = /\/$/
9 13
 
@@ -25,13 +29,14 @@ def start(blocking = true)
25 29
         end
26 30
 
27 31
         @worker_thread = Thread.new { @worker.run }
28  
-        @poll_thread   = Thread.new { poll_changed_dirs }
29 32
 
30 33
         # The FSEvent worker needs sometime to startup. Turnstiles can't
31 34
         # be used to wait for it as it runs in a loop.
32 35
         # TODO: Find a better way to block until the worker starts.
33  
-        sleep @latency
34  
-        @poll_thread.join if blocking
  36
+        sleep 0.1
  37
+
  38
+        @poll_thread = Thread.new { poll_changed_dirs } if @report_changes
  39
+        @worker_thread.join if blocking
35 40
       end
36 41
 
37 42
       # Stops the adapter.
@@ -43,8 +48,8 @@ def stop
43 48
         end
44 49
 
45 50
         @worker.stop
46  
-        Thread.kill(@worker_thread) if @worker_thread
47  
-        @poll_thread.join
  51
+        @worker_thread.join if @worker_thread
  52
+        @poll_thread.join if @poll_thread
48 53
       end
49 54
 
50 55
       # Checks if the adapter is usable on the current OS.
@@ -53,11 +58,7 @@ def stop
53 58
       #
54 59
       def self.usable?
55 60
         return false unless RbConfig::CONFIG['target_os'] =~ /darwin(1.+)?$/i
56  
-
57  
-        require 'rb-fsevent'
58  
-        true
59  
-      rescue LoadError
60  
-        false
  61
+        super
61 62
       end
62 63
 
63 64
       private
63  lib/listen/adapters/linux.rb
@@ -4,6 +4,10 @@ module Adapters
4 4
     # Listener implementation for Linux `inotify`.
5 5
     #
6 6
     class Linux < Adapter
  7
+      extend DependencyManager
  8
+
  9
+      # Declare the adapter's dependencies
  10
+      dependency 'rb-inotify', '~> 0.8.8'
7 11
 
8 12
       # Watched inotify events
9 13
       #
@@ -41,8 +45,9 @@ def start(blocking = true)
41 45
         end
42 46
 
43 47
         @worker_thread = Thread.new { @worker.run }
44  
-        @poll_thread   = Thread.new { poll_changed_dirs }
45  
-        @poll_thread.join if blocking
  48
+        @poll_thread   = Thread.new { poll_changed_dirs } if @report_changes
  49
+
  50
+        @worker_thread.join if blocking
46 51
       end
47 52
 
48 53
       # Stops the adapter.
@@ -55,20 +60,16 @@ def stop
55 60
 
56 61
         @worker.stop
57 62
         Thread.kill(@worker_thread) if @worker_thread
58  
-        @poll_thread.join
  63
+        @poll_thread.join if @poll_thread
59 64
       end
60 65
 
61  
-      # Check if the adapter is usable on the current OS.
  66
+      # Checks if the adapter is usable on the current OS.
62 67
       #
63 68
       # @return [Boolean] whether usable or not
64 69
       #
65 70
       def self.usable?
66 71
         return false unless RbConfig::CONFIG['target_os'] =~ /linux/i
67  
-
68  
-        require 'rb-inotify'
69  
-        true
70  
-      rescue LoadError
71  
-        false
  72
+        super
72 73
       end
73 74
 
74 75
     private
@@ -79,29 +80,31 @@ def self.usable?
79 80
       # @return [INotify::Notifier] initialized worker
80 81
       #
81 82
       def init_worker
82  
-        worker = INotify::Notifier.new
83  
-        @directories.each do |directory|
84  
-          worker.watch(directory, *EVENTS.map(&:to_sym)) do |event|
85  
-            if @paused || (
86  
-              # Event on root directory
87  
-              event.name == ""
88  
-            ) || (
89  
-              # INotify reports changes to files inside directories as events
90  
-              # on the directories themselves too.
91  
-              #
92  
-              # @see http://linux.die.net/man/7/inotify
93  
-              event.flags.include?(:isdir) and event.flags & [:close, :modify] != []
94  
-            )
95  
-              # Skip all of these!
96  
-              next
97  
-            end
98  
-
99  
-            @mutex.synchronize do
100  
-              @changed_dirs << File.dirname(event.absolute_name)
101  
-            end
  83
+        callback = lambda do |event|
  84
+          if @paused || (
  85
+            # Event on root directory
  86
+            event.name == ""
  87
+          ) || (
  88
+            # INotify reports changes to files inside directories as events
  89
+            # on the directories themselves too.
  90
+            #
  91
+            # @see http://linux.die.net/man/7/inotify
  92
+            event.flags.include?(:isdir) and event.flags & [:close, :modify] != []
  93
+          )
  94
+            # Skip all of these!
  95
+            next
  96
+          end
  97
+
  98
+          @mutex.synchronize do
  99
+            @changed_dirs << File.dirname(event.absolute_name)
  100
+          end
  101
+        end
  102
+
  103
+        INotify::Notifier.new.tap do |worker|
  104
+          @directories.each do |directory|
  105
+            worker.watch(directory, *EVENTS.map(&:to_sym), &callback)
102 106
           end
103 107
         end
104  
-        worker
105 108
       end
106 109
 
107 110
     end
1  lib/listen/adapters/polling.rb
@@ -10,6 +10,7 @@ module Adapters
10 10
     # file IO that the other implementations.
11 11
     #
12 12
     class Polling < Adapter
  13
+      extend DependencyManager
13 14
 
14 15
       # Initialize the Adapter. See {Listen::Adapter#initialize} for more info.
15 16
       #
48  lib/listen/adapters/windows.rb
@@ -3,9 +3,13 @@
3 3
 module Listen
4 4
   module Adapters
5 5
 
6  
-    # Adapter implementation for Windows `fchange`.
  6
+    # Adapter implementation for Windows `wdm`.
7 7
     #
8 8
     class Windows < Adapter
  9
+      extend DependencyManager
  10
+
  11
+      # Declare the adapter's dependencies
  12
+      dependency 'wdm', '~> 0.0.3'
9 13
 
10 14
       # Initializes the Adapter. See {Listen::Adapter#initialize} for more info.
11 15
       #
@@ -24,9 +28,15 @@ def start(blocking = true)
24 28
           super
25 29
         end
26 30
 
27  
-        @worker_thread = Thread.new { @worker.run }
28  
-        @poll_thread   = Thread.new { poll_changed_dirs(true) }
29  
-        @poll_thread.join if blocking
  31
+        @worker_thread = Thread.new { @worker.run! }
  32
+
  33
+        # Wait for the worker to start. This is needed to avoid a deadlock
  34
+        # when stopping immediately after starting.
  35
+        sleep 0.1
  36
+
  37
+        @poll_thread = Thread.new { poll_changed_dirs } if @report_changes
  38
+
  39
+        @worker_thread.join if blocking
30 40
       end
31 41
 
32 42
       # Stops the adapter.
@@ -38,8 +48,8 @@ def stop
38 48
         end
39 49
 
40 50
         @worker.stop
41  
-        Thread.kill(@worker_thread) if @worker_thread
42  
-        @poll_thread.join
  51
+        @worker_thread.join if @worker_thread
  52
+        @poll_thread.join if @poll_thread
43 53
       end
44 54
 
45 55
       # Checks if the adapter is usable on the current OS.
@@ -48,31 +58,27 @@ def stop
48 58
       #
49 59
       def self.usable?
50 60
         return false unless RbConfig::CONFIG['target_os'] =~ /mswin|mingw/i
51  
-
52  
-        require 'rb-fchange'
53  
-        true
54  
-      rescue LoadError
55  
-        false
  61
+        super
56 62
       end
57 63
 
58 64
     private
59 65
 
60  
-      # Initializes a FChange worker and adds a watcher for
  66
+      # Initializes a WDM monitor and adds a watcher for
61 67
       # each directory passed to the adapter.
62 68
       #
63  
-      # @return [FChange::Notifier] initialized worker
  69
+      # @return [WDM::Monitor] initialized worker
64 70
       #
65 71
       def init_worker
66  
-        FChange::Notifier.new.tap do |worker|
67  
-          @directories.each do |directory|
68  
-            worker.watch(directory, :all_events, :recursive) do |event|
69  
-              next if @paused
70  
-              @mutex.synchronize do
71  
-                @changed_dirs << File.expand_path(event.watcher.path)
72  
-              end
73  
-            end
  72
+        callback = Proc.new do |change|
  73
+          next if @paused
  74
+          @mutex.synchronize do
  75
+            @changed_dirs << File.dirname(change.path)
74 76
           end
75 77
         end
  78
+
  79
+        WDM::Monitor.new.tap do |worker|
  80
+          @directories.each { |d| worker.watch_recursively(d, &callback) }
  81
+        end
76 82
       end
77 83
 
78 84
     end
126  lib/listen/dependency_manager.rb
... ...
@@ -0,0 +1,126 @@
  1
+require 'set'
  2
+
  3
+module Listen
  4
+
  5
+  # The dependency-manager offers a simple DSL which allows
  6
+  # classes to declare their gem dependencies and load them when
  7
+  # needed.
  8
+  # It raises a user-friendly exception when the dependencies
  9
+  # can't be loaded which has the install command in the message.
  10
+  #
  11
+  module DependencyManager
  12
+
  13
+    GEM_LOAD_MESSAGE = <<-EOS.gsub(/^ {6}/, '')
  14
+      Missing dependency '%s' (version '%s')!
  15
+    EOS
  16
+
  17
+    GEM_INSTALL_COMMAND = <<-EOS.gsub(/^ {6}/, '')
  18
+      Please run the following to satisfy the dependency:
  19
+        gem install --version '%s' %s
  20
+    EOS
  21
+
  22
+    BUNDLER_DECLARE_GEM = <<-EOS.gsub(/^ {6}/, '')
  23
+      Please add the following to your Gemfile to satisfy the dependency:
  24
+        gem '%s', '%s'
  25
+    EOS
  26
+
  27
+    Dependency = Struct.new(:name, :version)
  28
+
  29
+    # The error raised when a dependency can't be loaded.
  30
+    class Error < StandardError; end
  31
+
  32
+    # A list of all loaded dependencies in the dependency manager.
  33
+    @_loaded_dependencies = Set.new
  34
+
  35
+    # class methods
  36
+    class << self
  37
+
  38
+      # Initializes the extended class.
  39
+      #
  40
+      # @param [Class] the class for which some dependencies must be managed
  41
+      #
  42
+      def extended(base)
  43
+        base.class_eval do
  44
+          @_dependencies = Set.new
  45
+        end
  46
+      end
  47
+
  48
+      # Adds a loaded dependency to a list so that it doesn't have
  49
+      # to be loaded again by another classes.
  50
+      #
  51
+      # @param [Dependency] dependency
  52
+      #
  53
+      def add_loaded(dependency)
  54
+        @_loaded_dependencies << dependency
  55
+      end
  56
+
  57
+      # Returns whether the dependency is alread loaded or not.
  58
+      #
  59
+      # @param [Dependency] dependency
  60
+      # @return [Boolean]
  61
+      #
  62
+      def already_loaded?(dependency)
  63
+        @_loaded_dependencies.include?(dependency)
  64
+      end
  65
+
  66
+      # Clears the list of loaded dependencies.
  67
+      #
  68
+      def clear_loaded
  69
+        @_loaded_dependencies.clear
  70
+      end
  71
+    end
  72
+
  73
+    # Registers a new dependency.
  74
+    #
  75
+    # @param [String] name the name of the gem
  76
+    # @param [String] version the version of the gem
  77
+    #
  78
+    def dependency(name, version)
  79
+      @_dependencies << Dependency.new(name, version)
  80
+    end
  81
+
  82
+    # Loads the registered dependencies.
  83
+    #
  84
+    # @raise DependencyManager::Error if the dependency can't be loaded.
  85
+    #
  86
+    def load_depenencies
  87
+      @_dependencies.each do |dependency|
  88
+        begin
  89
+          next if DependencyManager.already_loaded?(dependency)
  90
+          gem(dependency.name, dependency.version)
  91
+          require(dependency.name)
  92
+          DependencyManager.add_loaded(dependency)
  93
+          @_dependencies.delete(dependency)
  94
+        rescue Gem::LoadError
  95
+          args = [dependency.name, dependency.version]
  96
+          command = if running_under_bundler?
  97
+            BUNDLER_DECLARE_GEM % args
  98
+          else
  99
+            GEM_INSTALL_COMMAND % args.reverse
  100
+          end
  101
+          message = GEM_LOAD_MESSAGE % args
  102
+
  103
+          raise Error.new(message + command)
  104
+        end
  105
+      end
  106
+    end
  107
+
  108
+    # Returns whether all the dependencies has been loaded or not.
  109
+    #
  110
+    # @return [Boolean]
  111
+    #
  112
+    def dependencies_loaded?
  113
+      @_dependencies.empty?
  114
+    end
  115
+
  116
+    private
  117
+
  118
+    # Returns whether we are running under bundler or not
  119
+    #
  120
+    # @return [Boolean]
  121
+    #
  122
+    def running_under_bundler?
  123
+      !!(File.exists?('Gemfile') && ENV['BUNDLE_GEMFILE'])
  124
+    end
  125
+  end
  126
+end
4  listen.gemspec
@@ -15,10 +15,6 @@ Gem::Specification.new do |s|
15 15
   s.required_rubygems_version = '>= 1.3.6'
16 16
   s.rubyforge_project = 'listen'
17 17
 
18  
-  s.add_dependency 'rb-fsevent', '~> 0.9.1'
19  
-  s.add_dependency 'rb-inotify', '~> 0.8.8'
20  
-  s.add_dependency 'rb-fchange', '~> 0.0.5'
21  
-
22 18
   s.add_development_dependency 'bundler'
23 19
 
24 20
   s.files        = Dir.glob('{lib}/**/*') + %w[CHANGELOG.md LICENSE README.md]
27  spec/listen/adapter_spec.rb
@@ -31,14 +31,27 @@
31 31
         described_class.select_and_initialize('dir')
32 32
       end
33 33
 
34  
-      it "warns with the default polling fallback message" do
35  
-        Kernel.should_receive(:warn).with(Listen::Adapter::POLLING_FALLBACK_MESSAGE)
  34
+      it 'warns with the default polling fallback message' do
  35
+        Kernel.should_receive(:warn).with(/#{Listen::Adapter::POLLING_FALLBACK_MESSAGE}/)
36 36
         described_class.select_and_initialize('dir')
37 37
       end
38 38
 
  39
+      context 'when the dependencies of an adapter are not satisfied' do
  40
+        before do
  41
+          Listen::Adapters::Darwin.stub(:usable_and_works?).and_raise(Listen::DependencyManager::Error)
  42
+          Listen::Adapters::Linux.stub(:usable_and_works?).and_raise(Listen::DependencyManager::Error)
  43
+          Listen::Adapters::Windows.stub(:usable_and_works?).and_raise(Listen::DependencyManager::Error)
  44
+        end
  45
+
  46
+        it 'invites the user to satisfy the dependencies of the adapter in the warning' do
  47
+          Kernel.should_receive(:warn).with(/#{Listen::Adapter::MISSING_DEPENDENCY_MESSAGE}/)
  48
+          described_class.select_and_initialize('dir')
  49
+        end
  50
+      end
  51
+
39 52
       context "with custom polling_fallback_message option" do
40 53
         it "warns with the custom polling fallback message" do
41  
-          Kernel.should_receive(:warn).with('custom')
  54
+          Kernel.should_receive(:warn).with(/custom/)
42 55
           described_class.select_and_initialize('dir', :polling_fallback_message => 'custom')
43 56
         end
44 57
       end
@@ -102,6 +115,14 @@
102 115
 
103 116
   [Listen::Adapters::Darwin, Listen::Adapters::Linux, Listen::Adapters::Windows].each do |adapter_class|
104 117
     if adapter_class.usable?
  118
+      describe '.usable?' do
  119
+        it 'checks the dependencies' do
  120
+          adapter_class.should_receive(:load_depenencies)
  121
+          adapter_class.should_receive(:dependencies_loaded?)
  122
+          adapter_class.usable?
  123
+        end
  124
+      end
  125
+
105 126
       describe '.usable_and_works?' do
106 127
         it 'checks if the adapter is usable' do
107 128
           adapter_class.stub(:works?)
2  spec/listen/adapters/windows_spec.rb
@@ -7,7 +7,7 @@
7 7
     end
8 8
 
9 9
     it_should_behave_like 'a filesystem adapter'
10  
-    it_should_behave_like 'an adapter that call properly listener#on_change', :recursive => true, :adapter => :windows
  10
+    it_should_behave_like 'an adapter that call properly listener#on_change'
11 11
   end
12 12
 
13 13
   if mac?
107  spec/listen/dependency_manager_spec.rb
... ...
@@ -0,0 +1,107 @@
  1
+require 'spec_helper'
  2
+
  3
+describe Listen::DependencyManager do
  4
+  let(:dependency) { Listen::DependencyManager::Dependency.new('listen', '~> 0.0.1') }
  5
+
  6
+  subject { Class.new { extend Listen::DependencyManager } }
  7
+
  8
+  before { described_class.clear_loaded }
  9
+
  10
+  describe '.add_loaded' do
  11
+    it 'adds a dependency to the list of loaded dependencies' do
  12
+      described_class.add_loaded dependency
  13
+      described_class.already_loaded?(dependency).should be_true
  14
+    end
  15
+  end
  16
+
  17
+  describe '.already_loaded?' do
  18
+    it 'returns false when a dependency is not in the list of loaded dependencies' do
  19
+      described_class.already_loaded?(dependency).should be_false
  20
+    end
  21
+
  22
+    it 'returns true when a dependency is in the list of loaded dependencies' do
  23
+      described_class.add_loaded dependency
  24
+      described_class.already_loaded?(dependency).should be_true
  25
+    end
  26
+  end
  27
+
  28
+  describe '.clear_loaded' do
  29
+    it 'clears the whole list of loaded dependencies' do
  30
+      described_class.add_loaded dependency
  31
+      described_class.already_loaded?(dependency).should be_true
  32
+      described_class.clear_loaded
  33
+      described_class.already_loaded?(dependency).should be_false
  34
+    end
  35
+  end
  36
+
  37
+  describe '#dependency' do
  38
+    it 'registers a new dependency for the managed class' do
  39
+      subject.dependency 'listen', '~> 0.0.1'
  40
+      subject.dependencies_loaded?.should be_false
  41
+    end
  42
+  end
  43
+
  44
+  describe '#load_depenencies' do
  45
+    before { subject.dependency 'listen', '~> 0.0.1' }
  46
+
  47
+    context 'when dependencies can be loaded' do
  48
+      before { subject.stub(:gem, :require) }
  49
+
  50
+      it 'loads all the registerd dependencies' do
  51
+        subject.load_depenencies
  52
+        subject.dependencies_loaded?.should be_true
  53
+      end
  54
+    end
  55
+
  56
+    context 'when dependencies can not be loaded' do
  57
+      it 'raises an error' do
  58
+        expect {
  59
+          subject.load_depenencies
  60
+        }.to raise_error(described_class::Error)
  61
+      end
  62
+
  63
+      context 'when running under bundler' do
  64
+        before { subject.should_receive(:running_under_bundler?).and_return(true) }
  65
+
  66
+        it 'includes the Gemfile declaration to satisfy the dependency' do
  67
+          begin
  68
+            subject.load_depenencies
  69
+          rescue described_class::Error => e
  70
+            e.message.should include("gem 'listen', '~> 0.0.1'")
  71
+          end
  72
+        end
  73
+      end
  74
+
  75
+      context 'when not running under bundler' do
  76
+        before { subject.should_receive(:running_under_bundler?).and_return(false) }
  77
+
  78
+        it 'includes the command to install the dependency' do
  79
+          begin
  80
+            subject.load_depenencies
  81
+          rescue described_class::Error => e
  82
+            e.message.should include("gem install --version '~> 0.0.1' listen")
  83
+          end
  84
+        end
  85
+      end
  86
+    end
  87
+  end
  88
+
  89
+  describe '#dependencies_loaded?' do
  90
+    it 'return false when dependencies are not loaded' do
  91
+      subject.dependency 'listen', '~> 0.0.1'
  92
+      subject.dependencies_loaded?.should be_false
  93
+    end
  94
+
  95
+    it 'return true when dependencies are loaded' do
  96
+      subject.stub(:gem, :require)
  97
+
  98
+      subject.dependency 'listen', '~> 0.0.1'
  99
+      subject.load_depenencies
  100
+      subject.dependencies_loaded?.should be_true
  101
+    end
  102
+
  103
+    it 'return true when there are no dependencies to load' do
  104
+      subject.dependencies_loaded?.should be_true
  105
+    end
  106
+  end
  107
+end
4  spec/listen/directory_record_spec.rb
@@ -193,7 +193,7 @@
193 193
     it 'returns nil when the passed path is not inside the base-directory' do
194 194
       subject.relative_to_base('/tmp/some_random_path').should be_nil
195 195
     end
196  
-    
  196
+
197 197
     context 'when containing regexp characters in the base directory' do
198 198
       before do
199 199
         fixtures do |path|
@@ -1104,7 +1104,7 @@
1104 1104
       end
1105 1105
     end
1106 1106
 
1107  
-    context 'with symlinks' do
  1107
+    context 'with symlinks', :unless => windows? do
1108 1108
       it 'looks at symlinks not their targets' do
1109 1109
         fixtures do |path|
1110 1110
           touch 'target'
8  spec/spec_helper.rb
... ...
@@ -1,7 +1,5 @@
1 1
 require 'listen'
2 2
 
3  
-ENV["TEST_LATENCY"] ||= "0.25"
4  
-
5 3
 Dir["#{File.dirname(__FILE__)}/support/**/*.rb"].each { |f| require f }
6 4
 
7 5
 # See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
@@ -11,8 +9,12 @@
11 9
   config.filter_run :focus => true
12 10
   config.treat_symbols_as_metadata_keys_with_true_values = true
13 11
   config.run_all_when_everything_filtered = true
  12
+  config.filter_run_excluding :broken => true
14 13
 end
15 14
 
16 15
 def test_latency
17  
-  ENV["TEST_LATENCY"].to_f
  16
+  0.1
18 17
 end
  18
+
  19
+# Crash loud in tests!
  20
+Thread.abort_on_exception = true
337  spec/support/adapter_helper.rb
@@ -3,18 +3,29 @@
3 3
 # @param [Listen::Listener] listener the adapter listener
4 4
 # @param [String] path the path to watch
5 5
 #
6  
-def watch(listener, *paths)
7  
-  callback = lambda { |changed_dirs, options| @called = true; listener.on_change(changed_dirs, options) }
8  
-  @adapter = Listen::Adapter.select_and_initialize(paths, { :latency => test_latency }, &callback)
  6
+def watch(listener, expected_changes, *paths)
  7
+  callback = lambda { |changed_dirs, options| @called = true; listener.on_change(changed_dirs) }
  8
+  @adapter = Listen::Adapter.select_and_initialize(paths, { :report_changes => false, :latency => test_latency }, &callback)
  9
+
  10
+  forced_stop = false
  11
+  prevent_deadlock = Proc.new { sleep(10); puts "Forcing stop"; @adapter.stop; forced_stop = true }
  12
+
9 13
   @adapter.start(false)
10 14
 
11 15
   yield
12 16
 
13  
-  t = Thread.new { sleep(test_latency * 5); @adapter.stop }
14  
-  @adapter.wait_for_callback
  17
+  t = Thread.new(&prevent_deadlock)
  18
+  @adapter.wait_for_changes(expected_changes)
  19
+
  20
+  unless forced_stop
  21
+    Thread.kill(t)
  22
+    @adapter.report_changes
  23
+  end
15 24
 ensure
16  
-  Thread.kill(t) if t
17  
-  @adapter.stop
  25
+  unless forced_stop
  26
+    Thread.kill(t) if t
  27
+    @adapter.stop
  28
+  end
18 29
 end
19 30
 
20 31
 shared_examples_for 'a filesystem adapter' do
@@ -61,7 +72,7 @@ def watch(listener, *paths)
61 72
 
62 73
     context 'with a started adapter' do
63 74
       before { subject.start(false) }
64  
-      after { subject.start }
  75
+      after  { subject.stop }
65 76
 
66 77
       it 'returns true' do
67 78
         subject.should be_started
@@ -79,30 +90,26 @@ def watch(listener, *paths)
79 90
     context 'when a file is created' do
80 91
       it 'detects the added file' do
81 92
         fixtures do |path|
82  
-          if options[:recursive]
83  
-            listener.should_receive(:on_change).once.with([path], :recursive => true)
84  
-          else
85  
-            listener.should_receive(:on_change).once.with([path], {})
  93
+          listener.should_receive(:on_change).once.with do |array|
  94
+            array.should include(path)
86 95
           end
87 96
 
88  
-          watch(listener, path) do
  97
+          watch(listener, 1, path) do
89 98
             touch 'new_file.rb'
90 99
           end
91 100
         end
92 101
       end
93 102
 
94  
-      context 'given a symlink' do
  103
+      context 'given a symlink', :unless => windows? do
95 104
         it 'detects the added file' do
96 105
           fixtures do |path|
97  
-            if options[:recursive]
98  
-              listener.should_receive(:on_change).once.with([path], :recursive => true)
99  
-            else
100  
-              listener.should_receive(:on_change).once.with([path], {})
  106
+            listener.should_receive(:on_change).once.with do |array|
  107
+              array.should include(path)
101 108
             end
102 109
 
103 110
             touch 'new_file.rb'
104 111
 
105  
-            watch(listener, path) do
  112
+            watch(listener, 1, path) do
106 113
               ln_s 'new_file.rb', 'new_file_symlink.rb'
107 114
             end
108 115
           end
@@ -112,18 +119,14 @@ def watch(listener, *paths)
112 119
       context 'given a new created directory' do
113 120
         it 'detects the added file' do
114 121
           fixtures do |path|
115  
-            if options[:recursive]
116  
-              listener.should_receive(:on_change).once.with([path], :recursive => true)
117  
-            else
118  
-              listener.should_receive(:on_change).once.with do |array, options|
119  
-                array.should =~ [path, "#{path}/a_directory"]
120  
-              end
  122
+            listener.should_receive(:on_change).once.with do |array|
  123
+              array.should include(path, "#{path}/a_directory")
121 124
             end
122 125
 
123  
-            watch(listener, path) do
  126
+            watch(listener, 2, path) do
124 127
               mkdir 'a_directory'
125 128
               # Needed for INotify, because of :recursive rb-inotify custom flag?
126  
-              sleep 0.05 if @adapter.is_a?(Listen::Adapters::Linux)
  129
+              sleep 0.05
127 130
               touch 'a_directory/new_file.rb'
128 131
             end
129 132
           end
@@ -133,15 +136,13 @@ def watch(listener, *paths)
133 136
       context 'given an existing directory' do
134 137
         it 'detects the added file' do
135 138
           fixtures do |path|
136  
-            if options[:recursive]
137  
-              listener.should_receive(:on_change).once.with([path], :recursive => true)
138  
-            else
139  
-              listener.should_receive(:on_change).once.with(["#{path}/a_directory"], {})
  139
+            listener.should_receive(:on_change).once.with do |array|
  140
+              array.should include("#{path}/a_directory")
140 141
             end
141 142
 
142 143
             mkdir 'a_directory'
143 144
 
144  
-            watch(listener, path) do
  145
+            watch(listener, 1, path) do
145 146
               touch 'a_directory/new_file.rb'
146 147
             end
147 148
           end
@@ -151,15 +152,13 @@ def watch(listener, *paths)
151 152
       context 'given a directory with subdirectories' do
152 153
         it 'detects the added file' do
153 154
           fixtures do |path|
154  
-            if options[:recursive]
155  
-              listener.should_receive(:on_change).once.with([path], :recursive => true)
156  
-            else
157  
-              listener.should_receive(:on_change).once.with(["#{path}/a_directory/subdirectory"], {})
  155
+            listener.should_receive(:on_change).once.with do |array|
  156
+              array.should include("#{path}/a_directory/subdirectory")
158 157
             end