On OSX changes take a couple seconds to be noticed

[jeluard]

I can see barbarywatchservice is used as a WatchService implementation on OSX. On my machine it still takes a couple seconds before a change is noticed.

There is another way to reduce the time to notice a file change that works pretty well for me (JVM 1.8.0-b132):

(ns ...
   (:import com.sun.nio.file.SensitivityWatchEventModifier))

(.register p ws
             (into-array (type StandardWatchEventKinds/ENTRY_MODIFY) (vector StandardWatchEventKinds/ENTRY_MODIFY))
             (into-array (type SensitivityWatchEventModifier/HIGH) (vector SensitivityWatchEventModifier/HIGH)))

From my experience using this trick changes are noticed almost instantaneously. It would also reduce the codebase complexity. The downside is that it introduces a dependency on com.sun.nio that might need some special care for VM not shipping with this package.

[micha]

Unless I'm mistaken this sensitivity parameter is only relevant when the watch service falls back to polling mode, which should not be the case when barbarywatchservice is in place. Moving from FSEvent handler to polling would put more load on the CPU, which we'd like to avoid if possible.

Do you see this behavior always? Can you please make this simple test:

$ mkdir src

$ boot -s src watch -v show -e

then, in another terminal do

$ touch src/foo

$ touch src/foo

Does it still take a couple of seconds to see the watcher update?

[jeluard]

Unless I'm mistaken this sensitivity parameter is only relevant when the watch service falls back to polling mode, which should not be the case when barbarywatchservice is in place.

You might be right. I was referring to completely removing barbarywatchservice as when using the sensitivity parameter performance is good enough even if it is relying on polling.
I also noticed barbarywatchservice creates one thread per watched folder (including boot temp ones) which might be an issue if you have a large number of folders? Also this library seems abandoned.

I didn't have any significant latency doing your test so there must be something else going on. I will investigate further.
Thanks!

[micha]

How does the CPU utilization when you increase the sensitivity to the maximum and fall back to polling mode compare to the barbarysoftware fsevent implementation? With boot1 polling used too much CPU, but we didn't have access to the watchservice (this was java 1.6).

[jeluard]

I've been using this in a custom build on a significant project (a dozen folder, a hundred clj/cljs/cljx files) for the last couple months and CPU usage has never been an issue. I always have sub-second cljs recompilation for instance.
I didn't try with boot but I don't see how it could be different.

[jeluard]

I did some more experimentation on this and finally thinks this would be worth removing barbaryservice. You can find relevant changes in my fork.

I believe it is worth because:

• changes are picked up faster on my system and overall boot feels faster
• it removes all those useless threads
• no more dependency on a frozen project
• ClojureScript rely on the same logic

I did some tests on a project with more than 100 source files and performance are pretty good and CPU usage quite low.

Would you consider a PR?

[micha]

I can't find anyone else on OSX who experiences the delay you're reporting, so I'm hesitant to make this change. Is it possible that there is something about your system that's causing the issue?

[jeluard]

I understand your reluctancy given the potential impact it has. Still I think this patch would makes sense even without perf improvement as current implementation can be considered a workaround.

As a first step some configuration could be introduced to easily switch to the native implementation while still defaulting to the current one.

What do you think?

[micha]

I think I like it :)

We should be able to add a flag somewhere, (maybe a system property?) and just fall through to the Java watch service if the flag is set. If we can do this without disturbing the existing logic then I think we should, but if we need to make changes to the logic then I'd be more skeptical, because what we have there currently is the product of a lot of hard-won workarounds for different weird bugs.

[jeluard]

Great! I'll let you know how that works but it will probably take me some time.

[jeluard]

I now switched to a linux machine and don't have this issue anymore. I will close this issue for now as it seems no other OSX user were experiencing this.