kqueue improvements

[nathany]

What have I learned from experimenting with kqueue directly (primarily on OS X)? #23

• fsnotify doesn't watch for Extend, Link or Revoke events (tracked in Support additional event types #519 now).

• fsnotify has a 100ms timeout on reads, but instead we may be able to just block until kqueue has events available (nil timespec). this could save a little CPU time when idle, see: Increase syscall timeouts to decrease CPU usage? howeyc/fsnotify#58

  • closing the kq (on OS X) results in the read failing with Kevent: interrupted system call, so blocking in a goroutine shouldn't delay shutdown.

• fsnotify could add multiple file descriptors at once (such as all files in a directory), which may be a little more efficient then adding each one individually.

• investigated Udata which C programs sometimes use to point to the watched filename, but it's probably not worthwhile from Go (and NetBSD defines Udata differently than the other BSDs).

Other improvements are specific to fsnotify, such as less mutexes #13 and bug fixes #14.

[nathany]

I tried a nil timeout but there is more work involved. Currently syscall.Close(w.kq) is in the same for loop that blocks. Instead it should be called directly by watcher.Close() which will cause the read to abort, after which it can clean up and return.

[nathany]

FreeBSD 10.0 behaves a bit differently:

• Closing the kq descriptor doesn't abort the current read, in fact I still get the next event.
• Then when setting up the next read, we of course get Kevent: bad file descriptor.

It's not a huge deal if we're calling Close() before shutting down the app, but if not, then the Read goroutine will sit there blocking until it gets another event to send over the channel. And if I do a proper cleanup to DELETE the events, that next event will never come.

[nathany]

When registering file descriptors to watch, this could be useful if registering multiple watches at once (such as all the files in a directory):

If an error occurs while processing an element of the changelist and there is enough room in the eventlist, then the event will be placed in the eventlist with EV_ERROR set in flags and the system error in data. Otherwise, -1 will be returned, and errno will be set to indicate the error condition.

Otherwise I don't think I need to check for EV_ERROR. Fsnotify was doing this check, but eventlist was nil, so it should never be the case.

[nathany]

When watching a directory with kqueue it reports:

• a Write when a file is created or deleted
• a Write and Link when a subdirectory is created or deleted
• nothing at all when a file is touched (updated)
• but it does report a Write if modified by an editor that does an atomic save (create and rename)

fsnotify currently adds to this:

• watches all files in a directory to report updates to specific files.
• watches for deletes on subdirectories to report the deleted subdirectory, not just a Write+Link
• when a Write happens on a directory, it scans the directory for any new files/subdirectories and presents them as create events

[nathany]

In relation to #16, I was thinking of actually having an event of "something happened in this directory, but we don't know what." Then the scanning and comparing of the files in a directory could be a separate utility function instead of built into readEvents(). It could be used for determining which directory was deleted as well.

But there is still the problem of touching files not reporting any event at all. As well as making the calling code different for kqueue.

[Platin21]

I always wondered how it deals with atomic writes as i cannot get it to fire again when such a write happens.

[arp242]

fsnotify could add multiple file descriptors at once (such as all files in a directory), which may be a little more efficient then adding each one individually.

Add() + Remove() takes 10078ns on my slow VM inside my slow laptop, and I bet most of that is in file operations rather than the syscall:

BenchmarkAddRemove-2      128599             10078 ns/op             848 B/op         13 allocs/op

Not really worth optimizing IMO.