FSEvents on macOS

[nathany]

Requested by @robfig at howeyc/fsnotify#54 in order to watch large trees.

To be implemented as a stand-alone repository for experimentation (#23) before determining how to incorporate it into fsnotify.

System requirements:

• OS X 10.6 or better (matching the system requirements of Go 1.3). Though some features require 10.7, or perhaps even higher.
• Likely require cgo with Xcode installed.

Prior work:

• @samjacobson https://github.com/samjacobson/go.fsevents
• @ronbu https://github.com/ronbu/fsevents

Reference:

• Advanced Mac OS X by Mark Dalrymple (@markd2) contains a chapter on FSEvents
• FSEvents Reference
• ArsTechnica

[nathany]

Work has begun on this as an external project that can eventually be used from fsnotify.
https://github.com/go-fsnotify/fsevents

TODO:

• Integration tests
• Refine API
• EventFlags.String()

[joeljeske]

Any idea on what the plan would be for incorporating FSEvents? Were issues found in the experimentation?

What can the community do at this point to help move this cause?

[arp242]

github.com/fsnotify/fsevents uses CGO, and I'd really like to avoid using CGO. The main reason for this is that it just makes building applications that depend on fsnotify harder: you will need a C compiler, and cross-compiling from e.g. Linux to macOS – which is easy now – will be very hard as well. We can use some shims and build tags to allow people to use CGO_ENABLED=0 and -tags fsnotify_no_events, but then FSEvents won't be available in that build which would be confusing.

So the path forward on this is that support for FSEvents will have to be added to x/sys/unix first, similar to how illumos/Solaris FEN support was added there before we added support here.

I don't have a macOS machine; just a (slow) QEMU VM. It's kind of hard for me to work on this, and I'm unlikely to work on it in the near future, but this is the path forward to get it properly added to fsnotify. In the meanwhile, people can use github.com/fsnotify/fsevents protected with some build tags if they want, but it's a bit awkward.

[srlehn]

github.com/ebitengine/purego could be used to skirt the usage of cgo.

[nathany]

The go command now disables cgo by default on systems without a C toolchain.
https://go.dev/blog/go1.20

Falling back from FSEvents to kqueue on macOS would be confusing, that's true. It may work, but it would be confusing to the programmer or end-user compiling the software, and could make bug reports confusing too.

macOS doesn't have the C compiler installed by default, though a developer may not get very far without installing the Xcode Command Line Tools. It's been a while since I've done a fresh install, but I feel like it didn't take long after installing the Homebrew package manager before I had to get a C compiler installed too.

Cross-compiling seems like a more significant issue. Zig has been innovating in that space by bundling libc and a C/C++ compiler, but that could be a difficult proposal to sell for Go.

@arp242 Are you still without a dedicated Mac?

[arp242]

Are you still without a dedicated Mac?

I don't expect that'll change any time soon, if ever, and even if it did, I wouldn't be hugely interested to pick this up either to be honest. It's a comparatively big project: we're talking about a full week, maybe even two weeks of of full-time work? Even going with the existing cgo solution would need quite a bit of attention to get it up to spec.

But spending so much of my free time on something I'll never personally see the benefits of is kind of where I draw the line. That doesn't mean I wouldn't like to see support for it, or don't want to spend any time on it, but it'll have to be like the illumos/FEN stuff, where nshalman did the bulk of the work and I just helped out where I could, or something like that.