Add fswatch package, ported from @parcel/watcher with heavy modification (#3980)

Author: jakebailey

.github/workflows/ci.yml

@@ -197,6 +197,48 @@ jobs:
           name: ${{ matrix.config.os }}-${{ (matrix.config.race && 'race') || 'norace' }}-new-baselines-artifact
           path: testdata/baselines/local
 
+  fswatch-cross:
+    name: fswatch (${{ matrix.os.name }} ${{ matrix.os.version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        os:
+          - name: freebsd
+            version: '15.0'
+            install: sudo pkg install -y go
+          - name: openbsd
+            version: '7.9'
+            install: sudo pkg_add -I go
+          - name: netbsd
+            version: '10.1'
+            install: sudo pkgin -y install go && sudo ln -sf "$(ls /usr/pkg/bin/go1* | sort | tail -1)" /usr/pkg/bin/go
+          - name: dragonflybsd
+            version: '6.4.2'
+            install: echo 'nameserver 8.8.8.8' | sudo tee /etc/resolv.conf && sudo pkg install -y go
+    steps:
+      - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+        with:
+          submodules: false
+          sparse-checkout: |
+            go.mod
+            go.sum
+            internal/fswatch
+
+      - name: Start VM on ${{ matrix.os.name }}
+        uses: cross-platform-actions/action@be3d7e9ff5c8770b9c51b1a8c8c5446e1cad7cf9 # v1.2.0
+        with:
+          operating_system: ${{ matrix.os.name }}
+          version: ${{ matrix.os.version }}
+          shell: bash
+
+      - name: Test fswatch on ${{ matrix.os.name }}
+        shell: cpa.sh {0}
+        run: |
+          ${{ matrix.os.install }}
+          go version
+          go test -v ./internal/fswatch/...
+
   lint:
     strategy:
       fail-fast: ${{ github.event_name == 'merge_group' }}
@@ -369,6 +411,7 @@ jobs:
       - smoke
       - test
       - tidy
+      - fswatch-cross
 
     steps:
       - name: Check required jobs

.golangci.yml

@@ -96,7 +96,7 @@ linters:
       - path: internal/fourslash/tests/gen/
         linters:
           - misspell
-      - path: 'internal/(repo|testutil|testrunner|vfs|pprof|execute/tsctests|bundled)|cmd/tsgo'
+      - path: 'internal/(repo|testutil|testrunner|vfs|pprof|execute/tsctests|bundled|fswatch)|cmd/tsgo'
         text: should likely be used instead
       - path: '(.+)_test\.go$'
         text: should likely be used instead

internal/fswatch/CHANGES.md

@@ -0,0 +1,243 @@
+# Changes from upstream `@parcel/watcher`
+
+This Go port started from the C++
+[`@parcel/watcher`](https://github.com/parcel-bundler/watcher) (v2.5.6,
+`8926bb8`) and has diverged significantly. This document covers API differences,
+simplifications, new features, and bugfixes.
+
+## API differences
+
+### Method naming
+
+| C++ / JS               | Go                                 |
+| ---------------------- | ---------------------------------- |
+| `subscribe(dir, fn)`   | `WatchDirectory(dir, fn, opts...)` |
+| —                      | `WatchFile(path, fn)`              |
+| `unsubscribe(dir, fn)` | `w.Close()`                        |
+
+### Recursion default
+
+C++ `subscribe` is always recursive. Go's `WatchDirectory` is **non-recursive by
+default**, watching only direct children. Pass `WithRecursive()` to watch the
+entire tree. This matches TypeScript's `watchDirectory(path, cb, recursive?)`
+where recursive is opt-in.
+
+### Event kinds
+
+C++ has three event kinds: create, update, delete. Go has two: **`EventUpdate`**
+and **`EventDelete`**. File creation is reported as `EventUpdate`. `tsc --watch`
+doesn't distinguish between a file being created and a file being modified; both
+mean "something changed, rebuild." This also sidesteps a C++ FSEvents bug where
+pre-existing files are misclassified as "created" because the internal tree
+starts empty at subscribe time.
+
+### Watch options
+
+Go adds functional options not present in the C++ API:
+
+- **`WithRecursive()`**: opt in to recursive directory tree watching.
+- **`WithIgnore(func(path string) bool)`**: filter events per-subscriber before
+  delivery. Return true to drop.
+
+### File watching
+
+`WatchFile(path, fn)` watches a single file by watching its parent directory
+non-recursively and filtering events to the target path. Multiple file watches
+in the same directory share one OS watch. Not available in the C++ API.
+
+### Error delivery
+
+C++ delivers errors via a separate error callback or return value. Go delivers
+errors through the same `WatchCallback(events, err)` with sentinel errors:
+
+- `ErrOverflow`: recoverable, the watch stays active.
+- `ErrWatchTerminated`: terminal, call `Close()` to clean up.
+
+`ErrUnavailable` is returned directly from `WatchDirectory`/`WatchFile` (not
+through the callback) when the watcher is not supported on the current platform.
+
+## Simplifications
+
+### No in-memory directory tree
+
+C++ maintains an in-memory `DirTree` for every subscription on every backend,
+storing path, type, and mtime for every watched file. The tree serves two
+purposes: mtime-based event dedup (suppressing events when the mtime hasn't
+changed) and create-vs-update classification (if a path is in the tree it's an
+update, otherwise it's a create).
+
+Go removes the tree entirely on inotify, fanotify, Windows, and FSEvents. With
+mtime tracking removed and only two event kinds (update and delete), the tree
+became write-only on those backends: populated during setup and event handling
+but never read from. Event classification relies on kernel flags instead of stat
+calls, eliminating O(events) syscalls from the hot path. kqueue needs a
+path-to-fd mapping (kqueue identifies events by fd, not path), but uses a flat
+map holding only path and isDir.
+
+C++ also maintains a separate lazily-populated `DirTree` for FSEvents, used for
+create/update classification. Because the tree starts empty at subscribe time,
+pre-existing files aren't in it, and the first modification of any pre-existing
+file is misclassified as "create" instead of "update." Go's FSEvents backend
+classifies events using only the kernel-provided flags. Pure
+create/remove/modify cases need zero syscalls; only the ambiguous-flags case
+(multiple flags set) does one `Lstat` to check existence.
+
+### No attribute events
+
+C++ watches `IN_ATTRIB` (inotify), `FAN_ATTRIB` (fanotify), and
+`FILE_NOTIFY_CHANGE_ATTRIBUTES` (Windows). Go removes all three from the watch
+masks. `chmod`, `chown`, and other metadata-only changes don't trigger events.
+kqueue still receives `NOTE_ATTRIB` (needed for truncate on some BSDs), but the
+events are delivered as `EventUpdate` without special handling.
+
+### Simpler event coalescing
+
+With only two event kinds (update, delete), the `eventList` coalescing logic is
+simpler:
+
+- `create + delete` within one batch cancels out (the entry is skipped).
+- `delete + create` becomes update (the rapid delete+recreate pattern).
+- `update + delete` yields delete.
+- `delete + update` yields delete (a bare `update` does not resurrect a deleted
+  entry; only an explicit `create` does).
+
+### Per-backend debouncer
+
+Upstream uses one process-wide `Debounce::getShared()` singleton that batches
+events for every `Watcher` in the process. This is a fine choice for
+parcel-watcher's setting: Node consumers serialize through the libuv event loop
+anyway, so spawning multiple debounce threads wouldn't buy any downstream
+parallelism.
+
+Go can handle concurrent work cheaply, so the Go port creates one debouncer per
+backend (inotify, fanotify, kqueue, fsevents, windows) instead of one per
+process. Each backend's debouncer is created lazily on first subscribe and
+serves only that backend's `dirWatch`es, so a slow user callback on one backend
+can't starve event delivery on any of the others. In practice most callers will
+only ever use one backend (`Default()`), so this mainly matters for processes
+that mix backends, but the cost of the split is essentially nothing.
+
+## New backends
+
+**fanotify** (Linux, kernel ≥ 5.13) is the default on Linux when available. It
+uses FID-based event reporting, avoiding the inotify per-user watch limit
+entirely. Written from scratch rather than ported from the upstream
+[PR #180](https://github.com/parcel-bundler/watcher/pull/180), which has several
+bugs (see below). The backend runtime-probes `FAN_RENAME` (Linux 5.17+) and
+falls back to `FAN_MOVED_FROM`/`FAN_MOVED_TO`.
+
+## Pure Go, no cgo
+
+The C++ library requires a C++ compiler and platform-specific build
+configuration. The Go port is pure Go on all platforms:
+
+- **macOS FSEvents**: CoreFoundation/CoreServices calls via
+  `//go:cgo_import_dynamic` and hand-written assembly trampolines (amd64 and
+  arm64), following the pattern from Go's `crypto/x509/internal/macos`. The
+  FSEvents C callback runs on a libdispatch (GCD) thread, not a Go goroutine. An
+  assembly shim, staying entirely in C calling convention, retains the CFArray
+  of paths, allocates a per-callback payload on the C heap, copies the flags
+  array into it, and writes the payload pointer to the stream's event pipe,
+  waking a dedicated Go event-loop goroutine that classifies the events and
+  frees the payload. The shim then returns immediately, so the dispatch thread
+  never enters Go ABI and does not wait for Go-side event classification. Each
+  FSEventStream has its own serial GCD dispatch queue and event pipe, so
+  callbacks for different streams run concurrently without contention: a stuck
+  callback for one stream cannot back up callbacks for any other stream behind
+  it. Teardown invalidates the stream and uses a `dispatch_sync_f` barrier on
+  the stream's serial queue before closing the pipe, releasing the queue, and
+  unpinning the callback state.
+- **Windows**: direct `x/sys/windows` syscalls.
+- **Linux/BSD**: direct `x/sys/unix` syscalls.
+
+Cross-compilation works without cgo:
+`CGO_ENABLED=0 GOOS=darwin GOARCH=arm64 go build ./...`
+
+## Bugfixes from upstream C++
+
+### 1. Windows: dropped create event when GetFileAttributesEx fails
+
+`ReadDirectoryChangesW` reports `FILE_ACTION_ADDED` for files that may vanish
+before processing. C++ guards the event inside the attribute lookup success
+check, silently dropping it. Go always emits the event.
+
+### 2. Windows: race between subscribe and ReadDirectoryChangesW
+
+C++ queues an APC that eventually arms the watch. A filesystem operation between
+`subscribe()` returning and the APC firing is missed. Go arms the first
+`ReadDirectoryChangesW` synchronously before returning.
+
+### 3. kqueue: TOCTOU race and early-return in compareDir
+
+C++ emits a create event before confirming the file can be opened. If it
+vanishes, a phantom create is queued. Additionally, `watchDir` failure returns
+from the entire `compareDir`, skipping delete detection for other files.
+
+### 4. Event coalescing: create+delete+create yields wrong result
+
+C++ clears `isDeleted` without clearing `isCreated`, so a create+delete+create
+sequence produces a spurious "create" instead of the intended "update."
+
+### 5. Event drain race: getEvents + clear are separate locks
+
+C++ calls `getEvents()` then `clear()`, each independently locking. Events
+inserted between the two calls are silently lost. Go uses an atomic `drain()`
+that snapshots and clears under a single lock.
+
+### 6. inotify: IN_Q_OVERFLOW silently skipped
+
+C++ skips overflow events without notifying subscribers. Go delivers
+`ErrOverflow` to all active watches.
+
+### 7. inotify: descendant watches not cleaned on directory deletion
+
+C++ only removes exact-match watches when a directory is deleted. Watches for
+descendant paths remain and may receive stale events if watch descriptors are
+reused.
+
+### 8. kqueue: mtime guard suppresses NOTE_WRITE on coarse-mtime filesystems
+
+C++ guards all `NOTE_WRITE | NOTE_ATTRIB | NOTE_EXTEND` events behind an mtime
+check. On OpenBSD FFS (1-second mtime granularity), rapid writes share the same
+mtime and are suppressed.
+
+### 9. Windows: readTree follows symlinked directories
+
+C++ checks `FILE_ATTRIBUTE_DIRECTORY` without excluding
+`FILE_ATTRIBUTE_REPARSE_POINT`, causing symlinks and junctions to be traversed.
+
+### 10. kqueue: delete/create coalescing race and fd leak
+
+When a file is deleted and recreated, kqueue may deliver `NOTE_WRITE` on the
+parent before `NOTE_DELETE` on the file. C++ processes these in order, missing
+the create. Separately, deleted fds are erased from the map but never closed.
+
+### 11. kqueue: tryRewatchLocked race for directories
+
+On OpenBSD, `RemoveAll(dir)` can deliver `NOTE_DELETE` for a directory while
+`rmdir` is still in progress. `tryRewatchLocked` sees the directory still exists
+via `Lstat` and emits a spurious "update" instead of "delete." Go skips
+`tryRewatchLocked` for directories entirely.
+
+### 12. FSEvents: empty tree misclassifies updates as creates
+
+C++ maintains a lazily-populated `DirTree` for FSEvents. Pre-existing files
+aren't in the tree at subscribe time, so the first modification is classified as
+"create" instead of "update."
+
+## Bugfixes from upstream fanotify PR
+
+The upstream [PR #180](https://github.com/parcel-bundler/watcher/pull/180) adds
+a fanotify backend to the C++ library. Go's fanotify backend was written from
+scratch and avoids the following issues in the C++ PR:
+
+- **FAN_Q_OVERFLOW silently skipped.** C++ skips the event; Go delivers
+  `ErrOverflow`.
+- **Descendant watches not cleaned.** Same exact-match-only bug as inotify.
+- **Unchecked lstat/stat return values.** C++ feeds uninitialized stat data to
+  `tree->add()` on rapid create+delete. Go guards all stat calls.
+- **No merged-event disambiguation.** C++ processes `FAN_CREATE` before
+  `FAN_DELETE` in an if/else chain, so a merged create+delete always emits a
+  spurious create. Go stats the path to determine temporal order.
+- **No runtime FAN_RENAME probing.** C++ uses compile-time `#ifdef`; Go probes
+  at runtime and falls back gracefully.

internal/fswatch/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) Microsoft Corporation.
+Copyright (c) 2017-present Devon Govett
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.