Watching temp directory on OSX non-recursive does not work

[NiklasRosenstein]

To test a class that uses watchdog in a unit test, I create files using the tempfile module, and then uses it to listen to those changes to that temporary file. However, I encounter that using recursive=False does not work in this case (it does not trigger any events). I presume that this may have something to do with the temporary folder because it works fine in another folder.

Encountered this in watchdog 2.1.3, but it works with watchdog 0.10.2.

Steps to reproduce

#! /usr/bin/python

import atexit
import os
import logging
import tempfile
from watchdog.observers import Observer
from watchdog.events import LoggingEventHandler

if __name__ == "__main__":
  logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(message)s', datefmt='%Y-%m-%d %H:%M:%S')
  event_handler = LoggingEventHandler()
  with tempfile.NamedTemporaryFile(delete=False) as fp:
    fp.close()
    atexit.register(lambda: os.unlink(fp.name))
    watch_dir = os.path.dirname(fp.name)
    print(fp.name)
    print(watch_dir)

    observer = Observer()
    observer.schedule(event_handler, watch_dir, recursive=False)
    observer.start()
    try: observer.join()
    except KeyboardInterrupt: observer.stop()
    observer.join()

Running this script on watchdog 0.10.2:

$ pip show watchdog | grep Version
Version: 0.10.2
$ python test.py 
/var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T/tmpo_tjs89b
/var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T
# >>> Run touch /var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T/tmpo_tjs89b in another window
2021-07-21 22:50:37 - Modified file: /var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T/tmpo_tjs89b

Now when running the same script under watchdog 2.1.3:

$ pip show watchdog | grep Version
Version: 2.1.3
$ python test.py 
/var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T/tmpxw1nk7y3
/var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T
# >>> Run touch touch /var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T/tmpxw1nk7y3 in another window

(Note how there is no even triggered)

If you use recursive=True in watchdog 2.1.3, it will work as well.

After trying a few different versions, it appears to stop working with 2.1.0 and still works in 2.0.3.

[NiklasRosenstein]

Enabling debug logs, we can actually see that the event is received, but dropped.

2021-07-21 22:58:47 - drop event <FileModifiedEvent: event_type=modified, src_path='/private/var/folders/db/zzjx9cwj25q0mb7c9rmzk5t50852rj/T/tmpvuzz3i76', is_directory=False>

Most likely that is due to this piece of code introduced in 2.1.0: https://github.com/gorakhargosh/watchdog/compare/v2.0.3..v2.1.0

watchdog/src/watchdog/observers/fsevents.py

Lines 94 to 105 in d691038

	def queue_event(self, event):
	# fsevents defaults to be recursive, so if the watch was meant to be non-recursive then we need to drop
	# all the events here which do not have a src_path / dest_path that matches the watched path
	if self._watch.is_recursive:
	logger.debug("queue_event %s", event)
	EventEmitter.queue_event(self, event)
	else:
	if not self._is_recursive_event(event):
	logger.debug("queue_event %s", event)
	EventEmitter.queue_event(self, event)
	else:
	logger.debug("drop event %s", event)

[NiklasRosenstein]

/var links to private/var on my OSX machine. In _is_recursive_event(), it will compare the real path of /private/var/... of the file with the symlinked path /var/... and thus the result is False.

watchdog/src/watchdog/observers/fsevents.py

Lines 107 to 110 in d691038

	def _is_recursive_event(self, event):
	src_path = event.src_path if event.is_directory else os.path.dirname(event.src_path)
	if src_path == self._watch.path:
	return False

A potential fix is to convert the _absolute_watch_path with os.path.realpath()