OSXFUSE doesn't seem to be honoring fuse_lowlevel_notify_delete or fuse_lowlevel_notify_inval_entry.

[elderK]

Hey there,

I've been working on a FUSE filesystem for quite some time and have been hoping to release it for macOS as well as Linux.

Unfortunately for me, I've encountered an issue on macOS: No matter what I tell the system, it continues to believe that something is present when it isn't.

The filesystem I am developing is an interface to a cloud filesystem: Entities can be renamed or removed via the cloud. When this occurs, my filesystem notifies FUSE via the fuse_lowlevel_notify_* functions.

No matter what I do, the system keeps holding on to stale information.

I can confirm this by using the "vfs.generic.macfuse.control.print_vnodes" sysctl knob.

I'm not really sure how to proceed to resolve this issue: I've been poking about, studying the last open source release. I've tried specifying the novncache option (which seems to have no effect on my system.)

Are there any restrictions to how the fuse_lowlevel_notify_* functions can be used in macOS?

I'm using macFUSE 4.5.0 on macOS Ventura 13.4.

I look forward to hearing back from anyone who might be able to help.

Kind regards,
Dave Ward

EDIT: I should also note that I'm running on an M1 Mac Mini.

[bfleischer]

Are there any restrictions to how the fuse_lowlevel_notify_* functions can be used in macOS?

None that I would be aware of. They are implemented similar to their Linux counterparts.

Can you provide a minimal sample project showcasing the issue?

[elderK]

Hey @bfleischer,

Thank you for responding.

I'll implement a small test program that will highlight the issue.

I'll share it here, via GitHub, when it's complete. :)

~Dave

[elderK]

Hey @bfleischer,

You can find a little program that can reproduce the issue here:
https://hg.sr.ht/~kyretzn/fuse-bug

I originally hit the issue while running my filesystem's integration tests: There, I found that "realpath" was returning stale information. Say, we rename sd0 to sfx and then call realpath on sfx: sd0 would be returned. Interestingly, attempts to stat sd0 fail and if you ls the directory, sd0 would no longer be present.

I've found that calling fuse_lowlevel_notify_delete(...) will purge the stale entry and realpath will return what we expect. Interestingly, it is functioning today but yesterday, with no source changes and only a reboot between, notify_delete now functions as you'd expect.

Calling fuse_lowlevel_inval_entry does not seem to purge the entry: realpath will return incorrect information, even now. Specifying the -onovncache option seem to have no effect.

Unfortunately for me, using notify_delete isn't necessarily a solution for my use case either: I've found that if you open a file (or a directory) for reading, remove (or move) that file, and then attempt to call fstat on the descriptor you had, you will fail with either ENOENT or ENXIO. Which failure you get depends on whether you use notify_delete or inval_entry.

This behavior differs from libfuse on Linux.

I'm not sure how to proceed so I'll simply ask a question: When implementing a FUSE filesystem, what steps should we take such that the use-case above functions as you'd expect? That is, if you have a descriptor open for some file, and that file is removed (from the cloud side), you should still be able to read (and write) that file. The file should not be visible anymore (you cannot stat(path) it, nor should you be able to see it in listings.)

I look forward to hearing back from you.

Let me know if there's anything I can do to help or anything you'd like me to try.

Kind regards,
Dave Ward

[elderK]

Hey there @bfleischer ,

How's the investigation going? Any luck reproducing the issue?

Kind regards,
Dave Ward