New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fire OS file system events when a file is added/changed/removed #7

Open
duro opened this Issue Dec 13, 2013 · 9 comments

Comments

Projects
None yet
3 participants
@duro

duro commented Dec 13, 2013

I am using bindfs in combination with a NFS mount through Vagrant. The original source folder is on local OS, and bindfs is running in the VM, mirroring the NFS mounted folder to another location.

Furthermore, I have processes running on the VM that are watching the bindfs endpoint directory for changes and running build tasks when a file changes. However, these are not triggering when I make a change on my local host OS and the file is mirrored. Host** -> NFS -> bindfs*

directory being watched
*
directory where change occurs

Is there a way to setup bindfs to fire off filesystem events?

Is this something bindfs could be extended to do?

@mpartel

This comment has been minimized.

Show comment
Hide comment
@mpartel

mpartel Dec 13, 2013

Owner

I'm afraid this cannot be done easily. I'm not sure how reliably you can get notifications over NFS, but even if that part is OK, FUSE doesn't seem to have an API for sending inotify events. We'd have to extend the FUSE kernel module first.

If possible, try to configure your build watcher to use polling instead of inotify.

Owner

mpartel commented Dec 13, 2013

I'm afraid this cannot be done easily. I'm not sure how reliably you can get notifications over NFS, but even if that part is OK, FUSE doesn't seem to have an API for sending inotify events. We'd have to extend the FUSE kernel module first.

If possible, try to configure your build watcher to use polling instead of inotify.

@mpartel

This comment has been minimized.

Show comment
Hide comment
@mpartel

mpartel Dec 13, 2013

Owner

Actually there may be a horrible hack that makes this particular use case work. We could add an option that emulates some inotify events as it sees them in the source directory. For instance, if a file is changed in the source dir, we open the corresponding file in the bindfs mount for writing and immediately close it to generate a similar enough event. This might suffice to trigger rebuilds when needed.

I might try this in a few weeks, depending on how much time I have. Which file change watcher are you using?

Owner

mpartel commented Dec 13, 2013

Actually there may be a horrible hack that makes this particular use case work. We could add an option that emulates some inotify events as it sees them in the source directory. For instance, if a file is changed in the source dir, we open the corresponding file in the bindfs mount for writing and immediately close it to generate a similar enough event. This might suffice to trigger rebuilds when needed.

I might try this in a few weeks, depending on how much time I have. Which file change watcher are you using?

@duro

This comment has been minimized.

Show comment
Hide comment
@duro

duro Dec 13, 2013

I am using GruntJS for file watching.

On Thursday, December 12, 2013, Martin Pärtel wrote:

Actually there may be a horrible hack that makes this particular use case
work. We could add an option that emulates some inotify events as it sees
them in the source directory. For instance, if a file is changed in the
source dir, we open the corresponding file in the bindfs mount for writing
and immediately close it to generate a similar enough event. This might
suffice to trigger rebuilds when needed.

I might try this in a few weeks, depending on how much time I have. Which
file change watcher are you using?


Reply to this email directly or view it on GitHubhttps://github.com/mpartel/bindfs/issues/7#issuecomment-30491095
.

duro commented Dec 13, 2013

I am using GruntJS for file watching.

On Thursday, December 12, 2013, Martin Pärtel wrote:

Actually there may be a horrible hack that makes this particular use case
work. We could add an option that emulates some inotify events as it sees
them in the source directory. For instance, if a file is changed in the
source dir, we open the corresponding file in the bindfs mount for writing
and immediately close it to generate a similar enough event. This might
suffice to trigger rebuilds when needed.

I might try this in a few weeks, depending on how much time I have. Which
file change watcher are you using?


Reply to this email directly or view it on GitHubhttps://github.com/mpartel/bindfs/issues/7#issuecomment-30491095
.

@duro

This comment has been minimized.

Show comment
Hide comment
@duro

duro Dec 13, 2013

BTW, if you got this working, you would be making a lot of Vagrant users
very happy!!

On Thursday, December 12, 2013, Adam Duro wrote:

I am using GruntJS for file watching.

On Thursday, December 12, 2013, Martin Pärtel wrote:

Actually there may be a horrible hack that makes this particular use case
work. We could add an option that emulates some inotify events as it sees
them in the source directory. For instance, if a file is changed in the
source dir, we open the corresponding file in the bindfs mount for writing
and immediately close it to generate a similar enough event. This might
suffice to trigger rebuilds when needed.

I might try this in a few weeks, depending on how much time I have. Which
file change watcher are you using?


Reply to this email directly or view it on GitHubhttps://github.com/mpartel/bindfs/issues/7#issuecomment-30491095
.

duro commented Dec 13, 2013

BTW, if you got this working, you would be making a lot of Vagrant users
very happy!!

On Thursday, December 12, 2013, Adam Duro wrote:

I am using GruntJS for file watching.

On Thursday, December 12, 2013, Martin Pärtel wrote:

Actually there may be a horrible hack that makes this particular use case
work. We could add an option that emulates some inotify events as it sees
them in the source directory. For instance, if a file is changed in the
source dir, we open the corresponding file in the bindfs mount for writing
and immediately close it to generate a similar enough event. This might
suffice to trigger rebuilds when needed.

I might try this in a few weeks, depending on how much time I have. Which
file change watcher are you using?


Reply to this email directly or view it on GitHubhttps://github.com/mpartel/bindfs/issues/7#issuecomment-30491095
.

@duro

This comment has been minimized.

Show comment
Hide comment
@duro

duro Dec 14, 2013

@mpartel Keep going on this approach. It is a bit hacky, but I was able to get a similar thing working using a Grunt task on my host OS that actually SSH's into the Vagrant VM, and runs a 'touch' on a file that would trigger the watcher running in the Host VM.

It actually worked just fine. If something similar was implemented in bindfs, then I could have all my watchers running in the VM.

Basically, change occurs in source, it is NFS sync'd to a folder on the Vagrant VM, which is then mirrored to the bindfs destination, and bindfs triggers a 'touch' on that file (or however it actually works, that's just my layman's description)

duro commented Dec 14, 2013

@mpartel Keep going on this approach. It is a bit hacky, but I was able to get a similar thing working using a Grunt task on my host OS that actually SSH's into the Vagrant VM, and runs a 'touch' on a file that would trigger the watcher running in the Host VM.

It actually worked just fine. If something similar was implemented in bindfs, then I could have all my watchers running in the VM.

Basically, change occurs in source, it is NFS sync'd to a folder on the Vagrant VM, which is then mirrored to the bindfs destination, and bindfs triggers a 'touch' on that file (or however it actually works, that's just my layman's description)

@mpartel

This comment has been minimized.

Show comment
Hide comment
@mpartel

mpartel Dec 14, 2013

Owner

Nice! :)

Owner

mpartel commented Dec 14, 2013

Nice! :)

@duro

This comment has been minimized.

Show comment
Hide comment
@duro

duro Jul 29, 2014

@mpartel Have you made any progress on this? I am running into some instances where implementing this via my task runner and an outside SSH command is VERY difficult. If bindfs did a built in "touch" when it mirrored the file, it would solve my problem.

duro commented Jul 29, 2014

@mpartel Have you made any progress on this? I am running into some instances where implementing this via my task runner and an outside SSH command is VERY difficult. If bindfs did a built in "touch" when it mirrored the file, it would solve my problem.

@mpartel

This comment has been minimized.

Show comment
Hide comment
@mpartel

mpartel Jul 29, 2014

Owner

Sadly no progress. I did spend some time on it but I found no way to avoid the recursion that happens when bindfs opens a file inside itself.

The sequence of events:

  1. An inotify event comes in.
  2. bindfs opens the same file inside its mountpoint.
  3. This triggers an open of the same file in the source dir.
  4. This in turn triggers a new inotify event.

There seems to be no way to distinguish the inotify event of case (4) from case (1). One could do horrible hacks that have a chance of losing some inotify events but I'd prefer not to do that.

The "correct" way, as far as I can tell, would involve extending FUSE to support emitting inotify events. This might be a fun adventure into kernel land, if only I had the time to spare :(

Owner

mpartel commented Jul 29, 2014

Sadly no progress. I did spend some time on it but I found no way to avoid the recursion that happens when bindfs opens a file inside itself.

The sequence of events:

  1. An inotify event comes in.
  2. bindfs opens the same file inside its mountpoint.
  3. This triggers an open of the same file in the source dir.
  4. This in turn triggers a new inotify event.

There seems to be no way to distinguish the inotify event of case (4) from case (1). One could do horrible hacks that have a chance of losing some inotify events but I'd prefer not to do that.

The "correct" way, as far as I can tell, would involve extending FUSE to support emitting inotify events. This might be a fun adventure into kernel land, if only I had the time to spare :(

@kolorafa

This comment has been minimized.

Show comment
Hide comment
@kolorafa

kolorafa Jan 19, 2018

There is a light in the tunnel - fsnotify.

https://github.com/libfuse/libfuse/wiki/Fsnotify-and-FUSE
http://fuse-devel.narkive.com/jLikFBeF/fsnotify-and-fuse

Maybe it's already in the kernel and you could build change notifications on it?

kolorafa commented Jan 19, 2018

There is a light in the tunnel - fsnotify.

https://github.com/libfuse/libfuse/wiki/Fsnotify-and-FUSE
http://fuse-devel.narkive.com/jLikFBeF/fsnotify-and-fuse

Maybe it's already in the kernel and you could build change notifications on it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment