Add fanotify support #114

Open
purpleidea opened this Issue Jan 20, 2016 · 13 comments

Projects

None yet

4 participants

@purpleidea

Would there be any objections if someone sent in patches to add support for fanotify?

@purpleidea purpleidea referenced this issue in purpleidea/mgmt Jan 20, 2016
Closed

ENOSPC in file.go #1

@nathany nathany added the platform label Jan 21, 2016
@nathany
Member
nathany commented Jan 21, 2016

That would be great.

The only thing is, how does someone using Linux choose between them? We may need something along the lines of #104 first.

@purpleidea

Just wanted to get the approximate ACK, so that I can point people here if they're interested in fanotify benefits. Thanks!

@purpleidea purpleidea referenced this issue in purpleidea/mgmt Feb 15, 2016
Closed

file resource needs improvements #13

@nathany
Member
nathany commented Jan 31, 2017 edited

@purpleidea I think it would be best to build fanotify Go wrapper out as a separate repo and then look at integrating after that.

@nathany
Member
nathany commented Jan 31, 2017 edited

@amir73il is working on a "super block watch" for Linux, providing "the ability to set a single (fanotify) watch on a root directory and get notified on all the legacy inotify events without the need to recursively add watches on all directories." https://lkml.org/lkml/2016/12/20/312

This could avoid the need for a user-space recursive watcher (#16) on modern Linux kernels.

@purpleidea

@nathany Thanks for the info! Looking forward to @amir73il's patches!

Cheers

@amir73il

Well the patches are out there already in my github (applied to kernel 4.9), but for those of you hoping for this functionality to get upstream, I suggest to be patient.

I have no doubt it is going to be some time before this feature can be
merged to an official kernel.
My bet is that I will have to maintain it out of tree for a while, and only
after real users show genuine interest in the feature, it will be seriously
considered for upstream.

This is were you guys can be of help.
So far I had only one guy rooting for my patches on LKML
and he has also tested them on his system.

When promoting a feature for upstream it is important to bring solid use cases that require the feature and argue that the same cannot be achieved by user library code and existing kernel functionality.

However, if you can't test my work on a distro kernel then it is going to be harder to claim that it is beneficial for your use cases.
To solve this chicken and egg problem I plan to provide install-able
kernel modules for commonly used Linux distros, so using fanotify super block
should be as easy as e.g.: apt-get install fsntotify-tools.

I cannot guaranty when I will get to providing this level of installation though, so if there are any of you out there not afraid of building a custom kernel, I will gladly assist you if you want to test my patches.

Cheers.

@nathany
Member
nathany commented Jan 31, 2017

Thanks Amir.

Perhaps another option to make the patched kernel available would be to maintain a Vagrant box built with Packer. That way we could test fanotify super block using a VirtualBox VM from any operating system.

@amir73il

Yes, that could work. And I promise to assist the person who volunteers to work on this setup.

@tiwaana
tiwaana commented Jan 31, 2017

Amir, which kernel version you would like to target ?

@purpleidea

@amir73il I have pinged some kernel engineers at my company to look into your patch. In the meantime, if you have a moment, could you look into and recommend an algorithm or suggest an improvement to the recursive file watching which I've implemented for mgmt? The code is available here:
https://github.com/purpleidea/mgmt/blob/master/recwatch/recwatch.go#L134

Cheers!

@amir73il
amir73il commented Feb 1, 2017

@tiwaana question is moot. I would like to target the earliest kernel version possible, but since this is not a bug fix nor a trivial improvement, some things have to happen first not all of them depend on me, not necessarily in that order:

  1. Technical review of patches (I am working on getting that)
  2. Design review of patched (ditto)
  3. Review of the proposed kernel-user API
  4. Demonstrate a cut and clear benefit to Linux users community
  5. Demonstrate no performance regressions for users not using the feature
@amir73il
amir73il commented Feb 1, 2017

@purpleidea thanks for the ping. If your company will show interest in the super block watch, that can be a game changer. wrt your recursive watcher, I am new to golang and have zero knowledge about fsnotify library, but it appears your code is not calling addSubFolders() recursively from Init() more than 1 level of depth, so if you never get events on the direct sub folders you will never add watchers for level 2 subdirs, but I may be missing something. Also I don't see any handling of Move events for dirs, unless it is handled in lib by generating Rename/Create event pair.

@purpleidea
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment