-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
INotify orphaned file descriptor on file/directory deletion #9777
Comments
Link to PR: #1231 |
In changeset 07910d7
|
There is some question over whether this change was sensible or not - 073da70#commitcomment-133586418 |
I don't know why this change was made. If you want the INotify to go away, the application should call |
Yep, my bad, that was a bad review. If you can call |
Hi, looks like indeed this introduces an issue :( I looked into this a bit. I found an old script I had written when I made this PR to test various scenarios, but clearly not the one raised in the comment. I think one thing one could try if running the test script is:
With this PR, we don't have a leftover file descriptor linked to inotify, but without this PR (i.e. with that line commented), we do. For some background, I remember having an issue where I had tons of unclosed file descriptors after a long runtime of a twisted process. It could be that the right thing to do was to reap the file descriptors in some other way (the application calling I'll try to put in some more thought into it later in the week and try to remember the real context (it was a long time ago), but in the meantime, a second set of eyes on whether or not the use-case in this ticket is also an issue could be helpful, so please feel free to mess with the script if that's helpful. That being said, this is definitely introducing a regression in wesleys common use case, so feel free to revert this clearly buggy attempt at a fix o.O. You're right, it doesn't make too much conceptual sense either. Note: I quick n' dirty edited the script (separate gist) to watch two directories to help with validating wesleys case, and I indeed reproduced their issue. |
Also, it looks like the unit test introduced in this PR is of limited use, we actually want to test that we don't have zombie file descriptors, not that loseConnection() specifically is called (so more of an integration than a unit test perhaps -- not sure if you know a good way to do that without real IO) And of course we want to add a test for the case raised by wesley :) |
…letion INotify will transparently unwatch a directory if it has been deleted from the filesystem. Not making a call to .loseConnection() results in leaving permanently opened [orphaned] file descriptors. We now close that file descriptor since it can no longer trigger.
When a directory is watched via twisted's inotify wrappers and is deleted, it's automatically and transparently removed from the watchlist by forcing the catching of
inotify.IN_DELETE_SELF
.However, when this occurs, the
INotify
file descriptor is never closed and will therefore be forever hanging until the twisted process is closed. This could be checked vials -l /proc/<pid>/fd/
.A workaround to close the file descriptor is to have a callback (passed via watch() that will explicitly catch the mask
inotify.IN_DELETE_SELF
(even if the user hasn't specified this mask when initiating the watch) and close the file descriptor there (via a call to.loseConnection()
).Searchable metadata
The text was updated successfully, but these errors were encountered: