xdg-document-portal keeps files open indefinitely, preventing unmounting of removable media #689
Comments
|
Any progress on this? |
|
No activity means no progress |
|
This also means that when files are deleted the space they occupied isn't reclaimed. I was playing some large video files with Flatpak Celluloid and my drive filled up. |
Yeah this is how I discovered the bug. |
|
Ran into this while simply mounting a USB drive, opening an MP4 file on it in Gnome Videos via Nautilus and then not being able to unmount the USB drive again. |
|
@Fraetor @rowbawts Is the space being reclaimed once @GeorgesStavracas I had the same today with It kept these files open even after I've deleted them from trash using Gnome Files.
|
|
The correct way to do it is not do it. |
|
Thanks for replying. I understand that it's a bug and should not happen in the first place. My question was whether it was a correct workaround on my part (the bug already hit me after all) to do it that way in order to avoid filesystem inconsistency or not. I understand you may not know the answer to that question, so do you maybe know where or who to ask about it? |
|
Hi @alexlarsson, can you provide some insight please? From the git history of xdg-document-portal, you seem to have a deeper understanding of that code than most. This bug is quite egregious as #689 (comment) points out. Currently no one working on this project seems to know how to remedy the situation. |
|
Workaround for Ubuntu 23.04: Log out then log in. You should be able to unmount the removable media. |
|
Doesn't that equate to just terminating the service? |
|
Is this only a problem with vfat external drives or does it happen with other filesystems as well? A potentially better work around, anyhow, is |
I have no idea. I have not tried this. I closed all applications but files on my NFS share were still being accessed by xdg-document-portal.
I guess restaring it is also a good idea after closing all applications. [UPDATE] |
This comment was marked as off-topic.
This comment was marked as off-topic.
btrfs and ext4 external drives over here, same issue. |
|
Mine was an NFS mount. |
|
Guys/gals, no point in spamming the issue with the filesystem you use. The issue is not unique to a specific filesystem and is not just external drives that have this issue - it happens on all filesystems and all types of drives, you are just more likely to notice it using an external drive as you might want to unmount it, which would surface the issue. The actual issue seems to be xdg-document-portal not closing the files when an application using xdg-document-portal closes them or when the application exits. |
|
I can confirm there's a file descriptor leak somewhere, I'm just not sure on how to debug it. If I knew the places and functions necessary, I could go out and printf-debug them. |
|
@orowith2os I tried pinpointing the issue a while back. From what I remember the issue was in one of these files (I believe) (1, 2) My takeaway was that there is no straightforward solution, which is why it is being ignored. Either the portal system needs a fundamental rewrite or API changes are needed. Obviously feel free to take a look, the more eyes on the problem the better. This has to be addressed sooner or later. |
|
Perhaps @alexlarsson could shed some light on this issue, they seem to be familiar with the code, going by Also, this comment caught my eye, sounds like it might be describing this issue: xdg-desktop-portal/document-portal/document-portal-fuse.c Lines 85 to 95 in 04bfd72 |
hfiguiere
added
portal: documents
|
The documents store appears to not be able to run under valgrind, so trying to find the leak will take longer, unless someone is able to help resolve that. |
|
I don't think valgrind will show you this. In document-portal-fuse.c The The fix is to figure out when we can unref the corresponding Note: I am working on this. |
Issue flatpak#689 flatpak#689 Signed-off-by: Hubert Figuière <hub@figuiere.net>
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes #689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes flatpak#689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
This fixes #689 The mechanism we're using to flush the dcache and thus avoiding long-term caching of physical inodes which keep a file descriptor open doesn't seem to work quite right. A trivial test of cat /run/user/1000/doc/$docid/$filename properly leads to the PhysicalFile being freed. However, an application that keeps the file open for a longer time (longer than the delay we have before the current dcache flush workaround is run), ends up with the dcache entry for the inode to be kept in cache. This means that we when the last other kernel reference (like an open fd) to the inode is gone we don't get a FORGET event from fuse, and thus we never free the PhysicalInode. This is clearly visible, because if you end up with the file descriptor leak like in the issue, then flushing the dcache (`sudo sh -c 'echo 2 > /proc/sys/vm/drop_caches'`) fixes the leak. It seems the current workaround of just invalidating the toplevel directory entry doesn't work, so we replace it with tracking each individual lookup that ends up with a physical inode and flush all of them. This seems to fix the issue for me.
|
I've just hit this in Ubuntu 23.10. Hopefully the fix lands into Ubuntu at some point xdg-desktop-portal is already the newest version (1.18.0-1ubuntu1). |
|
As a workaround you can restart the portals using |
I'm trying out Chromium flatpak. I downloaded a few files using Chromium to a removable drive and then attempted to unmount it, but the system kept telling me that it can't do that, that the device is busy. I
lsofthe mount point to see what is using the fs and lo-and-behold:The files were downloaded hours ago, deleted hours ago too (notice the "deleted" after each file), and Chromium was closed long time ago, but xdg-document-portal still has them open and is preventing me from unmounting the drive, forcing me to kill it. What gives? I don't want having to fish its PID and kill it every time I want to unmount my drive. That's 0/10 usability.
The expected behavior is that when I download something via Chromium on my removable drive, once it's done downloading, I can unmount the drive without xdg-document-portal getting in my way, with Chromium still running, just like it would be the case with a non-Flatpak Chromium.
Running Debian Bullseye with KDE Plasma.
The text was updated successfully, but these errors were encountered: