Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

preview sets quarantine attribute on all files it opens on an SSHFS volume. #162

Closed
glyph opened this issue Sep 19, 2014 · 31 comments
Closed

preview sets quarantine attribute on all files it opens on an SSHFS volume. #162

glyph opened this issue Sep 19, 2014 · 31 comments
Assignees
Milestone

Comments

@glyph
Copy link

@glyph glyph commented Sep 19, 2014

If I ever open the same image twice with Preview from an SSHFS-mounted volume, I get the annoying “is damaged and can’t be opened” dialog.

The images aren't actually damaged. I can read them just fine if I manually remove the attribute.

Other image viewers and editing applications don't have this problem. It appears to be specific to Preview. I'm not sure if it's sshfs-specific or all osxfuse filesystems.

I realize this report is pretty vague. I'm happy to do any additional diagnostics that would be helpful.

@mehaase
Copy link

@mehaase mehaase commented Oct 14, 2014

I have the same issue with Keynote files on encfs. If I open, save, and close, then I can't open it from Finder again. I can open it just fine if I go into Keynote and choose File → Open.

Removing the com.apple.quarantine attribute fixes the problem, as does disabling GateKeeper altogether. I tried passing "-o noautoxattr" (the exact spelling escapes me at this moment) to no avail. I could determine from the verbose output and presence of Apple Double files that the switch does disable native xattr, but that didn't solve the Keynote "damaged" error.

I'm not sure if the issue is in encfs or osxfuse. I assumed the former until I saw this issue. I tried putting the same file into a loopback filesystem and it works fine, so it's not a general osxfuse issue.

@glyph
Copy link
Author

@glyph glyph commented Oct 14, 2014

@mehaase Thanks for letting me know I'm not taking crazy pills over here :-).

@glyph
Copy link
Author

@glyph glyph commented Oct 14, 2014

Also, I should note that I've reproduced this issue on the latest Yosemite beta, so it's not version-specific to Mavericks.

@netheril96
Copy link

@netheril96 netheril96 commented Oct 18, 2014

I have the same problem with both encfs and sshfs.

@netheril96
Copy link

@netheril96 netheril96 commented Oct 18, 2014

@mehaase What is GateKeeper and how do I disable it? It really does not increase security at all for me, but annoying as hell.

@mehaase
Copy link

@mehaase mehaase commented Oct 20, 2014

"GateKeeper" is Apple's branding for signed code. I posted an answer on Stack Exchange that explains how to disable it.

@ni4
Copy link

@ni4 ni4 commented Nov 5, 2014

It also happens on my custom Cloud filesystem.
I made sure that com.apple.quarantine attribute has the same value as set and it equals to value in Downloads folder.
So it seems that Finder/GateKeeper/whatever else do some wrong checks for this attribute value - when it is removed all works fine.

@mingchen
Copy link

@mingchen mingchen commented Jan 15, 2015

I run into the same error on Macbook.
Mac OS X: 10.10.1.
enfs: 1.7.5
osxfuse: 2.7.4

Here is a workaround:

$ xattr -r -d com.apple.quarantine /path/to/encfs/mount/point

Refer http://apple.stackexchange.com/questions/129966/files-wrongly-considered-as-damaged-in-encfs-volume

@glyph
Copy link
Author

@glyph glyph commented Jan 17, 2015

That workaround only fixes the problem temporarily - next time you open the files it will recur.

Is there a way to disable xattrs entirely so that it can't set the quarantine attribute in the first place? I realize that this has security implications, so I don't think it should be enabled by default, but if I have a volume that I know I don't download things to directly, I should be able to move images and such to it.

@davek44
Copy link

@davek44 davek44 commented Apr 7, 2015

I have this same frustrating issue.

@Nxt3
Copy link

@Nxt3 Nxt3 commented Apr 30, 2015

I have the same issue. Oddly enough, if I try to open a video via QuickTime Player, the file will be damaged. If I open that same file with Preview, it works fine.

@Nxt3
Copy link

@Nxt3 Nxt3 commented Apr 30, 2015

I should add that I use encfs and OSXFUSE

@glyph
Copy link
Author

@glyph glyph commented Apr 30, 2015

At some point between my Jan 16th comment and now, I updated OSXFuse and this issue stopped happening, at least in the few cases I've since tried. Is everyone still having this problem on the most recent version?

@abedwardsw
Copy link

@abedwardsw abedwardsw commented May 1, 2015

This issue drives me nuts. ;-) Good idea on the OSXFuse, i've just updated mine from 2.6.4 to 2.7.6, will let you know if it's resolved for me.

@Nxt3
Copy link

@Nxt3 Nxt3 commented May 1, 2015

@abedwardsw Where did you get the 2.7.6 update? Did you build it from source?

@bfleischer
Copy link
Member

@bfleischer bfleischer commented May 1, 2015

@Nxt3 The only difference between 2.7.6 and 2.7.5 is a minor modification to the build script to support building with Xcode 6.3 and 6.4 for package managers like Homebrew. Feature-wise there is no difference at all.

@abedwardsw
Copy link

@abedwardsw abedwardsw commented May 1, 2015

Sorry, I ended up doing 2.7.5, brew was telling me 2.7.6, but I used the .dmg and went with 2.7.5

Unfortunately I still get the same corruption error until I run xattr -r -d com.apple.quarantine any time I create a new file. I really don't want to completely change the "Allow apps downloaded from: " to "Anywhere" as I like this feature. It seems like the files from encfs are wrongly being distinguished as an app instead of a file...

I am using encfs.

Any ideas? I'll try rebooting, but it does seem the new osxfuse version is loaded properly.

@glyph
Copy link
Author

@glyph glyph commented May 2, 2015

As another data point, I'm using sshfs, not encfs. It's possible that the relevant thing I updated was sshfs and not osxfuse itself. I will report versions and re-test when I get back to the machines with this installed.

@glyph
Copy link
Author

@glyph glyph commented May 2, 2015

So, I tried again, and discovered that I still have the issue. Versions:

SSHFS version 2.5 (OSXFUSE SSHFS 2.5.0)
OSXFUSE library version: FUSE 2.7.3 / OSXFUSE 2.7.5

The reason I thought it was fixed, is that some software created .DS_Store files. In a directory with an existing .DS_Store file, files can be opened and created with no problem; touch .DS_Store in a directory fixes the issue.

@glyph
Copy link
Author

@glyph glyph commented Jun 2, 2015

Hi - I guess it's time for my monthly checkin!

I think I have a workaround for this, although I'm not super happy about it, it seems a reasonable compromise. Perhaps this investigation will be useful to the maintainers figuring out another way to handle it.

I thought to myself, OS X uses system users and system daemons for all kinds of jobs, perhaps the kernel is expecting to be able to do some work as another user, or as root, to these files, and marking them as damaged when that doesn't work.

So I marked my sshfs binary as setuid, and I added the -o allow_other mount option to my sshfs command line, and ... I appear to be able to open and edit documents reliably on the mounted volume. I will continue to experiment and follow up if it stops working.

I am of course concerned about a setuid root binary lying around, but it seems better than the option of running a daemon which requires root privileges on the file server side of things to get NFS or SMB. :)

@glyph
Copy link
Author

@glyph glyph commented Jun 2, 2015

I just placed this as an answer on Stack Overflow as well.

@jooray
Copy link

@jooray jooray commented Jun 22, 2015

What about OSX Fuse mount option to clear the attribute when someone wants to set it?

@mehaase
Copy link

@mehaase mehaase commented Jun 22, 2015

@glyph Thanks for the update. I tried using the setuid trick with encfs. I'm able to open documents without an error message, but the quarantine attribute is still getting set, and when I try to save, I get a "This file is locked" error. If I look in the Finder, it doesn't have the locked attribute set. xattr -l still shows the quarantine flag being set.

So it's new behavior for encfs, but it's still broken behavior.

Update

I spent more time debugging. By turning on FUSE debug output (e.g. encfs -f -d ...), I can see that it is Pages itself that is setting the Quarantine flag, not Finder. I tried compiling both osxfuse and encfs from scratch on their respective master branches, to no avail. There is a similar issue open on encfs that links back to here. Notably, that issue links to a blog post from BoxCryptor that indicates this is a bug in OS X. At this point, I'm inclined to agree, because Apple's documentation of how the quarantine system works is pitiful. Given that Apple is a black hole when it comes to bug reports, I am officially giving up on trying to fix this "the right way".

Instead, I just patched osxfuse to ignore calls to getxattr when the attribute name is com.apple.quarantine. This is still a bit dangerous – quarantine won't work at all in FUSE volumes – but it's safer (and less hassle) than disabling Gate Keeper or the setuid trick.

diff --git a/lib/fuse.c b/lib/fuse.c
index 50591d5..71f1aee 100644
--- a/lib/fuse.c
+++ b/lib/fuse.c
@@ -2363,7 +2363,9 @@ int fuse_fs_getxattr(struct fuse_fs *fs, const char *path, const char *name,
 #endif
 {
        fuse_get_context()->private_data = fs->user_data;
-       if (fs->op.getxattr) {
+       if (strncmp("com.apple.quarantine", name, 20) == 0) {
+               return -ENODATA;
+       } else if (fs->op.getxattr) {
                if (fs->debug)
 #ifdef __APPLE__
                        fprintf(stderr, "getxattr %s %s %lu %lu\n",

It's ugly, but it works... If you want to try reproducing this, remove your existing osxfuse, disable kext signature verification, then:

$ brew install autoconf automake libtool
$ git clone --recursive -b master git://github.com/osxfuse/osxfuse.git osxfuse
$ cd osxfuse/fuse
$ git apply patch # use the patch from above
$ ./build.sh -t distribution
$ open /tmp/osxfuse/distribution/Packages/Core.pkg

The last step will open an installer to install osxfuse.

Of course, the downside here is turning off kext signing. (I already had it turned off, for other reasons.) The change itself is only in the library, not in the kext, so it's probably possible to install the official, signed kext and then build just the library, but it's getting late and I don't have time to try that right now...

@glyph
Copy link
Author

@glyph glyph commented Jun 23, 2015

Is there a way to just add an additional trust root / sign the locally built kext without disabling kext verification entirely?

@glyph
Copy link
Author

@glyph glyph commented Jun 23, 2015

@mehaase - I'm not quite clear on the division of responsibility between FUSE and SSHFS; particularly, does an equivalent to -o allow_other exist in EncFS or exist at the FUSE layer, or is it an SSHFS-ism?

With -o allow_other and SSHFS, the com.apple.quarantine xattr is indeed still getting set, but I can happily open the file from Finder, from open in the shell, from File→Open, and so on. I am not sure what information the quarantine attribute conveys now, but it looks like it's just saying that a given application touched it.

@glyph
Copy link
Author

@glyph glyph commented Jun 23, 2015

Also: I tried Preview, Keynote, and Pages, just to be sure.

bfleischer added a commit to osxfuse/kext that referenced this issue Oct 17, 2015
Starting with OS X 10.9 syspolicyd (which is running as root) calls
getxattr(2) when opening items in Finder. Blocking these calls results
in Finder displaying an error message. Therefore we are no longer
blocking vnop_getxattr calls by root even if allow_root or allow_other
is not set.

See osxfuse/osxfuse#162 for details.
bfleischer added a commit to osxfuse/kext that referenced this issue Oct 25, 2015
Starting with OS X 10.9 syspolicyd (which is running as root) calls
getxattr(2) when opening items in Finder. Blocking these calls results
in Finder displaying an error message. Therefore we are no longer
blocking vnop_getxattr calls by root even if allow_root or allow_other
is not set.

See osxfuse/osxfuse#162 for details.
@bfleischer bfleischer added this to the 2.8.2 milestone Oct 25, 2015
@bfleischer bfleischer self-assigned this Oct 25, 2015
@bfleischer
Copy link
Member

@bfleischer bfleischer commented Oct 25, 2015

The issue has been fixed in version 2.8.2.

@bfleischer bfleischer closed this Oct 25, 2015
@glyph
Copy link
Author

@glyph glyph commented Oct 26, 2015

@bfleischer Do you have any more details on what the resolution was, or why this was happening?

@bfleischer
Copy link
Member

@bfleischer bfleischer commented Oct 26, 2015

@glyph By default only the user that mounted the FUSE volume can access it. Starting with OS X 10.9 syspolicyd (which is running as root) calls getxattr(2) when opening items in Finder. Blocking these calls results in Finder displaying the misleading "is damaged and can’t be opened" error message. Therefore we are no longer blocking getxattr(2) calls by root in version 2.8.2.

@glyph
Copy link
Author

@glyph glyph commented Oct 26, 2015

@bfleischer this seems like kind of a brittle fix; other users (including root) can read some sensitive data, but not others, by default? It seems like maybe root should just be able to do everything all the time, or at least that a thing like allow_other should not require root access to turn on…

(Nevertheless thank you very much for fixing this bug.)

@mehaase
Copy link

@mehaase mehaase commented Oct 26, 2015

@bfleischer This is really awesome, thanks! 2.8.2 doesn't seem to be compatible with the latest encfs, but I tried the 3.0.7 build and it is working swimmingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants
You can’t perform that action at this time.