Windows files with external zone information fail to copy onto an encrypted drive on a Samba share

[blinkenlight]

This may well be more an interaction problem between Samba and cppcryptfs rather than strictly a cppcryptfs bug, but it is still dang annoying. In short, I have an Ubuntu Samba server (ext4 based) which serves a network share I am accessing from a Windows 7 machine (NTFS based), on which I am running cppcryptfs. From this Windows PC I was able to create an encrypted drive that keeps its files on the Samba share from the Ubuntu machine, and everything works quite fine, actually - except when I try copying over a file that was downloaded from the internet.

Unsurprisingly, the Windows machine tags it with zone information (that results in that "this file came from another computer... Unblock" checkbox in the file's properties under Windows); however, every time I attempt to copy such a file with Windows Explorer onto the encrypted drive (X:\dirA\dirB), the copy operation apparently happens, then at the end it resets to zero progress and happens again only faster (wtf...), then I get an error message like this:

Could not find this item. This is no longer located in X:\dirA\dirB. Verify the item's location and try again. [try again] [skip] [cancel]

While the dialog is open, the new file seems to actually be present on the Ubuntu Samba server, but it gets deleted as soon as the dialog is closed.

This is of course a solvable problem for me - I can very well simply "unblock" all the files have this attribute and after that everything works fine; nonetheless, this still shouldn't be happening, especially considering that the error doesn't actually hint to the real cause of the issue, which might thoroughly confuse others. It would be nice to get rid of this if it's something simple, or at least have a warning somewhere about it if it's not really fixable...

Linux Samba server: Ubuntu 14.04.6 LTS x32 (ext4)
Windows client: Windows 7 SP1 x64 (NTFS)
cppcryptfs 1.4.0.26 using Dokany 1.2.2.1000

[bailey27]

I think Windows keeps track of if a file was downloaded from the internet vial a named stream it adds to the file (called Zone.Identifier).

It looks like Samba does not support named streams.

When the filesystem is mounted, the OS queries cppcryptfs for the filesystem feature flags it supports. cppcryptfs queries the underlying filesystem for the feature flags it supports, and then it reports to the OS that it supports the intersection (AND) of the feature flags that it knows how to support and what the underlying filesystem also knows how to support.

If for some reason the underlying filesystem fails to return its supported feature flags, then cppcryptfs reports that it can do everything that cppcryptfs knows how to do (assumes the underlying fileystem can do all those things as well).

I'm running Ubuntu 16.04 LTS and my Samba correctly reports that it does not have the Named Data Stream feature. So this Name Streams bit gets cleared.

When I download a PDF, I can see it has the Zone.Identifier stream in it. If I copy it to cppcrypts volume that resides on a samba share, then explorer just copies the file. The file loses the Zone.Identifier stream when it gets copied.

My theory about what is happening with your setup is that either your 14.04 samba incorrectly reports that it supports named streams, or it fails to return the feature flags and cppcryptfs is just returning the bits for what it supports without taking in account what the underlying os can really do.

There's a program that comes with windows called fsutil.

If you run "fsutil fsinfo volumeinfo d:" it shows you what that volume reports that it can do.

Could you please run

fsutil fsinfo volumeinfo k:

(change k to the drive letter of your mounted cppcryptfs volume that is using samba for file storage)

And let me know if you see "Supports Named Streams" or if it fails. The whole output would be better.

Here is what I get:

C:\>fsutil fsinfo volumeinfo c:  // my system drive, needs administrator command prompt to work
Volume Name : Acer
Volume Serial Number : 0x229c6e0f
Max Component Length : 255
File System Name : NTFS
Is ReadWrite
Not Thinly-Provisioned
Supports Case-sensitive filenames
Preserves Case of filenames
Supports Unicode in filenames
Preserves & Enforces ACL's
Supports file-based Compression
Supports Disk Quotas
Supports Sparse files
Supports Reparse Points
Returns Handle Close Result Information
Supports POSIX-style Unlink and Rename
Supports Object Identifiers
Supports Encrypted File System
Supports Named Streams
Supports Transactions
Supports Hard Links
Supports Extended Attributes
Supports Open By FileID
Supports USN Journal

C:\>fsutil fsinfo volumeinfo m: // dokany mirror drive
Volume Name : DOKAN
Volume Serial Number : 0x19831116
Max Component Length : 255
File System Name : NTFS
Is ReadWrite
Error:  Access is denied.
Supports Case-sensitive filenames
Preserves Case of filenames
Supports Unicode in filenames
Preserves & Enforces ACL's
Supports Named Streams

C:\>fsutil fsinfo volumeinfo j: // non-samba cppcryptfs
Volume Name :
Volume Serial Number : 0xed9e3224
Max Component Length : 255
File System Name : NTFS
Is ReadWrite
Error:  Access is denied.
Supports Case-sensitive filenames
Preserves Case of filenames
Supports Unicode in filenames
Preserves & Enforces ACL's
Supports Named Streams

C:\>fsutil fsinfo volumeinfo k: // samba cppcryptfs
Volume Name :
Volume Serial Number : 0x4d01a63
Max Component Length : 255
File System Name : NTFS
Is ReadWrite
Error:  Access is denied.
Supports Case-sensitive filenames
Preserves Case of filenames
Supports Unicode in filenames
Preserves & Enforces ACL's

And post if it prints "Supports Named Streams" in the output? It should not print that (mine doesn't with samba but does without samba).

Note: I see an "Error: access denied" when I do it either on samba or non-samba cppcryptfs volumes and on dokany mirror volumes at the point where I think fsutil is querying if the volume is thinly provisioned or not. I think this error isn't related to the problem you are describing.

If this turns out to be the problem, then I could add a setting in cppcryptfs to disable named streams.

[blinkenlight]

Yup, you got it - it's reporting support for Named Streams; I'm not actually surprised at all subtleties like this don't get preserved on what isn't actually an NTFS file system, but yeah, it's still claiming to support them. Output follows:

C:\>fsutil fsinfo volumeinfo Z:
Volume Name : Private
Volume Serial Number : 0xc40c17be
Max Component Length : 255
File System Name : NTFS
Supports Case-sensitive filenames
Preserves Case of filenames
Supports Unicode in filenames
Preserves & Enforces ACL's
Supports Remote Storage
Supports Named Streams

[blinkenlight]

Okay, the plot definitely chickens, as they say... I remembered I actually have Wireshark installed, and in spite of being a complete idiot when it comes to actually using it, apparently I managed to snoop the exchange that happens when cppcryptfs mounts the drive, specifically the message that contained the FS_INFO/FileFsAttributeInformation query (or rather, the reply to it) and... now I'm confused, because it apparently reports NO support for named streams:

FS Attributes: 0x0001006f, Case Sensitive Search, Case Preserving, Unicode On Disk, Persistent ACLs, Volume Quotas, Sparse Files, Supports OIDs
    .... .... .... .... .... .... .... ...1 = Case Sensitive Search: This FS supports CASE SENSITIVE SEARCHes
    .... .... .... .... .... .... .... ..1. = Case Preserving: This FS supports CASE PRESERVED NAMES
    .... .... .... .... .... .... .... .1.. = Unicode On Disk: This FS supports UNICODE NAMES
    .... .... .... .... .... .... .... 1... = Persistent ACLs: This FS supports PERSISTENT ACLs
    .... .... .... .... .... .... ...0 .... = Compression: This FS does NOT support compressed files
    .... .... .... .... .... .... ..1. .... = Volume Quotas: This FS supports VOLUME QUOTAS
    .... .... .... .... .... .... .1.. .... = Sparse Files: This FS supports SPARSE FILES
    .... .... .... .... .... .... 0... .... = Reparse Points: This FS does NOT support reparse points
    .... .... .... .... .... ...0 .... .... = Remote Storage: This FS does NOT support remote storage
    .... .... .... .... .0.. .... .... .... = LFN APIs: This FS does NOT support lfn apis
    .... .... .... .... 0... .... .... .... = Volume Is Compressed: This FS volume is NOT compressed
    .... .... .... ...1 .... .... .... .... = Supports OIDs: This FS supports OIDs
    .... .... .... ..0. .... .... .... .... = Supports Encryption: This FS does NOT support encryption
    .... .... .... .0.. .... .... .... .... = Named Streams: This FS does NOT support named streams
    .... .... .... 0... .... .... .... .... = Read Only Volume: This is a read/write volume

[bailey27]

I'm sorry for the delay in getting back to you.

I've added a setting in gocryptfs.conf that lets you disable the named streams support.

It's called FsFeatureDisableFlags. It's a hexadecimal mask that gets inverted and then anded with the feature flags that cppcryptfs claims to support.

So if you want to disable named streams, you can set it to "40000"

It should go in the same level of the JSON as Version, VolumeName, and FeatureFlags.

e.g. my test gocryptfs.conf looks like this

{
        "Creator": "cppcryptfs v1.4.0.25",
        "EncryptedKey": "jhxZmj6M1e2l7yq3Yswn9vZYNCZTEv/MoCVputIus6a2V7GzkiZjl3+/H5R2L11zIxzImcSVRsEdQzb8AmXpvg==",
        "ScryptObject": {
                "Salt": "DAAZ/5QOIduKezVitQOVXkFByQZaIZqmkV4R7pxy7V8=",
                "N": 65536,
                "R": 8,
                "P": 1,
                "KeyLen": 32
        },
        "Version": 2,
        "VolumeName": "",
        "FsFeatureDisableMask": "40000",
        "FeatureFlags": [
                "EMENames",
                "LongNames",
                "DirIV",
                "HKDF",
                "Raw64",
                "GCMIV128"
        ]
}

I recommend making a backup of your gocryptfs.conf before editing it. Also, you'll probably need to make the file not read-only so you can save it.

I recommend editing it on Linux so the line endings don't get messed up or changed.

Please try this and let me know if it solves the problem.

If it does, then I'll add UI for it on the create filesystem tab.

I'm attaching a signed test executable in a zip file.
cppcryptfs_name_streams_test.zip

The sha256 of the included exe (not the zip file) is

0dd85cd2ba56f3ab3ec426fa991873a312add3d8183e4462a0eb9b9d73e394f0 *cppcryptfs.exe

[bailey27]

@blinkenlight
I don't know why cppcryptfs is saying that streams are supported if samba is saying they are not.

Using FsFeatureDisableMask should work for disabling streams support anyway.

[blinkenlight]

Thanks, you handled this quite quickly actually... :) I tested the new version and sure enough I can copy even downloaded files without any problems now with the mask in place. Nicely done!

[bailey27]

@blinkenlight

Great! I'll add this feature to the next release.

[bailey27]

I just now released 1.4.0.27 which has this feature and UI for it (on the create fs tab).

If anybody needs to add the option retroactively, please look through this ticket to see how, or re-open it and ask a question if you need clarification.

[heikoh81]

I have the exact same problem. Ticking "Disabled Named Streams" does NOT resolve the problem.
Encrypted Folder resides on NTFS-system-drive C: The error only appears to files (PDf for testing) downloaded via Firefox. If I use Chrome to save the files, they can be opened without problems.

I don't have the problem if the encrypted folder resides on a Linux Samba share.
I don't have the problem if I save the PDF with Firefox to the unencrypted Windows download folder.

As far as I remember the problem first occured in the middle of 2019, I did not spend much time on it, back then. Probably there was an update to Firefox that now adds these streams?

Files downloaded with Firefox to encrypted folder get the extra data stream, I checked this with:
https://www.nirsoft.net/utils/alternate_data_streams.html