FUSE over Samba: creating multiple files, every other file fails 'file already exists'

[JasonMiesionczek]

Sorry, this might be a bit long, bear with me.

Basics
So I have a FUSE filesystem using the low-level API that connects to a remote file store. I share the mount point via samba. On the other end I have a windows machine that has a drive mapped to the samba share. I have an app that creates a folder structure, and in the last folder, copies several files of various sizes into that last folder.

Issue
During the file copy process, very consistently, every other file, starting with the second file will fail with 'file already exists'. Each of the files i am copying are randomly generated and there is no possibility of collision. The files are stored in a 'cache' (just a designated folder on the system) before they are uploaded to the server once they are released.

Environment
Host: Ubuntu 16.04 server, Kernel 4.4.0-92-generic
Samba: 4.3.11-Ubuntu
libFUSE: current master from this repo as of 8/23/2017

Windows Box:
Freshly installed Windows 10

My Code
https://github.com/JasonMiesionczek/davfs

Research/Things i've tried
So I have sprinkled syslog calls throughout my fuse operation functions so i can see the various values of things as the directories are being created and the files are being copied.

paste bin of syslog calls

The above is the logs i generate. The interesting thing i've seen, is the first file creates correctly, is copied over, and then nothing else happens to that file until the next create operation. Once the second create operation completes, the second file is immediately flushed and released, and THEN the first file is flushed and released once the upload is complete:

Aug 24 09:52:09 demo-1 mount.davfs[2503]: creating file: 0zi2j5rh.cxv in folder: 3828783957664848378
Aug 24 09:52:09 demo-1 mount.davfs[2503]: created new file with id: 7345869211399090358
Aug 24 09:52:09 demo-1 mount.davfs[2503]: cache path: /var/cache/davfs/mnt/webdav/0zi2j5rh.cxv-7345869211399090358
... several seconds later, after first file has copied over ...
Aug 24 09:52:14 demo-1 mount.davfs[2503]: creating file: 3lwdqflk.ief in folder: 3828783957664848378
Aug 24 09:52:14 demo-1 mount.davfs[2503]: created new file with id: 11571085775386205333
Aug 24 09:52:14 demo-1 mount.davfs[2503]: cache path: /var/cache/davfs/mnt/webdav/3lwdqflk.ief-11571085775386205333
Aug 24 09:52:14 demo-1 mount.davfs[2503]: flushing: 11571085775386205333
Aug 24 09:52:14 demo-1 mount.davfs[2503]: releasing: 11571085775386205333
Aug 24 09:52:14 demo-1 mount.davfs[2503]: flushing: 7345869211399090358

This sequence of events seems strange to me. I have tried running the mount in single-thread mode and the result is the same.

I've tried stepping through my driver code and the FUSE code itself to see if there was a reply_err of EEXIST, which is what I am guessing would be returned if the file exists, but perhaps that is an incorrect assumption.

I have tried Samba 4.8.0, their current master branch, and result is also the same.

I've tried googling this issue, and found this post from 2005: https://sourceforge.net/p/fuse/mailman/message/7408249/ which indicated something similar, as an issue in the FUSE kernel module. But that seemed to occur on any file copy, so not exactly the same issue as what I am seeing.

I have some strace logs of the samba process, and can provide those if requested (they are pretty large).

At this point, I'm really stuck. Not sure where the error is coming from. Any help with this would be greatly appreciated.

[Nikratio]

Thanks for the report! Can you replicate this with one of the example filesystems, e.g. passthrough_fh or passthrough_ll? This would help a lot. libfuse is getting a lot of bug reports for issues that turn out to be bugs in the specific filesystem implementation, so I have become wary of spending too much time on issues that can't be reproduced with one of the examples.

[JasonMiesionczek]

I will try it with passthrough_ll and see if it happens there. Thanks.

[JasonMiesionczek]

My test app works fine when I use the passthrough_ll example. To your knowledge, has anyone used C++11 (with all its new bells and whistles) to create a filesystem?

[JasonMiesionczek]

Would you be willing to take a quick look at the code link I posted? specifically the DavFuse.cpp file where the fuse operations are done, just to see if I am doing something stupid somewhere? Any help would be greatly appreciated.

[Nikratio]

I did take a look but that's a little too much code to quickly determine if something isn't right, sorry. Also, I have no idea if someone has written a FUSE filesystem in C++11.

I'd suggest to enable the FUSE debug output and look for EEXIST replies - see if you can correlate that with the error that you get from Samba.

Since this doesn't seem to be a libfuse bug, I'd also suggest to take the discussion to the FUSE mailing list at https://lists.sourceforge.net/lists/listinfo/fuse-devel. You'll reach a lot more FUSE filesystem developers there, maybe someone else has an idea.

I wish I could be more helpful.

[JasonMiesionczek]

Yeah I've stepped through the code looking for EEXIST codes in both my code and the libfuse code, but haven't seen any. I already posted to the mailing list, so hopefully someone there might have some insight. Thanks.

[JasonMiesionczek]

Woohoo! I figured it out. I wasn't setting the ctim value in the stat struct. In the passthrough examples this is set via the stat system call, but in my case, I have to populate the stat struct manually, I was missing this value.