Use of the libfuse compatibility layer (from fusepy) without cygwin

[tstordyallison]

Would it be possible to build the fuse compatibility API to be loadable without cygwin?

If I've understood what is going on properly between fusepy (via ctypes) <-> 'win fsp fuse compat' <-> winfsp then I think it should be.

Would be great for using fusepy directly (and then being able to use the same code, or something close on *nix).

[billziss-gh]

It is actually possible to use the WinFsp FUSE API without Cygwin.

WinFsp provides two different API's:

• A native API that is useful primarily to Windows developers.
• A FUSE API that can be used by Cygwin and MSVC applications. [Basically it is done by adding a "hidden" parameter to every FUSE API that captures important information about the environment like its malloc/free implementation, etc.]

So the WinFsp FUSE API can be readily used by MSVC Windows apps written in C/C++ (that can include C header files).

Regarding FUSEPY: it works on Cygwin (see this PR: terencehonles/fusepy#54), but unfortunately I expect that it cannot currently be used directly by FUSEPY on Windows, because FUSEPY expects to find fuse_* symbols and WinFsp exports fsp_fuse_* symbols (because of the "hidden" environment parameter explained above).

On Cygwin this gap is bridged by using the cygfuse package, which translates calls to fuse_* to fsp_fuse_* symbols. On Windows this bridge layer does not exist. But you make me realize that it is needed for Windows as well (for projects like FUSEPY on Windows) and for this reason I will mark this as an enhancement to WinFsp.

[tstordyallison]

Thanks - that would be great.

I had already found your branch and given it go :)

If I have time, I'll probably have a go at putting together some ctypes bindings for the WinFsp native API.

[billziss-gh]

If I have time, I'll probably have a go at putting together some ctypes bindings for the WinFsp native API.

That would be great. Keep me posted if you do so.

[tstordyallison]

Still rather interested in this one if you have some spare time to rejig the exports :)

We have our filesystem working now on fuse on Linux, so rather inclined to use that if at all possible via fusepy...

[billziss-gh]

If it is a Linux fusepy file system, how do you intend to run it on Windows? Via Cygwin or Windows Python?

If you are planning to use Cygwin there is now a separate Cygfuse project. This can be readily used by fusepy, because it exports the right symbols (fuse_* instead of fsp_fuse_*).

If you do not want to go through Cygwin, the problem of fsp_fuse_* vs fuse_* symbols still exists. Unfortunately I have not had the time to look into this.

[tstordyallison]

There's nothing very Linux specific about it - the fusepy implementation communicates with a remote system where the data resides. Being able to use winfsp (from a Windows Python runtime) with fusepy would be great.

[tstordyallison]

Took a quick look at this - exporting the fuse.h functions (just adding the __dllspec(dllexport) instead of static) seems to do the trick for me for the basics.

I'll try to get around to running something more complete and let you know.

[billziss-gh]

Took a quick look at this - exporting the fuse.h functions (just adding the __dllspec(dllexport) instead of static) seems to do the trick for me for the basics.

@tstordyallison you are correct. This is pretty much how the original version of Cygfuse was implemented:

https://github.com/billziss-gh/winfsp/blob/master/opt/cygfuse/cygfuse.c#L62

It may be worth following a similar approach and creating a "Winfuse" DLL that it is just a thin layer over WinFsp.

I'll try to get around to running something more complete and let you know.

Fantastic!

[billziss-gh]

Commit db05667 should fix this issue. See issue #60. I include some of the more relevant comments here:

In the end I chose the path of least resistance and added the fuse_* symbols to the WinFsp DLL (winfsp-x64.dll, winfsp-x86.dll). Adding a new winfuse.dll would probably increase confusion for users. It would also require me to change the WinFsp installer and the FLOSS exception to the license.

These new symbols are not supposed to be used by native C/C++ programs. Such programs are supposed to include the <fuse.h> headers, which expose the fuse_* symbols as static inline functions that reference the fsp_fuse_* symbols. This way the proper "environment" (malloc, etc.) is picked up automatically.

The symbols are currently added to the import libraries (winfsp-x64.lib, winfsp-x86.lib). I would rather they did not, but I do not know of any obvious way to exclude them. [Please let me know if you do; I do not want programs to link with these symbols accidentally to avoid the caveats we discussed.]

These symbols are for use with programs that use FFI technology such as jnr-fuse and fusepy ONLY.

[billziss-gh]

BTW, the fix above makes fusepy usable from native Windows. My fusepy fork already works for Cygwin, and with a couple of minimal changes it also works for Windows.

[tstordyallison]

Brill.

Are you planning on submitting a PR to the fusepy project?

Also, what are your thoughts on supporting #44 in fusepy? I found my FS was struggling with thousands of getattr calls... was hoping it would help (especially given the GIL).

[billziss-gh]

Are you planning on submitting a PR to the fusepy project?

I have already submitted PR terencehonles/fusepy#54 for Cygwin, but it is still lingering. I will update it to include native Windows. We might have to make some noise so that the fusepy maintainer notices :)

Hello, @terencehonles!

Also, what are your thoughts on supporting #44 in fusepy?

Issue #44 is already resolved for WinFsp (I think it is included in RC3). We might have to do some extra work on the fusepy side to get it to work, but I think it is a good idea.

[billziss-gh]

Also, what are your thoughts on supporting #44 in fusepy?

I added billziss-gh/fusepy#1 to track this.

[billziss-gh]

You may have seen this already from #60, but just in case:

WinFsp 2017 is out 🎉

Get it here: https://github.com/billziss-gh/winfsp/releases/tag/v1.0

If this works for you, we can probably (finally) close this issue.

[billziss-gh]

My fusepy fork now runs on Windows (and Cygwin) without any issues. The changes are in the windows branch.

[tstordyallison]

Great - I had more or less made the same change locally.

The 1.0 is working nicely for me now too, thanks!

[tstordyallison]

(with the exception that you do a much more proper job of it and look it up in the registry etc!)

[billziss-gh]

@tstordyallison I am glad 1.0 works for you.

Consider adding your voice to the terencehonles/fusepy#54 PR, so that it may be looked at and possibly accepted.

I will close this, but will ping you directly if I make additional changes to my fusepy fork (esp. regarding the readdir improvements).