Make fuse_custom_io support multi-threaded event loop

[legezywzh]

Define a new clone_fd() helper for fuse_custom_io, users can implement their own clone fd logic.

[bsbernd]

There is a something similar like that in my uring branch (fuse-over-uring also needs fd-clone, but per queue).

Could you please change subject a bit? I don't see the direct relation of "multi-threaded event loop" to clone-fd. Maybe that is how your custom IO works, but I think that is an implementation details.
So maybe something like "Add clone-fd to custom IO". The commit message can then describe that you need that for multi-threading.

[legezywzh]

There is a something similar like that in my uring branch (fuse-over-uring also needs fd-clone, but per queue).

io_uring is a great feature, hope it'll improve fuse filesystem's performance.

Could you please change subject a bit? I don't see the direct relation of "multi-threaded event loop" to clone-fd. Maybe that is how your custom IO works, but I think that is an implementation details. So maybe something like "Add clone-fd to custom IO". The commit message can then describe that you need that for multi-threading.

OK, thanks for this suggestion.

[bsbernd]

Thank you, merged.

[ashleypittman]

Sorry, I was going to comment on this one. fuse_session_custom_io doesn't have the same protection for adding entries to the end of structs that fuse_session_new does so extending this struct causes interoperability problems.

https://github.com/libfuse/libfuse/blob/master/lib/fuse_lowlevel.c#L3119

vs

libfuse/lib/fuse_lowlevel.c

Lines 3023 to 3034 in 73cd124

	struct fuse_session *fuse_session_new(struct fuse_args *args,
	const struct fuse_lowlevel_ops *op,
	size_t op_size, void *userdata)
	{
	int err;
	struct fuse_session *se;
	struct mount_opts *mo;
	if (sizeof(struct fuse_lowlevel_ops) < op_size) {
	fuse_log(FUSE_LOG_ERR, "fuse: warning: library too old, some operations may not work\n");
	op_size = sizeof(struct fuse_lowlevel_ops);
	}

The clever bit is passing in op_size and using that as the memcpy so that any new members are blank. With this PR then using master against binaries built with older versions will now read a out-of-band value and use it for the new callback.

[bsbernd]

Ah oh right. Well, we cannot change fuse_session_new because that breaks everything and not only custom IO. Better would be if all these structs would be private and would need to be allocated as transparent object through libfuse. Given that custom IO is a rather new feature, I think we can switch it to that.

[ashleypittman]

I think the trick that fuse_session_new uses is a good one, what would be ideal would be if fuse_session_custom_io had used it from the start.

I was putting fuse_session_new forward as a way that this should be done. Perhaps we could use the symbol versioning to have old vs new versions of fuse_session_custom_io here?

[bsbernd]

It would break the API - the worst case. We can update fuse_session_custom_io

[ashleypittman]

It would break the API - the worst case. We can update fuse_session_custom_io

That's probably a good idea, at least as a start. It looks like this affects 3.13-3.16 only.

Do you want me to push a patch?

[bsbernd]

I'm just in the middle to type it

[bsbernd]

PR is here: #930
@legezywzh could you please also review? We also need to try to get other users of the custom-io path to review it.

[bsbernd]

@legezywzh Please also see @ashleypittman patch to make it versioned #935. Actually I would like to get 3.17 with out ASAP - basically as soon as I'm done to finish putting the libfuse version into fuse_session_new.
I'm going to revert this patch here, if we don't have ABI/API compatibility ready for fuse_custom_io in time

[legezywzh]

Sorry for late response, I'm away from libfuse work for days, I'll go through your discussions today.