Added support for setting max-pages via config

[darthShadow]

Added support for setting max-pages via config. Also increased the max write from 128k to 1M.

Signed-off-by: Anagh Kumar Baranwal 6824881+darthShadow@users.noreply.github.com

[hanwen]

This needs a tests. Have a look at fs/directio_test.go for some inspiration.

You can do a large write by calling syscall.Write on 1 MB slice. On the FUSE side, the test should check that the requests are coming in as 1 MB in the Write call.

[rfjakob]

With this PR plus the change below against loopback I was able to get 1 MiB writes [1] but not 1 MiB reads [2]. @darthShadow did you manage to get large reads (how?)?

diff --git a/example/loopback/main.go b/example/loopback/main.go
index 4204c54..b2d5abe 100644
--- a/example/loopback/main.go
+++ b/example/loopback/main.go
@@ -104,6 +104,8 @@ func main() {
        opts.MountOptions.Options = append(opts.MountOptions.Options, "fsname="+orig)
        // Second column in "df -T" will be shown as "fuse." + Name
        opts.MountOptions.Name = "loopback"
+       opts.MountOptions.MaxWrite = 1024 * 1024
+       opts.MountOptions.MaxPages = 256
        // Leave file permissions on "000" files as-is
        opts.NullPermissions = true
        // Enable diagnostics logging

[1]:

dd if=/dev/zero bs=1M count=1 of=mnt/foo

20:23:19.336027 rx 38: WRITE i5004682 {Fh 1 [0 +1048576)  L 0 WRONLY,0x8000}  1048576b
20:23:19.336635 tx 38:     OK

[2]:

dd if=mnt/foo of=/dev/null bs=1M count=1

20:23:19.338086 rx 50: READ i5004682 {Fh 1 [0 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338119 tx 50:     OK,  131072b data (fd data)
20:23:19.338235 rx 52: READ i5004682 {Fh 1 [131072 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338242 tx 52:     OK,  131072b data (fd data)
20:23:19.338282 rx 54: READ i5004682 {Fh 1 [262144 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338289 tx 54:     OK,  131072b data (fd data)
20:23:19.338342 rx 56: READ i5004682 {Fh 1 [393216 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338353 tx 56:     OK,  131072b data (fd data)
20:23:19.338408 rx 58: READ i5004682 {Fh 1 [524288 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338426 tx 58:     OK,  131072b data (fd data)
20:23:19.338454 rx 60: READ i5004682 {Fh 1 [655360 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338460 tx 60:     OK,  131072b data (fd data)
20:23:19.338529 rx 62: READ i5004682 {Fh 1 [786432 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338534 tx 62:     OK,  131072b data (fd data)
20:23:19.338587 rx 64: READ i5004682 {Fh 1 [917504 +131072)  L 0 RDONLY,0x8000} 
20:23:19.338592 tx 64:     OK,  131072b data (fd data)

[rfjakob]

PS: As for the public API, I think we should do what libfuse does: calculate MaxPages from MaxWrite ( libfuse/libfuse@027d0d1 ) and don't expose MaxPages to the user at all

[rfjakob]

I went down the rabbit hole about the read sizes a little bit:

A normal (=not direct i/o) read ends up in filemap_read(), and, quoting from the function comment:

 * Copies data from the page cache.  If the data is not currently present,
 * uses the readahead and readpage address_space operations to fetch it.

So it will use readahead to get the data. Following some indirections, the size of readahead requests for FUSE filesystems is ultimately limited by the default readahead size:

#define VM_READAHEAD_PAGES (SZ_128K / PAGE_SIZE)

For normal disks, you can change the readahead size via /sys/block/*/queue/read_ahead_kb, but FUSE uses the dummy disk device noop_backing_dev_info, and I don't see how you would change anything there.

Now because O_DIRECT bypasses readahead, the limit does not apply, so 1 MiB reads work with O_DIRECT:

dd if=b/foo iflag=direct bs=1M of=/dev/null

14:17:30.773557 rx 34: READ i5004682 {Fh 1 [0 +1048576) LOCKOWNER L 15366234143055493023 DIRECT,0x8000} 
14:17:30.773997 tx 34:     OK,  1048576b data (fd data)

[rfjakob]

Pushed my attempt: https://review.gerrithub.io/c/hanwen/go-fuse/+/530724

[ShuranZhang]

May I ask if there are any possibilities to reconsider or continue on this issue? Since Linux now supports 1Mib i/o for up to 256 pages, but for this library it seems there is no way to configure up to 1Mib i/o. I am implementing a FUSE system that each read request will need to do some pre-processing, so enlarging the max page per read could significantly shorten the latency adds up from each read request.

[rfjakob]

Oh, looks like this got stuck on me. Maybe I'll find time to pick this up again.

[ShuranZhang]

Thank you for the response and I appreciate your contributions to this library! It has helped me a lot and I would like to quickly follow up on this opened issue. Sorry for keeping asking and I would love to pick up to see if there is anything I can do if you have no bandwidth recently.

[rfjakob]

Yeah you can update the patch to address Han-Wen's comments at https://review.gerrithub.io/c/hanwen/go-fuse/+/530724

[andrewchambers]

Really interested in this change - It seems very close to the finish line.

[andrewchambers]

It looks like the patch has been updated and as far as I can tell all the issues are resolved - though the gerrit interface is somewhat confusing to me.

[rfjakob]

No this is still unresolved: https://review.gerrithub.io/c/hanwen/go-fuse/+/530724/comments/15afc10b_ce660eff

Note that this went on the backburner for me because it did not significantly improve performance for gocryptfs.

[andrewchambers]

@rfjakob doesn't this check test resolve the problem?

if srv.KernelSettings().Flags&fuse.CAP_MAX_PAGES == 0 {

afaik macos wouldn't set this flag and the tests will be skipped.

[rfjakob]

Yes.

[andrewchambers]

@rfjakob if that is the case, what is the remaining work? or should @hanwen be able to approve/merge this now?

[rfjakob]

What do you mean? The check is not in the test...?

[rfjakob]

Oh I'm looking at an old version of the patchset.

[rfjakob]

Let me test this on an old kernel and I will ping Han-Wen if things pass.

[hanwen]

see 265a392