Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fs: TestReaddirplusParallel is flaky #476

Closed
navytux opened this issue Jun 12, 2023 · 6 comments
Closed

fs: TestReaddirplusParallel is flaky #476

navytux opened this issue Jun 12, 2023 · 6 comments

Comments

@navytux
Copy link
Contributor

navytux commented Jun 12, 2023

From #351 (comment) :

TestReaddirplusParallel:

warning: Inode.Path: n2 is orphaned, replacing segment with ".go-fuse.9192819535914916238/deleted"
--- FAIL: TestReaddirplusParallel (0.13s)
    mem_test.go:295: got 97 want 100
20:30:57.716748 writer: Write/Writev failed, err: 2=no such file or directory. opcode: RELEASE
warning: Inode.Path: n3 is orphaned, replacing segment with ".go-fuse.5138903199487262716/deleted"
warning: Inode.Path: n4243 is orphaned, replacing segment with ".go-fuse.17225733708800861888/deleted"
warning: Inode.Path: n4575 is orphaned, replacing segment with ".go-fuse.4889753782423046442/deleted"
warning: Inode.Path: n18972 is orphaned, replacing segment with ".go-fuse.138001850428569319/deleted"
warning: Inode.Path: n19274 is orphaned, replacing segment with ".go-fuse.1962055963177458141/deleted"
warning: Inode.Path: n19401 is orphaned, replacing segment with ".go-fuse.7308332124064692491/deleted"
warning: Inode.Path: n19526 is orphaned, replacing segment with ".go-fuse.1308850176864522172/deleted"
warning: Inode.Path: n20210 is orphaned, replacing segment with ".go-fuse.2188351064224111264/deleted"
warning: Inode.Path: n21518 is orphaned, replacing segment with ".go-fuse.15131317205544715670/deleted"
FAIL
FAIL    github.com/hanwen/go-fuse/fs    2.116s
...
FAIL

real    0m25,285s
user    0m15,996s
sys     0m11,576s
@navytux
Copy link
Contributor Author

navytux commented Jun 12, 2023

Maybe #472 is related.

@navytux
Copy link
Contributor Author

navytux commented Jun 14, 2023

Reliable way to reproduce:

(neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/github.com/hanwen/go-fuse/fs$ go test -run TestReaddirplusParallel -count 1000 -failfast
--- FAIL: TestReaddirplusParallel (0.03s)
    mem_test.go:295: got 99 want 100
FAIL
exit status 1
FAIL    github.com/hanwen/go-fuse/fs    1.256s

(neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/github.com/hanwen/go-fuse/fs$ go test -run TestReaddirplusParallel -count 1000 -failfast
--- FAIL: TestReaddirplusParallel (0.04s)
    mem_test.go:295: got 98 want 100
FAIL
exit status 1
FAIL    github.com/hanwen/go-fuse/fs    8.927s

(neo) (z-dev) (g.env) kirr@deca:~/src/neo/src/github.com/hanwen/go-fuse/fs$ go test -run TestReaddirplusParallel -count 1000 -failfast
--- FAIL: TestReaddirplusParallel (0.04s)
    mem_test.go:295: got 95 want 100
FAIL
exit status 1
FAIL    github.com/hanwen/go-fuse/fs    10.289s

go-fuse v2.3.0-11-g255ab74

@hanwen
Copy link
Owner

hanwen commented Jun 16, 2023

There is another element of randomization in the hashmap iteration order. (sigh).

@hanwen
Copy link
Owner

hanwen commented Jun 16, 2023

See golang/go@3be4d95

this change is old, but testing suggests this is still the method used. There are 8 different iteration orders. With the current approach in go-fuse you have to be somewhat unlucky for entries to move around between READDIRPLUS responses, but it's essentially the same problem as in #391

@hanwen
Copy link
Owner

hanwen commented Jun 16, 2023

plan: store children as

childrenMap map[string]int
children []struct {name string; inode*Inode}

this could also provide for persistent readdir semantics, by storing the changeCounter in the children list, and handing that out as a readdir cookie.

@hanwen hanwen closed this as completed in 0b3e1fd Jun 22, 2023
@navytux
Copy link
Contributor Author

navytux commented Jun 22, 2023

Thanks for the fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants