Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FUSE client resilliency #1268

Closed
onlyjob opened this issue Apr 8, 2020 · 1 comment
Closed

FUSE client resilliency #1268

onlyjob opened this issue Apr 8, 2020 · 1 comment

Comments

@onlyjob
Copy link

onlyjob commented Apr 8, 2020

With replication 001 and two volume servers I'm rsync'ing data to FUSE mount.
Then I'm adding a third volume server and after some time stopping second volume server which knocks off rsync with the following error:

rsync: close failed on "{{FILENAME}}": Input/output error (5)
rsync error: error in file IO (code 11) at receiver.c(853) [receiver=3.1.3]

weed mount logs the following:

I0408 19:11:20 73058 upload_content.go:196] failing to upload to http://192.168.0.2:9334/1,3ba72cb0d0cf Post "http://192.168.0.2:9334/1,3ba72cb0d0cf": dial tcp 192.168.0.2:9334: connect: connection refused
I0408 19:11:20 73058 dirty_page.go:179] upload data {{FILENAME}} to http://192.168.0.2:9334/1,3ba72cb0d0cf: Post "http://192.168.0.2:9334/1,3ba72cb0d0cf": dial tcp 192.168.0.2:9334: connect: connection refused
I0408 19:11:20 73058 dirty_page.go:133] {{FILENAME}} saveToStorage [0,25719): upload data: Post "http://192.168.0.2:9334/1,3ba72cb0d0cf": dial tcp 192.168.0.2:9334: connect: connection refused
E0408 19:11:20 73058 filehandle.go:169] flush {{FILENAME}}: upload data: Post "http://192.168.0.2:9334/1,3ba72cb0d0cf": dial tcp 192.168.0.2:9334: connect: connection refused

FUSE client should have gracefully used available volume servers. There were enough volume servers to continue. Client-side failure was unexpected.

@chrislusf
Copy link
Collaborator

Added a fix bb78ab9 to retry in case master assigned the request to the wrong volume servers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants