Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

weed mount option to exit on first failure #1034

Closed
SvenDowideit opened this issue Aug 8, 2019 · 5 comments
Closed

weed mount option to exit on first failure #1034

SvenDowideit opened this issue Aug 8, 2019 · 5 comments

Comments

@SvenDowideit
Copy link

I have a container that runs weed mount, and because i forgot to hook its network up to the seaweedfs network, I expected it to run, fail and exit - interestingly, it hasn't exited

I'm going to assume this is intentional for interactive use, but it would be really cool to have a --no-retrys that exists faster

sven@yoga260:~$ docker logs seaweed-volume-plugin-sven
I0808 03:26:07     1 config.go:25] Reading security.toml from 
This is SeaweedFS version 30GB 1.42 linux amd64
I0808 03:26:07     1 config.go:28] Reading : Config File "security" Not Found in "[/ /root/.seaweedfs /etc/seaweedfs]"
mount point owner uid=0 gid=0 mode=drwxr-xr-x
current uid=0 gid=0
I0808 03:26:14     1 dir.go:230] list /mnt/docker-volumes/sven: rpc error: code = Unavailable desc = all SubConns are in TransientFailure, latest connection error: connection error: desc = "transport: Error while dialing dial tcp: lookup filer on 10.10.10.1:53: no such host"
@chrislusf
Copy link
Collaborator

In a distributed cluster, usually retry is needed.

@chrislusf
Copy link
Collaborator

close this? It's unlikely to implement this.

@SvenDowideit
Copy link
Author

mmm, ok, let me try to give a more concrete example where it would be good to have something.

I'm running weed mount any time a container is started / created that needs access to a subset of the data. This slows things down a little, but gives us the ability to tune the ownership and permissions to that container's needs. The problem is, in the rare event of a failure, we don't want the container to be stuck in a limbo - where its running, but unable to access the storage - we want the I/O error to cause the container to fail, as it'll get restarted.

this is especially important for the startup of short lived containers - which also makes me think it needs to be a workload specific tunable.

so - having thought this through a little - how would you feel about weed mount --timeout= and a similar --startup-timeout option - where it exits with an error code if things things haven't settled after that time

Defaulting to infinite would make it clear that this is a compromise choice (somewhat like the tradeoff of always writing, or failing a write when not enough volumes are available to match the replication policy)

@SvenDowideit
Copy link
Author

happy for you to tell me no, yes, do this instead, or yes, but you'd need to make the PR cos its not the most important thing - I'm very much in the explore/learn phase, and seaweedfs is only a part of the solution i'm working through

@chrislusf
Copy link
Collaborator

added a fix for this. Thanks for the suggestion!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants