Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could it fail fast on the propose dropping while leader changed #8975

Closed
absolute8511 opened this issue Dec 5, 2017 · 3 comments
Closed

Comments

@absolute8511
Copy link
Contributor

While leader is transferred, the proposal may be dropped. But we can not get the notify so we can do some retry work.
For example, as below

https://github.com/coreos/etcd/blob/b041ce5d514a4b4aaeefbffb008f0c7570a18986/etcdserver/v3_server.go#L572

	cctx, cancel := context.WithTimeout(ctx, s.Cfg.ReqTimeout())
	defer cancel()

	start := time.Now()
	s.r.Propose(cctx, data)
	proposalsPending.Inc()
	defer proposalsPending.Dec()

	select {
	case x := <-ch:
		return x.(*applyResult), nil
	case <-cctx.Done():
		proposalsFailed.Inc()
		s.w.Trigger(id, nil) // GC wait
		return nil, s.parseProposeCtxErr(cctx.Err(), start)
	case <-s.done:
		return nil, ErrStopped
	}

We just wait context timeout to get to know we failed to propose which is blocking too long. I think we should handle the proposal dropping event to fail fast. One possible way is to wake up the cctx.Done while dropping proposal.

@absolute8511 absolute8511 changed the title Fail fast on the propose dropping while leader changed Could it fail fast on the propose dropping while leader changed Dec 5, 2017
@xiang90
Copy link
Contributor

xiang90 commented Dec 13, 2017

We could be smarter about this. However, the raft library does not return an error for dropping messages right now. Do you want to help on this one?

@absolute8511
Copy link
Contributor Author

Yeah, I am looking into this.

@wenjiaswe
Copy link
Contributor

wenjiaswe commented Oct 1, 2018

@absolute8511 Vincent, I saw that you have a coupled of PR merged, thanks! I am closing this issue. Please feel free to reopen it if it's not fixed and need further action here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

No branches or pull requests

4 participants