Hinted handoff should not process inactive nodes #4339
Conversation
otoolep
force-pushed
the
hh_drop_server
branch
2 times, most recently
from
a59c3cb
to
ad29b88
Compare
otoolep
changed the title
It is possible that a node is dropped, while data is still pending in the HH queues for that node. If this happens, the HH processor should skip those nodes, or the system continually complains about unreachable nodes. Data deletion is still explicitly left the purge system.
otoolep
force-pushed
the
hh_drop_server
branch
from
74922fa
to
dd8e804
Compare
| for nodeID, q := range p.queues { | ||
| // Only process queues for active nodes. | ||
| var activeQueues map[uint64]*queue | ||
| if p.MetaProvider == nil { |
There was a problem hiding this comment.
Can this actually happen? There is always a meta store (remote or local) on each server right?
There was a problem hiding this comment.
I wanted to make the Provider optional, so if you want all the complexity, you add it. Otherwise the Provider will just handle the nodes it got. It also makes testing cleaner. I don't feel strongly about this.
|
Just a couple questions, but still gets a +1. |
| res := make(chan error, len(p.queues)) | ||
| for nodeID, q := range p.queues { | ||
| // Only process queues for active nodes. | ||
| var activeQueues map[uint64]*queue |
There was a problem hiding this comment.
I think pulling this activeQueues logic into a func activeQueues() help keep the Process function simpler.
|
Thanks for the reviews @jwilder @corylanou Now that I see @jwilder originally considered that this functionality would be part of "purge", let me think about adding it there. It might be cleaner. |
|
I still plan to fix this, and leaving the deletion of data to happen within This seems more in-line with the principle of |
|
Sounds good. |
|
Closing favour of #4377 |
It is possible that a node is dropped, while data is still pending in the HH queues for that node. If this happens, the HH processor should skip those nodes, or the system continually complains about unreachable nodes.
Data deletion is still explicitly left to the purge system, though an end-user can always remove HH data by hand if necessary.
This issue is not very serious, but can cause confusion and results in connection errors in the logs.