Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Full command forwarding #75

Merged
merged 21 commits into from May 20, 2019
Merged

Full command forwarding #75

merged 21 commits into from May 20, 2019

Conversation

jumaffre
Copy link
Contributor

This PR introduces support for full RPC forwarding if a Write command is issued on a follower.

Changes

  • In rpcendpoint.h, the rpc response is no longer synchronously written to the session if the command requires forwarding. Instead, nothing is written to the session (i.e. the session stays open) until the forwarded response comes back from the leader.
  • There is a new Forwarder object that is only known by enclave.h and the frontends that are allowed to forward (only the USERS one for now) and that deals with the appending/reading the appropriate headers to the message before being sent/received (encrypted).
  • The previous CMake RPC_FORWARD_TO_LEADER is gone as forwarding is now enabled by default.

Notes

  • Nothing is store in the enclave while a command is being forwarded. Instead, the session id of the client that initiated the forwarded command is passed to the leader and back to the follower so that the follower knows which client connection to send the forwarded response back to.
  • This does not support daisy chaining, i.e. if a follower forwards a command to a leader and if, by the time the forwarded command is received by that leader an election has occurred and the leader has changed, the command is not re-forwarded to the new leader. Instead, the previous redirection error message with the IP/Port of the new leader is sent back. In practice, this is unlikely to happen.
  • This new behaviour is currently tested by the e2e_logging and e2e_scenarios tests.

Still to do

  • Write better unit tests for it [in progress now]
  • Make the join protocol work with this new async behaviour
  • Support redirection for all frontends (i.e. management, members and nodes)

@jumaffre jumaffre requested a review from a team as a code owner May 17, 2019 09:40
src/enclave/enclave.h Show resolved Hide resolved
CMakeLists.txt Show resolved Hide resolved
@jumaffre jumaffre merged commit b10b931 into master May 20, 2019
@achamayou achamayou deleted the full_forwarding branch May 21, 2019 12:37
eddyashton referenced this pull request in eddyashton/CCF Mar 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants