Pull lots of generated code out into the gorums package

[johningve]

This PR attempts to move as much of the generated code out into the gorums package. What remains in the generated code is mostly wrappers that convert types between the application types and the generic ProtoMessage types.

Fixes #61
Fixes #42 probably
Related #17

[meling]

Looks great!

[meling]

Don't we want this?

[meling]

This is a re-posting of @Raytar's comment that were unexpectedly deleted by a bug in GitHub.

I've been thinking about the change done in 8b3912e, and I'm not sure if it's best to keep it or revert it.

Option 1: To sum it up, the commit removes the context parameter from the unicast and multicast calltypes, and instead creates a timeout context internally before sending the message. This timeout can be set by using a manager option. I am not a fan of needing timeouts everywhere, which is why I am now reconsidering this commit.

These contexts are needed to abort the stream if it gets stuck (if the receiving node stops receiving messages or is very slow). The reason I removed the context parameter is that the code that calls the calltype does not know whether or not a message has been sent, making it difficult to cancel the context correctly.

Before this commit, the calling code would have to deal with the cancel function somehow.
One solution would be to ignore the cancel:

ctx, _ := context.WithTimeout(context.Background(), 10 * time.Millisecond)
node.Unicast(ctx, msg)

But now we get a govet warning:

the cancel function returned by context.WithTimeout should be called, not discarded, to avoid a context leak

Looking at the source code for the timeout context, the context is obviously canceled when the timeout happens, and so I assume that the code above does not actually leak a context after the timeout. However, it is not ideal to ignore the documentation.

Option 2: Another solution would be to keep the cancel function and run it at a time when the message is assumed to be sent. For example, in HotStuff, votes are sent to the leader using a unicast. We could store the context cancel function until the next time the node votes, at which point it is safe to cancel the last context because it doesn't matter whether or not the previous vote was sent. This way we might be able to use a cancel context instead of a timeout context as well. It would look like this:

func handlePropose(msg Proposal) {
  // prepare vote
  ...
  // cancels the context from the previous proposal.
  // ensures that the stream will be reconnected if it gets stuck,
  // so that the next vote can be put in the queue without blocking.
  cancel()
  ctx, cancel = context.WithCancel(context.Background())
  leader.Vote(ctx, vote)
}

Option 3: Another possible solution is to make the Unicast/Multicast methods in Gorums block until the message is actually sent. I haven't looked into how this could be done, but then the context can simply be canceled as soon as the method returns. But this might create a problem in HotStuff: Currently, a proposal is handled synchronously for each leader, and the votes are sent at the end of the handlers. But if the voting becomes a blocking function, then the node may not be able to handle another message from the proposer until the vote times out and unblocks.

I am leaning towards reverting this commit for now and implementing the second solution (canceling the previous context when the next message is being prepared) in HotStuff.

[meling]

Option 2: I like this option for the HotStuff scenario, and similar scenarios, where the next call to the same method actually cancels/overrides any previous calls. Hence, this should work for scenarios like the Propose scenario in HotStuff and idempotent messages. The question I have is whether or not there are scenarios where that's not the desired behavior.

Option 3: This might be possible now that we no longer depend on gRPC for this stuff; we might actually know when a message has been sent, which wasn't possible in gRPC. I haven't studied this in-depth.