New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] Transition-Frontier-Controller #1168

Merged
merged 7 commits into from Nov 20, 2018

Conversation

Projects
None yet
3 participants
@bkase
Contributor

bkase commented Nov 18, 2018

Refactoring the existing ledger-builder-controller to a new simpler set of
components we're calling the transition-frontier-controller. At the top level,
we'll connect these components together in a similar manner to coda_lib --
just wiring together pipes. The new transition-frontier-controller includes the
merkle-mask and a safer history-catchup mechanism in addition to simplifying
asynchronous logic. The goal is to end up with a simple set of components that
will be robust towards future requirements changing and easy to trace dataflow
and test and debug.

bkase added some commits Nov 18, 2018

We are refactoring the existing ledger-builder-controller in order to make it
easier to reason about the async control flow of information in the system. The
old design didn't take this into account and as such, it's very hard to trace

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

The old design didn't take what into account? It's unclear from this sentence.

I would say something like "The old design was designed without taking asynchronous transactions and network delays into account".

The new transition-frontier-controller is responsible for handling incoming
transitions created by proposers either locally or remotely by: (a) validating
them, (b) adding them to a transition-frontier-tree as described below if

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

I have been calling this just the Transition_frontier instead of the Transition_frontier_tree.

so we must get enough info to go back to the locked state (where we know we've
achieved consensus).
2. We take advantage of the merkle-mask to instead of transitions, putting a `Breadcrumb.t` which contains an `External_transition.t` and a now light-weight `Ledger_builder.t` it also has a notion of the prior breadcrumb. See [RFC-0007](0007-persistent-ledger-builder-controller) for more info on merkle masks, and [Transition frontier](#transition-frontier) for more info about how these masks are configured.

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

This first sentence is a little weird. Also, what is the lightweight version of a ledger builder? I believe you are referring to the use of a mask and the aux data from external transition, right?

2. We take advantage of the merkle-mask to instead of transitions, putting a `Breadcrumb.t` which contains an `External_transition.t` and a now light-weight `Ledger_builder.t` it also has a notion of the prior breadcrumb. See [RFC-0007](0007-persistent-ledger-builder-controller) for more info on merkle masks, and [Transition frontier](#transition-frontier) for more info about how these masks are configured.
Rationale: We no longer need the asynchronous `Path_traversal` job that is present in the current ledger-builder-controller (simplifying complexity), and the frequent operation of answering sync-ledger queries no longer require materializing ledger-builders.

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

This is not the point that allows us to avoid Path_traversal. We can avoid Path_traversal by using the lookup table. This point allows us to avoid intermediate materialization of nodes.

#### Validator
* Input: `External_transition.t`
* Output: `External_transition.t` (we should consider making an `External_transition.Validated.t` for output here)

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

Good call.

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

Just thought of something. Where does rebroadcast fit into this? Should we rebroadcast after validation, or after processing?

This comment has been minimized.

@bkase

bkase Nov 19, 2018

Contributor

I'll update the RFC -- it should happen after validation I think

### Ledger Catchup
Input: `External_transition.t` (from catchup monitor)
Output: `Breadcrumb.t` (to processor)

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

Breadcrumb.t list here as well.

This comment has been minimized.

@bkase

bkase Nov 19, 2018

Contributor

Also think this can't be a list

additional step of invoking the sync ledger process.
<a href="sync-handler"></a>
### Sync Handler

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

I think handling both sync queries and history queries makes sense. However, I think that we should rename this then from Sync_handler to something like Query_handler.

Consult the following diagram:
![](https://github.com/CodaProtocol/coda/blob/55b4ca6ee66d3783669494cb28cb9f979dcd48c0/docs/res/transition_frontier_controller.png)

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

You should just reference the path instead. That way, this link works across history and always has the most recent version. This is part of the point of LFS.

![](https://github.com/CodaProtocol/coda/blob/55b4ca6ee66d3783669494cb28cb9f979dcd48c0/docs/res/transition_frontier_controller.png)
Blue arrows represent pipes and asynchronous boundaries of the system. Each arrow is annotated with the behavior when overflow of the pipe occurs.

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

Red arrows represent synchronous lines of access. Red components represent data and not processes.

### History catchup
@es92 notes that there is some nuance to catching up to breadcrumbs for

This comment has been minimized.

@nholland94

nholland94 Nov 19, 2018

Contributor

@es92 I would like to document some of this nuance to understand it better, especially in the context of this RFC.

This comment has been minimized.

@es92

es92 Nov 20, 2018

Contributor

Basically we should plan and document a system that

  1. has a way to download diffs back to the locked tip from a disconnected state (who will that be downloaded from?)
  2. can walk back the chain for multiple disconnected chains in parallel
  3. can accept children of disconnected chains while the chains are being connected up

This comment has been minimized.

@bkase

bkase Nov 20, 2018

Contributor

oh great we've discussed these all already

bkase added some commits Nov 19, 2018

@bkase bkase changed the title from RFC for transition-frontier-controller to [RFC] Transition-Frontier-Controller Nov 20, 2018

@bkase bkase merged commit 1498983 into master Nov 20, 2018

8 of 9 checks passed

Codacy/PR Quality Review Not up to standards. This pull request quality could be better.
Details
ci/circleci: build Your tests passed on CircleCI!
Details
ci/circleci: build_public Your tests passed on CircleCI!
Details
ci/circleci: build_withsnark Your tests passed on CircleCI!
Details
ci/circleci: test-all_sig_integration_tests Your tests passed on CircleCI!
Details
ci/circleci: test-all_stake_integration_tests Your tests passed on CircleCI!
Details
ci/circleci: test-unit-test Your tests passed on CircleCI!
Details
ci/circleci: test-withsnark Your tests passed on CircleCI!
Details
ci/circleci: tracetool Your tests passed on CircleCI!
Details

@nholland94 nholland94 deleted the rfc/transition-frontier-controller branch Nov 21, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment