Lightnode redesign

[ross-pure]

Motivation

The current light node is not up to date with both phi and the current darknode. The lightnode can no longer easily use the server implementation in the darknode, and so this part will need to be rewritten. This is also a chance to take lessons from the previous implementations and simplify the design somewhat - requests that get sent to the lightnode should mostly just be passed through to the darknodes, and the only logic that the lighnode should be concerned with is deciding which darknodes to send requests to and how to consolidate the (possibly multiple) responses from the darknodes into one response for the user.

Architecture

The architecture will be a basic pipeline, with some additions.

Pipeline Stages

Server

The first stage of the pipeline will be the server. We will include rate limiting here. The only other task before handing the request on will be some type converting.

Validator

This stage will include logic that checks whether or not a request should be sent to the darknode. The more we can filter out at this stage, the easier it is going to be for the darknodes.

Cache

The lightnode will use caching to improve response times for repeat requests. A response in the cache can have as a key the hash of the request object.

Dispatcher

This is responsible for sending a request to the darknodes and propagating a response. The logic here can vary in two ways:

• how many/which darknodes will the request be sent to?
• when sending a request to multiple darknodes, how will their responses be combined into a single response for the user that sent the request to the lightnode?

The interface for the dispatcher should be such that an implementer need only consider these two pieces of logic.

Additional Parts

Another thing that the lightnode will need to do is have a background task that keeps track of the darknodes that are currently in the network. This can be achieved by the task periodically selecting a random subset of the known darknodes and querying them for their peers. After receiving these peers, it will send them to the dispatcher task, which will contain a store of darknode addresses, and the dispatcher will update its store with any new addresses.

[loongy]

@ross-pure I thought we decided against the background task for querying peers, in favour of a general pattern for handling requests.

[ross-pure]

@loongy Yes, but I realised afterwards that we need some way for the lightnode to know about what darknodes are in the network. Obviously it will start with some bootstrap addresses, but I think the background task is necessary for it to learn about the other darknodes - I don't think a user of the lightnode should have to trigger this to happen.

[loongy]

@ross-pure sounds reasonable to me.

[jazg]

Closed as #9 has been merged.