Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
routing+routerrpc: improve prob. estimation for untried connections #3462
This PR changes mission control to partially base the estimated probability for untried connections on historical results obtained in previous payment attempts. This prevents the payment loop from getting stuck in an area of the network that is not of good quality. It also incentivizes routing nodes to keep all
The configuration parameter
Using this parameter, senders can configure how aggressive nodes should be penalized. A low a priori weight will make mission control switch quickly to using a different node if a tried node doesn't perform well. The downside of this is that the sender may miss out on low fee channels that could possibly complete the payment successfully.
Indirectly, the collective of senders also shapes the network with their settings of this parameter. When senders start to demand a higher performance from routing nodes, routing nodes need to meet that expectation in order to keep generating revenue.
Did a bit of performance testing with and without these changes. With a clean mission control history, there is no noticeable difference. This is expected, because there are hardly iterations in the loop inside
To see a performance effect, it is probably needed to have a fully loaded mission control history. It is a question how likely this situation is, especially on low-powered devices where pathfinding performance becomes important. There is also a default limit to the mission control history of 1000 past payments, although this can be changed.
I tried a simulation of a full mission control by inserting 10 fake results for every node visited, but didn't see an effect either. Db access during path finding is likely still where time is mainly spent.
I'd propose to not prematurely add optimizations like caching of the node probability in this pr and first see what feedback we get on it from users. I've added another commit that logs useful performance metrics.
Roasbeef left a comment
Completed an initial surface-level pass. Will do another one focusing on the core estimation changes to ensure I fully grok them. Overall I think this is a step in the right direction, but wonder how we can test these assumptions in the wild, and also if it's too early to make a change like this given that as we know it, node operators aren't very sophisticated yet.
We can always set the default a priori weight quite high, so the effect of this will be light initially. Give some time for users and operators to become aware of the new scoring.
halseth left a comment
I like the simplicity and conceived impact of this PR! I'm down with the current approach, deemed that we test it a bit to make sure we don't have any obvious regressions.
In addition to logging performance metrics as now, should we consider logging more (on trace) how the probabilities are calculated? I think we now log the final probability found for a channel, but not how it was calculated. Could maybe be useful to log whether it was calculated from a previous success, failure or node level failure?
guggero left a comment
Nice, clean PR! I was able to follow the changes quite easily even though the subject matter is very new to me. Only found a few nits in the code.
Not scope of this PR, just thoughts in general:
This commit changes the in-memory structure of the mission control state. It prepares for calculation of a node probability. For this we need to be able to efficiently look up the last results for all channels of a node.
This commit modifies the interpretation of node-level failures. Previously only the failing node was marked. With this commit, also the incoming and outgoing connections involved in the route are marked as failed. The change prepares for the removal of node-level failures in mission control probability estimation.
Probability estimates are amount dependent. Previously we assumed an amount, but that starts to make less sense when we make probability more dependent on amounts in the future.
This commit changes mission control to partially base the estimated probability for untried connections on historical results obtained in previous payment attempts. This incentivizes routing nodes to keep all of their channels in good shape.