Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Routing heuristics #821

Merged
merged 86 commits into from Feb 22, 2019
Merged
Changes from 1 commit
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
65e11a6
WIP integrating heuristics
araspitzu Jan 14, 2019
64b95ee
Use heuristics to evaluate the edge weight
araspitzu Jan 14, 2019
0049558
Update test for routing weight heuristics
araspitzu Jan 15, 2019
a506cea
Add route-weight-ratios to SendPayment/RouteRequest
araspitzu Jan 15, 2019
01bdaac
Add test for CLTV-optimized route selection
araspitzu Jan 16, 2019
ce6e953
Merge branch 'master' into routing_heuristics
araspitzu Jan 18, 2019
4d3fa2b
Finish merging master
araspitzu Jan 18, 2019
dfc1772
Merge branch 'master' of github.com:ACINQ/eclair into routing_heuristics
araspitzu Jan 23, 2019
bf8508d
WIP Normalize the weights in graph calculation
araspitzu Jan 23, 2019
beb1bc0
WIP Normalize the weights in graph calculation
araspitzu Jan 24, 2019
7e86292
Fix pathWeight calculation and rawCost aggregation
araspitzu Jan 24, 2019
e4a9075
Set global block count in RouteCalculationSpec
araspitzu Jan 24, 2019
fd88879
WIP adding boundaries to graph search, try a new flaky test parameter
araspitzu Jan 24, 2019
ce209e2
Add fee limit and test
araspitzu Jan 24, 2019
5fd2f4a
Update test channel_update with real world fee values
araspitzu Jan 24, 2019
6550661
Ignore test for route-too-expensive
araspitzu Jan 24, 2019
8b4836d
Add maxFeeBase and maxFeePct to SendCommand, use high fee tolerance i…
araspitzu Jan 24, 2019
c034dc3
Increase blockHeight difference in RouteCalculationSpec
Jan 25, 2019
7e2dcee
Improve docs and comments in Graph class
Jan 25, 2019
a8e2844
Add more comments to edgeWeightCompound and pathWeight
Jan 25, 2019
0f68ff6
Enforce searching boundaries when selecting a new shortest candidate,…
Jan 25, 2019
4d29e52
Use non lazy set in RouteCalculationSpec for Globals.blockCount
Jan 25, 2019
911a900
Enforce the cheapest route in flaky integration test
Jan 25, 2019
19928cd
Increase cltv factor in route calculation test
Jan 25, 2019
edd5d53
Expose randomized route selection feature in SendPayment, used in int…
Jan 27, 2019
569f2b0
Use maxHtlxMsat and widen the gap between scoreFactor and other facto…
araspitzu Jan 28, 2019
8efd793
Increase the scoreFactor in score heuristic test
araspitzu Jan 28, 2019
64b8b3d
Do correct percentage comparison in router (boundaries feature)
araspitzu Jan 28, 2019
1f66105
New cost function (fees not normalized)
araspitzu Jan 28, 2019
f0b12ec
Update comments, rename fields
araspitzu Jan 28, 2019
cccfcb7
Update test for max fee cap
araspitzu Jan 28, 2019
79be16c
Use wider blochHeight difference in RouteCalculationSpec test
araspitzu Jan 29, 2019
67029a5
Do not flatten the total factor to 1
araspitzu Jan 29, 2019
3fa9b34
Add comment in graph search cost function, use channelCapacity in test
araspitzu Jan 29, 2019
6f51328
Add CLTV max cap, set blockCount only in specific test
araspitzu Jan 29, 2019
1660744
Add maxCltv to SendPayment/RouteRequest
araspitzu Jan 29, 2019
105a792
Merge branch 'master' into routing_heuristics
araspitzu Jan 29, 2019
98deed9
Avoid making a copy of all the vertices in the graph in dijkstra
araspitzu Jan 30, 2019
96f66e0
Implement boundaries for graph searching with cost, cltv, and size
araspitzu Jan 30, 2019
ead28db
Enable searching for routes with size/CLTV/fee limits
araspitzu Jan 31, 2019
6559f21
expose the RouteParams in RouteRequest
araspitzu Jan 31, 2019
2ae43f5
Expose the RouteParams in SendPayment
araspitzu Jan 31, 2019
0ac2538
Rename DEFAULT_ROUTE_MAX_LENGTH
araspitzu Jan 31, 2019
2a55963
Use relaxed params for route request in integration test
araspitzu Jan 31, 2019
df77fee
If we couldn't find a route on the first attempt, retry relaxing the …
araspitzu Jan 31, 2019
ac272b7
Merge branch 'master' into route_fast_searching
araspitzu Jan 31, 2019
09aa95c
Relax maxFeePct in route request during RouterSpec
araspitzu Jan 31, 2019
05b5015
Avoid returning an empty path, collapse the route not found cases int…
araspitzu Feb 6, 2019
bfb9df3
When retrying to search for a route, relax 'maxCltv'
araspitzu Feb 6, 2019
7ac15a0
Merge branch 'master' into route_fast_searching
araspitzu Feb 6, 2019
4487b0a
Finish merging master
araspitzu Feb 6, 2019
4f7efc3
Move the default params for route searching in the conf, refactor tog…
araspitzu Feb 6, 2019
a0ffe0e
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 7, 2019
31c3f9e
WIP merging
araspitzu Feb 7, 2019
e7f5cb3
Finish merging route_fast_searching
araspitzu Feb 7, 2019
e9f537c
Pass the parameters along from SendPayment to RouteRequest
araspitzu Feb 8, 2019
6e43284
Remove weight-ratios values from the conf
araspitzu Feb 8, 2019
0bc2808
Remove max-payment-fee in favor of router.search-max-fee-pct
araspitzu Feb 8, 2019
966ba29
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 8, 2019
61e7174
Remove max-payment-fee in favor of router.search-max-fee-pct
araspitzu Feb 8, 2019
753c998
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 8, 2019
f95ce08
Group search params configurations into a block
araspitzu Feb 8, 2019
582f2f7
Add comments
araspitzu Feb 8, 2019
e4e69f4
Rename ROUTE_MAX_LENGTH
araspitzu Feb 8, 2019
daee9fe
Add formatter commands for tighter formatting
araspitzu Feb 8, 2019
faffac3
Merge branch 'master' into route_fast_searching
araspitzu Feb 8, 2019
b13b2dd
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 8, 2019
69cf71a
Finish merging
araspitzu Feb 8, 2019
e8eb900
Merge branch 'master' into routing_heuristics
araspitzu Feb 12, 2019
60fa52e
Finish merging master
araspitzu Feb 12, 2019
bc7bec8
Merge branch 'master' into routing_heuristics
araspitzu Feb 14, 2019
9cb8517
Rework comments, remove amountMsat from edgeFeeCost
araspitzu Feb 14, 2019
0cd9ddb
Use the returning edges in 'ignoredEdges' when looking for a spur path
araspitzu Feb 14, 2019
5c3e67d
Merge branch 'master' into routing_heuristics
araspitzu Feb 20, 2019
0465bdb
Move WeightRatios into RouteParams
araspitzu Feb 20, 2019
b1bf584
Set weight ratios in the conf
araspitzu Feb 20, 2019
d3748b0
Log path-finding params when receiving a route request
araspitzu Feb 21, 2019
5c94cef
Remove unnecessary changes
araspitzu Feb 21, 2019
a0caf94
Enforce weight ratios to be between (0,1]
araspitzu Feb 21, 2019
0acbc5d
Make path-finding heuristics optional
araspitzu Feb 21, 2019
0273dae
re-enable test with different thresholds
araspitzu Feb 21, 2019
edd60ff
Rework compareTo with RichWeight - nicer syntax
araspitzu Feb 21, 2019
cc9196e
Re-enable heuristics in integration test
araspitzu Feb 21, 2019
2a1cfd0
Add integration test for heuristics
araspitzu Feb 22, 2019
05768ca
Formatting, reorg log entries in Router
araspitzu Feb 22, 2019
e544e31
Remove unused imports
araspitzu Feb 22, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.
+16 −5
Diff settings

Always

Just for now

Improve docs and comments in Graph class

  • Loading branch information...
araspitzu
araspitzu committed Jan 25, 2019
commit 7e2dceeb73a7e27ab29d1b76d1cb1b7c9fd508f3
@@ -14,9 +14,11 @@ object Graph {
def weight(wr: WeightRatios): Double = compoundWeight.totalWeight(wr)
}

// Carries a compound weight for an edge, values are normalized except for 'rawCost' which contains the actual fees
// Carries a compound weight for an edge, values are normalized except for 'rawCost' which contains the actual fees in millisatoshi
// Weight is used consistently to refer to the value used to 'weight' the edges in the graph, cost is used consistently to refer to fees.
case class CompoundWeight(feesNormalized: Double, cltvDeltaNormalized: Double, scoreNormalized: Double, rawCost: Long) {

// Computes the aggregate weight with given the ratios
def totalWeight(wr: WeightRatios): Double = {
feesNormalized * wr.costFactor + cltvDeltaNormalized * wr.cltvDeltaFactor + scoreNormalized * wr.scoreFactor
}
@@ -49,11 +51,15 @@ object Graph {
/**
* Yen's algorithm to find the k-shortest (loopless) paths in a graph, uses dijkstra as search algo. Is guaranteed to terminate finding
* at most @pathsToFind paths sorted by cost (the cheapest is in position 0).
*
* @param graph
* @param sourceNode
* @param targetNode
* @param amountMsat
* @param pathsToFind
* @param wr an object containing the ratios used to 'weight' edges when searching for the shortest path
* @param currentBlockHeight the height of the chain tip (latest block)
* @param boundaries a predicate function that can be used to impose limits on the outcome of the search
* @return
*/
def yenKshortestPaths(graph: DirectedGraph,
@@ -159,6 +165,9 @@ object Graph {
* @param amountMsat the amount (in millisatoshis) we want to transmit
* @param ignoredEdges a list of edges we do not want to consider
* @param extraEdges a list of extra edges we want to consider but are not currently in the graph
* @param wr an object containing the ratios used to 'weight' edges when searching for the shortest path
* @param currentBlockHeight the height of the chain tip (latest block)
* @param boundaries a predicate function that can be used to impose limits on the outcome of the search
* @return
*/

@@ -210,6 +219,7 @@ object Graph {
case true => g.getIncomingEdgesOf(current.key)
case false =>
val extraNeighbors = extraEdges.filter(_.desc.b == current.key)
// the resulting set must have only one element per shortChannelId
g.getIncomingEdgesOf(current.key).filterNot(e => extraNeighbors.exists(_.desc.shortChannelId == e.desc.shortChannelId)) ++ extraNeighbors
}

@@ -221,15 +231,15 @@ object Graph {
// calculate the length of the partial path given the new edge (current -> neighbor)
val neighborPathLength = pathLength.get(current.key) + 1

// note: 'cost' contains the smallest known cumulative cost (amount + fees) necessary to reach 'current' so far
// note: there is always an entry for the current in the 'cost' map
// note: 'weight' contains the smallest known cumulative weight necessary to reach 'current' so far
// note: there is always an entry for the current in the 'weight' map
val newMinimumCompoundWeight = edgeWeightCompound(amountMsat, edge, weight.get(current.key), neighbor == sourceNode, currentBlockHeight)

// test for ignored edges
if (edge.update.htlcMaximumMsat.forall(newMinimumCompoundWeight.rawCost + amountMsat <= _) &&
newMinimumCompoundWeight.rawCost + amountMsat >= edge.update.htlcMinimumMsat &&
neighborPathLength <= ROUTE_MAX_LENGTH && // ignore this edge if it would make the path too long
boundaries(newMinimumCompoundWeight) &&
boundaries(newMinimumCompoundWeight) && // check if this neighbor edge would break off the 'boundaries'
!ignoredEdges.contains(edge.desc)
) {

@@ -251,7 +261,7 @@ object Graph {
// update the queue
vertexQueue.insert(WeightedNode(neighbor, newMinimumCompoundWeight)) // O(1)

// update the minimum known distance array
// update the minimum known weight array
weight.put(neighbor, newMinimumCompoundWeight)
}
}
@@ -297,6 +307,7 @@ object Graph {
val cltvFactor = normalize(channelCltvDelta, CLTV_LOW, CLTV_HIGH)

// Weights every edge by its cost in fees, normalized. The actual cost is carried away separately.
// NB. 'edgeFees' here is only the fee that must be paid to traverse this @param edge
val edgeFees = edgeCost(edge, amountMsat + compoundCostSoFar.rawCost, isNeighborTarget) - amountMsat
val costFactor = normalizeCost(amountMsat, edgeFees)

ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.