Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Routing heuristics #821

Merged
merged 86 commits into from Feb 22, 2019
Merged
Changes from 77 commits
Commits
Show all changes
86 commits
Select commit Hold shift + click to select a range
65e11a6
WIP integrating heuristics
araspitzu Jan 14, 2019
64b95ee
Use heuristics to evaluate the edge weight
araspitzu Jan 14, 2019
0049558
Update test for routing weight heuristics
araspitzu Jan 15, 2019
a506cea
Add route-weight-ratios to SendPayment/RouteRequest
araspitzu Jan 15, 2019
01bdaac
Add test for CLTV-optimized route selection
araspitzu Jan 16, 2019
ce6e953
Merge branch 'master' into routing_heuristics
araspitzu Jan 18, 2019
4d3fa2b
Finish merging master
araspitzu Jan 18, 2019
dfc1772
Merge branch 'master' of github.com:ACINQ/eclair into routing_heuristics
araspitzu Jan 23, 2019
bf8508d
WIP Normalize the weights in graph calculation
araspitzu Jan 23, 2019
beb1bc0
WIP Normalize the weights in graph calculation
araspitzu Jan 24, 2019
7e86292
Fix pathWeight calculation and rawCost aggregation
araspitzu Jan 24, 2019
e4a9075
Set global block count in RouteCalculationSpec
araspitzu Jan 24, 2019
fd88879
WIP adding boundaries to graph search, try a new flaky test parameter
araspitzu Jan 24, 2019
ce209e2
Add fee limit and test
araspitzu Jan 24, 2019
5fd2f4a
Update test channel_update with real world fee values
araspitzu Jan 24, 2019
6550661
Ignore test for route-too-expensive
araspitzu Jan 24, 2019
8b4836d
Add maxFeeBase and maxFeePct to SendCommand, use high fee tolerance i…
araspitzu Jan 24, 2019
c034dc3
Increase blockHeight difference in RouteCalculationSpec
Jan 25, 2019
7e2dcee
Improve docs and comments in Graph class
Jan 25, 2019
a8e2844
Add more comments to edgeWeightCompound and pathWeight
Jan 25, 2019
0f68ff6
Enforce searching boundaries when selecting a new shortest candidate,…
Jan 25, 2019
4d29e52
Use non lazy set in RouteCalculationSpec for Globals.blockCount
Jan 25, 2019
911a900
Enforce the cheapest route in flaky integration test
Jan 25, 2019
19928cd
Increase cltv factor in route calculation test
Jan 25, 2019
edd5d53
Expose randomized route selection feature in SendPayment, used in int…
Jan 27, 2019
569f2b0
Use maxHtlxMsat and widen the gap between scoreFactor and other facto…
araspitzu Jan 28, 2019
8efd793
Increase the scoreFactor in score heuristic test
araspitzu Jan 28, 2019
64b8b3d
Do correct percentage comparison in router (boundaries feature)
araspitzu Jan 28, 2019
1f66105
New cost function (fees not normalized)
araspitzu Jan 28, 2019
f0b12ec
Update comments, rename fields
araspitzu Jan 28, 2019
cccfcb7
Update test for max fee cap
araspitzu Jan 28, 2019
79be16c
Use wider blochHeight difference in RouteCalculationSpec test
araspitzu Jan 29, 2019
67029a5
Do not flatten the total factor to 1
araspitzu Jan 29, 2019
3fa9b34
Add comment in graph search cost function, use channelCapacity in test
araspitzu Jan 29, 2019
6f51328
Add CLTV max cap, set blockCount only in specific test
araspitzu Jan 29, 2019
1660744
Add maxCltv to SendPayment/RouteRequest
araspitzu Jan 29, 2019
105a792
Merge branch 'master' into routing_heuristics
araspitzu Jan 29, 2019
98deed9
Avoid making a copy of all the vertices in the graph in dijkstra
araspitzu Jan 30, 2019
96f66e0
Implement boundaries for graph searching with cost, cltv, and size
araspitzu Jan 30, 2019
ead28db
Enable searching for routes with size/CLTV/fee limits
araspitzu Jan 31, 2019
6559f21
expose the RouteParams in RouteRequest
araspitzu Jan 31, 2019
2ae43f5
Expose the RouteParams in SendPayment
araspitzu Jan 31, 2019
0ac2538
Rename DEFAULT_ROUTE_MAX_LENGTH
araspitzu Jan 31, 2019
2a55963
Use relaxed params for route request in integration test
araspitzu Jan 31, 2019
df77fee
If we couldn't find a route on the first attempt, retry relaxing the …
araspitzu Jan 31, 2019
ac272b7
Merge branch 'master' into route_fast_searching
araspitzu Jan 31, 2019
09aa95c
Relax maxFeePct in route request during RouterSpec
araspitzu Jan 31, 2019
05b5015
Avoid returning an empty path, collapse the route not found cases int…
araspitzu Feb 6, 2019
bfb9df3
When retrying to search for a route, relax 'maxCltv'
araspitzu Feb 6, 2019
7ac15a0
Merge branch 'master' into route_fast_searching
araspitzu Feb 6, 2019
4487b0a
Finish merging master
araspitzu Feb 6, 2019
4f7efc3
Move the default params for route searching in the conf, refactor tog…
araspitzu Feb 6, 2019
a0ffe0e
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 7, 2019
31c3f9e
WIP merging
araspitzu Feb 7, 2019
e7f5cb3
Finish merging route_fast_searching
araspitzu Feb 7, 2019
e9f537c
Pass the parameters along from SendPayment to RouteRequest
araspitzu Feb 8, 2019
6e43284
Remove weight-ratios values from the conf
araspitzu Feb 8, 2019
0bc2808
Remove max-payment-fee in favor of router.search-max-fee-pct
araspitzu Feb 8, 2019
966ba29
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 8, 2019
61e7174
Remove max-payment-fee in favor of router.search-max-fee-pct
araspitzu Feb 8, 2019
753c998
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 8, 2019
f95ce08
Group search params configurations into a block
araspitzu Feb 8, 2019
582f2f7
Add comments
araspitzu Feb 8, 2019
e4e69f4
Rename ROUTE_MAX_LENGTH
araspitzu Feb 8, 2019
daee9fe
Add formatter commands for tighter formatting
araspitzu Feb 8, 2019
faffac3
Merge branch 'master' into route_fast_searching
araspitzu Feb 8, 2019
b13b2dd
Merge branch 'route_fast_searching' into routing_heuristics
araspitzu Feb 8, 2019
69cf71a
Finish merging
araspitzu Feb 8, 2019
e8eb900
Merge branch 'master' into routing_heuristics
araspitzu Feb 12, 2019
60fa52e
Finish merging master
araspitzu Feb 12, 2019
bc7bec8
Merge branch 'master' into routing_heuristics
araspitzu Feb 14, 2019
9cb8517
Rework comments, remove amountMsat from edgeFeeCost
araspitzu Feb 14, 2019
0cd9ddb
Use the returning edges in 'ignoredEdges' when looking for a spur path
araspitzu Feb 14, 2019
5c3e67d
Merge branch 'master' into routing_heuristics
araspitzu Feb 20, 2019
0465bdb
Move WeightRatios into RouteParams
araspitzu Feb 20, 2019
b1bf584
Set weight ratios in the conf
araspitzu Feb 20, 2019
d3748b0
Log path-finding params when receiving a route request
araspitzu Feb 21, 2019
5c94cef
Remove unnecessary changes
araspitzu Feb 21, 2019
a0caf94
Enforce weight ratios to be between (0,1]
araspitzu Feb 21, 2019
0acbc5d
Make path-finding heuristics optional
araspitzu Feb 21, 2019
0273dae
re-enable test with different thresholds
araspitzu Feb 21, 2019
edd60ff
Rework compareTo with RichWeight - nicer syntax
araspitzu Feb 21, 2019
cc9196e
Re-enable heuristics in integration test
araspitzu Feb 21, 2019
2a1cfd0
Add integration test for heuristics
araspitzu Feb 22, 2019
05768ca
Formatting, reorg log entries in Router
araspitzu Feb 22, 2019
e544e31
Remove unused imports
araspitzu Feb 22, 2019
File filter...
Filter file types
Jump to…
Jump to file or symbol
Failed to load files and symbols.
+360 −165
Diff settings

Always

Just for now

@@ -103,6 +103,12 @@ eclair {
max-cltv = 1008 // max acceptable cltv expiry for the payment (1008 ~ 1 week)
fee-threshold-sat = 21 // if fee is below this value we skip the max-fee-pct check
max-fee-pct = 0.03 // route will be discarded if fee is above this value (in percentage relative to the total payment amount); doesn't apply if fee < fee-threshold-sat

// channel 'weight' is computed with the following formula: channelFee * (cltvDelta * ratio-cltv + channelAge * ratio-channel-age + channelCapacity * ratio-channel-capacity)
// the following parameters can be used to ask the router to use heuristics to find i.e: 'cltv-optimized' routes
ratio-cltv = 0.0 // when computing the weight for a channel, consider its CLTV delta in this proportion
This conversation was marked as resolved by araspitzu

This comment has been minimized.

Copy link
@sstone

sstone Feb 21, 2019

Member

We should document valid bounds for these ratio (between 0 and 1 and sum is 1 for example)

This comment has been minimized.

Copy link
@araspitzu

araspitzu Feb 21, 2019

Author Member

Done in 0acbc5d

ratio-channel-age = 0.0 // when computing the weight for a channel, consider its AGE in this proportion
ratio-channel-capacity = 0.0 // when computing the weight for a channel, consider its CAPACITY in this proportion
}
}

@@ -246,7 +246,10 @@ object NodeParams {
searchMaxRouteLength = config.getInt("router.path-finding.max-route-length"),
searchMaxCltv = config.getInt("router.path-finding.max-cltv"),
searchMaxFeeBaseSat = config.getLong("router.path-finding.fee-threshold-sat"),
searchMaxFeePct = config.getDouble("router.path-finding.max-fee-pct")
searchMaxFeePct = config.getDouble("router.path-finding.max-fee-pct"),
searchRatioCltv = config.getDouble("router.path-finding.ratio-cltv"),
This conversation was marked as resolved by araspitzu

This comment has been minimized.

Copy link
@sstone

sstone Feb 21, 2019

Member

We should check that configured ratios are valid

This comment has been minimized.

Copy link
@araspitzu

araspitzu Feb 21, 2019

Author Member

Done in a0caf94

searchRatioChannelAge = config.getDouble("router.path-finding.ratio-channel-age"),
searchRatioChannelCapacity = config.getDouble("router.path-finding.ratio-channel-capacity")
),
socksProxy_opt = socksProxy_opt
)
@@ -46,6 +46,7 @@ import fr.acinq.eclair.channel.Register
import fr.acinq.eclair.crypto.LocalKeyManager
import fr.acinq.eclair.io.{Authenticator, Server, Switchboard}
import fr.acinq.eclair.payment._
import fr.acinq.eclair.router.Graph.WeightRatios
This conversation was marked as resolved by araspitzu

This comment has been minimized.

Copy link
@sstone

sstone Feb 21, 2019

Member

Is it still needed ?

This comment has been minimized.

Copy link
@araspitzu

araspitzu Feb 21, 2019

Author Member

Nope, removed in 5c94cef

import fr.acinq.eclair.router._
import fr.acinq.eclair.tor.TorProtocolHandler.OnionServiceVersion
import fr.acinq.eclair.tor.{Controller, TorProtocolHandler}
@@ -54,7 +54,7 @@ class Autoprobe(nodeParams: NodeParams, router: ActorRef, paymentInitiator: Acto
case Some(targetNodeId) =>
val paymentHash = randomBytes(32) // we don't even know the preimage (this needs to be a secure random!)
log.info(s"sending payment probe to node=$targetNodeId payment_hash=$paymentHash")
paymentInitiator ! SendPayment(PAYMENT_AMOUNT_MSAT, paymentHash, targetNodeId, maxAttempts = 1)
paymentInitiator ! SendPayment(PAYMENT_AMOUNT_MSAT, paymentHash, targetNodeId, maxAttempts = 1, randomize = Some(true))
case None =>
log.info(s"could not find a destination, re-scheduling")
scheduleProbe()
@@ -28,6 +28,7 @@ import fr.acinq.eclair.crypto.Sphinx.{ErrorPacket, Packet}
import fr.acinq.eclair.crypto.{Sphinx, TransportHandler}
import fr.acinq.eclair.payment.PaymentLifecycle._
import fr.acinq.eclair.payment.PaymentRequest.ExtraHop
import fr.acinq.eclair.router.Graph.WeightRatios
import fr.acinq.eclair.router._
This conversation was marked as resolved by araspitzu

This comment has been minimized.

Copy link
@sstone

sstone Feb 21, 2019

Member

It seems that now this file not need to be changed

import fr.acinq.eclair.wire._
import scodec.Attempt
@@ -96,12 +97,12 @@ class PaymentLifecycle(sourceNodeId: PublicKey, router: ActorRef, register: Acto
// in that case we don't know which node is sending garbage, let's try to blacklist all nodes except the one we are directly connected to and the destination node
val blacklist = hops.map(_.nextNodeId).drop(1).dropRight(1)
log.warning(s"blacklisting intermediate nodes=${blacklist.mkString(",")}")
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes ++ blacklist, ignoreChannels, c.randomize, c.routeParams)
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes ++ blacklist, ignoreChannels, randomize = c.randomize, routeParams = c.routeParams)
goto(WAITING_FOR_ROUTE) using WaitingForRoute(s, c, failures :+ UnreadableRemoteFailure(hops))
case Success(e@ErrorPacket(nodeId, failureMessage: Node)) =>
log.info(s"received 'Node' type error message from nodeId=$nodeId, trying to route around it (failure=$failureMessage)")
// let's try to route around this node
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes + nodeId, ignoreChannels, c.randomize, c.routeParams)
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes + nodeId, ignoreChannels, randomize = c.randomize, routeParams = c.routeParams)
goto(WAITING_FOR_ROUTE) using WaitingForRoute(s, c, failures :+ RemoteFailure(hops, e))
case Success(e@ErrorPacket(nodeId, failureMessage: Update)) =>
log.info(s"received 'Update' type error message from nodeId=$nodeId, retrying payment (failure=$failureMessage)")
@@ -129,18 +130,18 @@ class PaymentLifecycle(sourceNodeId: PublicKey, router: ActorRef, register: Acto
// in any case, we forward the update to the router
router ! failureMessage.update
// let's try again, router will have updated its state
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes, ignoreChannels, c.randomize, c.routeParams)
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes, ignoreChannels, randomize = c.randomize, routeParams = c.routeParams)
} else {
// this node is fishy, it gave us a bad sig!! let's filter it out
log.warning(s"got bad signature from node=$nodeId update=${failureMessage.update}")
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes + nodeId, ignoreChannels, c.randomize, c.routeParams)
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes + nodeId, ignoreChannels, randomize = c.randomize, routeParams = c.routeParams)
}
goto(WAITING_FOR_ROUTE) using WaitingForRoute(s, c, failures :+ RemoteFailure(hops, e))
case Success(e@ErrorPacket(nodeId, failureMessage)) =>
log.info(s"received an error message from nodeId=$nodeId, trying to use a different channel (failure=$failureMessage)")
// let's try again without the channel outgoing from nodeId
val faultyChannel = hops.find(_.nodeId == nodeId).map(hop => ChannelDesc(hop.lastUpdate.shortChannelId, hop.nodeId, hop.nextNodeId))
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes, ignoreChannels ++ faultyChannel.toSet, c.randomize, c.routeParams)
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes, ignoreChannels ++ faultyChannel.toSet, randomize = c.randomize, routeParams = c.routeParams)
goto(WAITING_FOR_ROUTE) using WaitingForRoute(s, c, failures :+ RemoteFailure(hops, e))
}

@@ -159,7 +160,7 @@ class PaymentLifecycle(sourceNodeId: PublicKey, router: ActorRef, register: Acto
} else {
log.info(s"received an error message from local, trying to use a different channel (failure=${t.getMessage})")
val faultyChannel = ChannelDesc(hops.head.lastUpdate.shortChannelId, hops.head.nodeId, hops.head.nextNodeId)
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes, ignoreChannels + faultyChannel, c.randomize, c.routeParams)
router ! RouteRequest(sourceNodeId, c.targetNodeId, c.amountMsat, c.assistedRoutes, ignoreNodes, ignoreChannels + faultyChannel, randomize = c.randomize, routeParams = c.routeParams)
goto(WAITING_FOR_ROUTE) using WaitingForRoute(s, c, failures :+ LocalFailure(t))
}

@@ -194,6 +195,8 @@ object PaymentLifecycle {
maxAttempts: Int = 5,
randomize: Option[Boolean] = None,
routeParams: Option[RouteParams] = None) {


require(amountMsat > 0, s"amountMsat must be > 0")
}
case class CheckPayment(paymentHash: BinaryData)
Oops, something went wrong.
ProTip! Use n and p to navigate between commits in a pull request.
You can’t perform that action at this time.