askrene: handle maxparts better. #8688
Conversation
I added amount_msat_accumulate for the "a+=b" case, but I was struggling with a name for the subtractive equivalent. After some prompting, ChatGPT suggested deduct. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This is not worth optimizing that I can see. Using a non-debug build I get the following times for tests/test_askrene.py::test_real_data Before: 143 seconds After: 141 seconds. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We've already freed the working_ctx, and the fail path does that again. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
It can only fail on overflow, but if it did, the fail path frees working_ctx and returns "error_message". Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Make it calculate on demand. This will be useful when we call it from elsewhere. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't need the indexes array, we can use this directly. We still set up the indexes array (for now) after we call this. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This removes the index array from code after increase_flows()m, so we use the flows array directly. The next step will be to make increase_flows() use the flows array, and remove the index array indirection entirely. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We don't need to convert to strings, we can compare directly. This removes the final use of the index arrays. This of course changes the order of returned routes, which alters test_real_biases, since that biases against the final channel in the *first* route. Took me far too long to diagnose that! Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
We added _noidx versions of the sort functions, but now they're the only ones, we can rename them to the old names. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Rewrite it, so it properly takes into account interactions between flows by using reservations. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
For 1, we use single-path. For 0, reject. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Now we simply call it at the end. We need to check it hasn't violated fee maxima, but otherwise it's simple. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Changelog-Fixed: Plugins: `askrene` now handles limits on number of htlcs much more gracefully.
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
| (*flows)[i]->delivers)) | ||
| abort(); | ||
| tal_arr_remove(flows_index, i); | ||
| tal_arr_remove(flows, i); |
There was a problem hiding this comment.
Maybe add tal_free((*flows)[i]); before removing the pointer.
| if (!amount_msat_is_zero((*flows)[i]->delivers)) | ||
| break; | ||
| tal_arr_remove(&flows_index, i); | ||
| tal_arr_remove(flows, i); |
| if (error_message) | ||
| goto fail; | ||
| /* htlc_min is not met for this flow */ | ||
| tal_arr_remove(flows, i); |
| if (fa->path[i] < fb->path[i]) | ||
| return -1; | ||
| if (fa->path[i] > fb->path[i]) | ||
| return 1; |
There was a problem hiding this comment.
This bothers me a bit. Because path[i] is just a pointer to a struct gossmap_chan.
So that gives us some ordering but only to discriminate that one path is different than another
but otherwise not well defined in real life. I think it is ok for what refine does anyways.
| (*flows)[flows_index[i]]->delivers = combined; | ||
| tal_arr_remove(&flows_index, j); | ||
| flow->delivers = combined; | ||
| tal_arr_remove(flows, i+1); |
There was a problem hiding this comment.
Should we free (*flows)[i+1]?
| struct amount_msat addition; | ||
|
|
||
| /* Because flows can interact, we reserve them all, removing one at a time. */ | ||
| reservations = tal_arr(NULL, struct reserve_hop *, tal_count(flows)); |
There was a problem hiding this comment.
I would allocate in the working_ctx.
| /* Keep the largest-capacity flows */ | ||
| size_t orig_num_flows = tal_count(*flows); | ||
| asort(*flows, orig_num_flows, reverse_cmp_capacity, cast_const(struct route_query *, rq)); | ||
| tal_resize(flows, num_parts); |
There was a problem hiding this comment.
Like previous comments. I am pedantic about flows removed that are not freed.
| return false; | ||
| /* Keep the largest-capacity flows */ | ||
| size_t orig_num_flows = tal_count(*flows); | ||
| asort(*flows, orig_num_flows, reverse_cmp_capacity, cast_const(struct route_query *, rq)); |
There was a problem hiding this comment.
I think it is better to keep the largest-deliver flows.
If we only go for the highest capacity we may throw away the low cost benefits of the MCF.
Hypothetical scenario: MCF finds 3 flows but maxparts=2,
flow 1: deliver=10, cost=0, capacity=0
flow 2: deliver=7, cost=1, capacity=5
flow 3: deliver=1, cost=10, capacity=100
It is better to keep flows 1 and 2 by accomodating 1 more unit of flow in flow2 at 1 value expense (per flow),
than to keep flows 2 and 3 by accomodating 5 more units of flow in flow2 at cost 1 and 5 in flow3 at cost 100.
There was a problem hiding this comment.
The trade-off is: if we prioritize the delivery value already computed by MCF then we find better solutions, but
we might fail to find feasible solutions sometimes. If we prioritize capacity then we generally find
bad solutions though we find feasibility more often than the alternative.
I prefer the first and argue that MCF does its thing but it has limitations, and one of them is
it cannot handle a constraint on the maximum number of parts. I can accept that.
Maybe in the future if we extend the MCF to constant costs then we can become smarter in limiting
maxparts.
Lagrang3
left a comment
Lagrang3
left a comment
There was a problem hiding this comment.
1. Removal of unnecessary caching.
Yes. Caching at this phase was overzealous, when in fact we spend most of the time in doing MCF/
2. Simplify sorting code (sort array of flows directly, don't use index arrays)
Indexing was a necessity after using cached channel information.
3. make increase_flows() more general, so it can be used for > 2% increase.
I like it. It is preferred to the previous approach in my opinion.
Looping and remove 1/n of the shortage at every iteration worries me a little bit because
the shortage amount decays exponentially but the factor decreases the more flows there are
and we won't stop until 10sats.
For example, it takes 7 iterations to reduce some shortage to 1% of its initial value if
there are 2 paths. 44 iterations for 10 paths and 459 iterations for 100 paths.
Also the complexity is the number of iterations times the number of paths, hence
2 * 7 = 14
10 * 44 = 440
100 * 459 = 45900
respectively.
In practice it will be ok since we don't expect too many paths
or the shortage to be too large.
4. Use increase_flows() to top up the largest-capacity flows if we hit maxparts.
I strongly believe that reduce_num_flows shoudl sort flows based on delivery amount
and keep the largest delivery amount first.
2 participants
This is the algorithm recommended by @Lagrang3 . Much is neatening and tweaking existing code, though.