Cannot complete test payment in controlled environment

[joostjager]

We are experimenting with a reproducible pathfinding benchmark setup: https://github.com/bottlepay/pathfinding-benchmark. The basic idea is to spin up a set of nodes, open channels according to a predefined list and execute a number of test payments.

All nodes are lnd-based, but for the test node (called node_start in the docker environment) a choice can be made between lnd and cln.

With lnd, all four test payments are completed successfully. With cln however, the payment process fails at the first payment:

testrunner_1                | 2022-05-16T08:17:54.036Z	INFO	Sending payment	{"invoice": "lnbcrt250u1p3gyz9jpp5evn7vnh94acway3ksazm66qey683vakzava0a8krlfstkmjsk4kqdqqcqzpgxqyz5vqsp5d8fxnw8cldmzmcveenyxm3r70c7s5rg704eztupn7gdnvdagrres9qyyssqw8lhj286ajm0w6zlnkwdn4k2gsqukcjucdnu4y57zq289j2up845lqy5pjlrs7zmmnraz0hwt4k2fyeq845rjq235x528n88d8u2whgplt5h4l"}
testrunner_1                | 2022-05-16T08:19:55.041Z	ERROR	Exiting	{"err": "210:Ran out of routes to try after 252 attempts: see `paystatus`"}

Repro

• Checkout https://github.com/bottlepay/pathfinding-benchmark/tree/cln-5262
• Run run.sh cln

Gossip is a bit flakey, so it may take a few runs to get the test node fully synced. Only after that, the test payments are kicked off.

[joostjager]

@vincenzopalazzo this is the issue that describes a problem with completing the payments. It will also show the inconsistencies in gossip syncing.

[joostjager]

The benchmark master branch (currently https://github.com/bottlepay/pathfinding-benchmark/tree/4b0347432bc5f24266a2f0de119256f505a5311f) can be used too.

[vincenzopalazzo]

before jump into it I want ping our gossip guy @endothermicdev because he add a new feature to "feature" the network with gossip.

Maybe we need to activate this feature to make the gossip more reliable?