bug: failed to discover more P2P nodes

[brilliant-lx]

System information

Geth version: v1.3.1
OS & Version: Windows/Linux/OSX
Commit hash : 4493ab8

Expected behaviour

Restarted a node to sync. On restart, it can find useable peers and able to sync for about half an hour, after that it can not sync blocks, for detail pls the attach log file

command line:

bsc --http.api eth,web3,debug --pprof --pprof.addr 0.0.0.0 --metrics --txlookuplimit 0 --datadir ./node --config ./config.toml

config.toml:

[Eth]
NetworkId = 56
NoPruning = false
NoPrefetch = false
LightPeers = 100
TrieTimeout = 1000000000000
EnablePreimageRecording = false
EWASMInterpreter = ""
EVMInterpreter = ""

[Eth.Miner]
GasFloor = 30000000
GasCeil = 40000000
GasPrice = 1000000000
Recommit = 10000000000

[Eth.TxPool]
Locals = []
NoLocals = true
Journal = "transactions.rlp"
Rejournal = 3600000000000
PriceLimit = 1000000000
PriceBump = 10
AccountSlots = 512
GlobalSlots = 10000
AccountQueue = 256
GlobalQueue = 5000
Lifetime = 10800000000000

[Eth.GPO]
Blocks = 20
Percentile = 60
OracleThreshold = 20

[Node]
IPCPath = "geth.ipc"
HTTPHost = "localhost"
NoUSB = true
InsecureUnlockAllowed = false
HTTPPort = 8545
HTTPVirtualHosts = ["localhost"]
HTTPModules = ["eth", "net", "web3", "txpool", "parlia"]
WSPort = 8546
WSModules = ["net", "web3", "eth"]

[Node.P2P]
MaxPeers = 30
NoDiscovery = false
StaticNodes = []
ListenAddr = ":30311"
EnableMsgEvents = false

[Node.HTTPTimeouts]
ReadTimeout = 30000000000
WriteTimeout = 30000000000
IdleTimeout = 120000000000

[Node.LogConfig]
FilePath = "bsc.log"
MaxBytesSize = 10485760
Level = "info"
FileRoot = ""

attach to the geth.ipc and dump the connected peers:

> admin.peers
[{
    caps: ["eth/66"],
    enode: "enode://e5c51ddeca05bd0b01e1cbb5db9aa7b9751a2da51e6b45daee1295b9d3c5b6383c0ae00e2b78ca462e69a6e2aa34119d4c62ca2a4c6b8d18cc3706e799916945@74.201.193.217:56393",
    id: "0e08befb7fcaf704858393a6af1c5352145ec7f92d63a2e74cdd4b602798d7c9",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:33668",
      remoteAddress: "74.201.193.217:56393",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://af2917fe6a3422a503f1524f413f745ef58c710c141e24a3a66da33bee1185e9329752d2fd14cc34efcb4c3474170b5722fba17d801bd2bff9d151ce508c9c3f@72.251.237.16:44184",
    id: "0e98a6183335aa3eb754c0072c13b1821aefafba6c46fb876bd9ec9f391732c2",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:53874",
      remoteAddress: "72.251.237.16:44184",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://c6ae17ba15985bc97fa820afc0cdf37f7496ad959d162e102a95cf058d8d99a7619ceafa6bc49167ff7e942495e78b425a017690b1aef6cf4768f396832e5b84@72.251.237.24:55704",
    id: "954ad2ef41febcf41d089a763950aa0a13f29b253a13c69daa88bf51a4826488",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:45996",
      remoteAddress: "72.251.237.24:55704",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://bc433623901812ac2fb41ef21322d2bee0a74681a64c9bfa3246634fc45ba2062925b402ce44bc70cb9ba32b3e37abac0f0b59569f985b09e8bbe4b9551389c8@64.46.105.38:47570",
    id: "9569cd3415cb69f31100c97efed7e681ef3e8a209426ece71dcf06bbb49ac9b1",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:54326",
      remoteAddress: "64.46.105.38:47570",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://ad86ca66c95da02741ab2b8ad38effbb5af26fffc0bae3cfb5c739bebbdc2b5071890a62d35ecef657608473e2fa044835f952005d95ec56b2b39d861438e0ba@74.201.193.214:32091",
    id: "9576fd24a17ee917c2188b6514fd15c40e419629bbc51098e305c9d549c16e2d",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:34214",
      remoteAddress: "74.201.193.214:32091",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://cd35fc1af250217fafa30867a9daab0f04ea976b7e18b4b0ef374779ede781402ef590b87743c878e4769e3e5e8e03e1dcb6b3ff176c3e213e13802cac2462b1@168.119.68.44:30403",
    id: "a2e75f53b8fa3a5088dd1cba9daa49cc7cea499427e6e852238e75ef3b9cbf96",
    name: "erigon/v2.36.0-dev-ee7fd418/linux-amd64/go1.19.4",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:41056",
      remoteAddress: "168.119.68.44:30403",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://f5cf4f7781a9898defc335822ebf1d12875d950104a814b940cb401d1245ad9175c26e04c112a2defc222ca687dd6ee69025f384c65882946e29457aa4a3a03b@74.201.193.169:50384",
    id: "aa7212845c1347207192a3ba51ab514a1edeaa05f8414f50e2632515a8615044",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:52100",
      remoteAddress: "74.201.193.169:50384",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://739a609c51e1a02e0fca3d923065872e8cba8a68e539f4296a2373f0f928cdfcaf43be098374f4e0e2939b13474be215d8517a72a50d7746591a695187df0caa@195.201.161.149:30400",
    id: "aab491eb6cc219d90d7db2d61db1d94aeff9a1e3288fc264e391e970252ff69b",
    name: "erigon/v2.36.0-dev-ee7fd418/linux-amd64/go1.19.4",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:45512",
      remoteAddress: "195.201.161.149:30400",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://b559ec3bc65ab3f05d3f877b07058fbd6f1af51b388cbbf0d85e314452f8a43ba64aea03548efdb9318b8228b97488a1a368efb0294fb6f1d6e8acea6ede043a@74.201.193.132:44669",
    enr: "enr:-Je4QBoDccNqp4V1-0EL9zsoTEzj_16UJDhcEkvwEsOSYoztL0bcNVs4OMEcyM9Wl_uRvbriONS3Rsoq_FU3AjXj-HMTg2V0aMfGhLGd9KKAgmlkgnY0gmlwhErJwYSJc2VjcDI1NmsxoQK1Wew7xlqz8F0_h3sHBY-9bxr1GziMu_DYXjFEUvikO4N0Y3CCrn2DdWRwgq59",
    id: "cf9a51c31717bfecce66209c389c5ac08b17812d577d6aab75efe6e2c85d74a7",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:50060",
      remoteAddress: "74.201.193.132:44669",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}, {
    caps: ["eth/66"],
    enode: "enode://302e63dfb4f550753c8a64ca21d2bae8e807b105fc42fac26de1340359446b520fa47f227387104d18306e5f1756d5ebd48b42ad01398c83894a333220b77519@72.251.237.32:54213",
    id: "db31b28b11600d117aec090f1f7aa4978c2b445dbac29ee140c86000e531adbe",
    name: "erigon/v2.40.0-dev-3da15fcb/linux-amd64/go1.19.7",
    network: {
      inbound: false,
      localAddress: "172.22.42.227:41946",
      remoteAddress: "72.251.237.32:54213",
      static: false,
      trusted: false
    },
    protocols: {
      eth: {
        version: 66
      }
    }
}]

Actual behaviour

Failed to sync.
But if I remove the folder: "<datadir>/geth/nodes", and restart, then the node can find useable peers and sync.

Steps to reproduce the behaviour

restart a node several times, sometime it could be reproduced.

Backtrace

[backtrace]
[bsc.log](https://github.com/bnb-chain/bsc/files/13162178/bsc.log)

When submitting logs: please submit them as text and not screenshots.

[du5]

same

[brilliant-lx]

close, will be tracked in: #1947