monerod: incoming peers unreasonably low

[li5lo]

Running monero mainnet, stagenet, testnet and wownero mainnet on the same hardware and IP address.

Since months i have a very low count of in peers (5 at most and sometimes 0 over many days).
It started on testnet and stagenet long before they forked and since the recent hardfork mainnet is affected too.

Sometimes the in peers suddenly go up to 50-150 (mainnet) and are back close to zero a few hours later.

Changing IP address doesn't help and switching monerod off for a couple of days doesn't help either.

wownerod seems to be unaffected so i don't think it's related to my hardware or internet and Monero CLI was downloaded and hash verified from the official website.

I watched the peerlist size behavior on wownerod and it looks healthy how the greylist decreases and the whitelist increases until a new chunk is added to the greylist but with monerod the amount of peers in both lists seems to stay almost the same which could indicate a problem with greylist housekeeping in monerod but i don't know how to investigate further.

[selsta]

Since which version do you have this issue?

I run multiple nodes and so far did not notice this behavior.

[li5lo]

Since which version do you have this issue?

I don't know anymore which version.

I wrote this post 9 months ago and i already had the problem at least 4 months earlier.
It could be even earlier because i was not checking peerlist sizes back then.

I got inspired to finally write this bug report because it seems i might not be alone anymore.

[li5lo]

Wait, what i wrote in the post 9 months ago is not the same problem...

[li5lo]

9 months ago apparently i had the problem to find outgoing peers, now i have a problem to find incoming peers.

[selsta]

Which OS do you have? 4 of my Linux nodes are okay, my macOS node can reproduce the issue.

[li5lo]

Which OS do you have? 4 of my Linux nodes are okay, my macOS node can reproduce the issue.

Ubuntu 22.04

[selsta]

When did this issue started showing up for you? Also are the out peers unaffected? Can you set them higher than default?

[selsta]

Can you run your daemon with log level 2 for a couple minutes and then upload logs? Just trying to confirm if you have the same error.

[li5lo]

When did this issue started showing up for you?

The issue that i don't have enough outgoing connections apparently resolved itself somewhere between 9 months ago and some time ago and i recall incoming and outgoing were normal for some time and i stopped to frequently check them.

The issue that i don't have enough incoming connections started recently (maybe after the hardfork on each of the chains but i am not sure because i just learned that the chains forked at different blocks. Some day just saw 0 connections and thought it's the same problem like before so i don't remember when it exactly started on testnet and stagenet.)

But i was checking my mainnet node peers during the hardfork and at the forkblock the incoming connections dropped significantly, recovered, then stayed at the low level it is now.

Also are the out peers unaffected?

Almost.

Mainnet: 12 (out) 1 (in)
Testnet: 12 (out) 0 (in)
Stagenet: 5 (out) 0 (in)

Can you set them higher than default?

I can :)
Should i? To what number?

[selsta]

I'm more interested in log level 2, you don't have to change out peers. It is not really relevant.

Hi I'm u/MPodom352 from reddit. Thanks for posting the link here.
Here's my monerod log, at log level 2
monerod.log
I ran status before killing it. There was only 12 outgoing peers and 1 incoming peer.
I synced the node before running it and thought it would help filtering "useless" sync info but connection status. Let me know if I did it wrong.
Here's how I ran monerod
./monerod --zmq-pub tcp://127.0.0.1:18083 --disable-dns-checkpoints --enable-dns-blocklist --log-level 2 --log-file /home/pc/Downloads/monerod.log
I'm running monero-gui v0.18.1 release | Fedora 36 without firewall

[selsta]

I don't see anything interesting in your log, but I only skimmed over it. Let's see if we can also get a log from @computerfreak94

[selsta]

@AsBenDoge The issue is that 3 minutes uptime is likely not enough for this issue to show up, that's why I don't see anything in your log.

monerod.log
Here's the log of monerod running for ~17 minutes
I added --out-peers 50 --in-peers 50 to the command this time. Status show 50(out)+1(in) before killed.

[selsta]

I still don't see anything interesting, will have to let someone else look. @AsBenDoge if you completely restart your computer, do you still not get more than 1 incoming connection?

At this point I'm unsure if the issue I have on my macOS node is related to this issue here. This is my macOS node after a complete restart:

Height: 2693377/2693377 (100.0%) on mainnet, not mining, net hash 2.37 GH/s, v16, 12(out)+54(in) connections, uptime 0d 1h 9m 50s