So many orphaned blocks as found today have never been

[ilsawa]

Developers, during the day almost all found blocks is orphaned:

794315 XnwuwfPhuFANd9Bz8fPEt9G3wZSgJS9126 http://crypto.mine.nu:7903/static/classic/share.html#000000000000001056f9f50dd835621237b09c85ced27b452230ab1e4dac19b5
794249 XhstnrgjTeVKWddgPoZZJunAaAeran2QH7 http://crypto.mine.nu:7903/static/classic/share.html#000000000000002b4a0639ec9f3192d7581d18db6e1f5eadbfccaacbafc8fefd
794042 XcADs7dYBgZwp7GY4r25GV9Bzx6BdVj6DY http://crypto.mine.nu:7903/static/classic/share.html#00000000000000222f0af4f43ff173987fde9410fc5f9865f34a5b410ed0f92f
793988 XnwuwfPhuFANd9Bz8fPEt9G3wZSgJS9126 http://crypto.mine.nu:7903/static/classic/share.html#00000000000000251e04dc738858a29f214e24c061f9194bc4eea23e42fb86b5
793913 XhXMahooTgty7jLQAbbRLLHBWeJWy4F1Lv http://crypto.mine.nu:7903/static/classic/share.html#0000000000000000f012b91a259d3cd5af4bba44eacc02aa11bf9ca73c3f9c2b
Maybe we ran into the same problem as the jtoomim fork on the BTC p2pool? Large blocks were rejected.
jtoomim fixed this problem with this patches:
jtoomim/p2pool@b2f6613
jtoomim/p2pool@498bf97

[ilsawa]

Another orphan. This is uniquely associated with the transition to the latest version of the wallet

10 minutes ago | Thu Dec 28 2017 04:19:23 GMT+0600 | 794391 | 000000000000001760587fc7c94532b8148bd43fe447c711fc75f74439689bbc

[UdjinM6]

Orphan != rejected, all these blocks were relayed and seen by the network. You can check for yourself in ~/.dashcore/debug.log or via explorer that stores orphans e.g. http://insight.dash.org/insight/block/000000000000001056f9f50dd835621237b09c85ced27b452230ab1e4dac19b5
There is indeed a spike of orphans today but I think this is not p2pool related, see https://chainz.cryptoid.info/dash/orphans.dws - many of orphan blocks were mined not by p2pool.

[ilsawa]

I agree.

I know that at least one orphan was generated on the node with the old version of the daemon.
Can this cause so many orphans?

[UdjinM6]

I don't think so. Again, once the block is out there it has nothing to do with the node it was generated on. It's up to miners (pools) which chain to extend and for some reason they were choosing the later block more often today. Probably they just try to mine on both chain tips and the second one was "luckier" a lot more than usual. We'll see if this trend continues or is it just an anomaly in a few days I guess.

[ghost]

Hello!

I have found that since upgrade dashd to 0.12.2.2 (from 0.12.2.1) the number of inbound connections decreased dramatically. May be it is a cause of multiple orphan blocks?
Here is a chart of my dashd connections:

[UdjinM6]

@Alex3402 0.12.2.2 no longer accepts connections from old 70206 nodes.

2017-12-28 10:31:13 peer=55032 using obsolete version 70206; disconnecting
2017-12-28 10:31:13 ProcessMessages(version, 106 bytes) FAILED peer=55032

My nodes have 30+ connections. Maybe most of your known peers were old nodes?

Can someone else confirm inbound connections count drop to almost 0?

[chaeplin]

dash-cli getpeerinfo | grep inbound | sort -n | uniq -c
      9     "inbound": false,
     29     "inbound": true,

dash-cli getpeerinfo | grep inbound | sort -n | uniq -c
      9     "inbound": false,
     24     "inbound": true,

dash-cli getpeerinfo | grep inbound | sort -n | uniq -c
      9     "inbound": false,
     44     "inbound": true,

[ilsawa]

Server 1
Before update:

dash-cli getinfo
{
  "version": 120200,
  "protocolversion": 70208,
  "walletversion": 61000,
  "balance": 0.00000000,
  "privatesend_balance": 0.00000000,
  "blocks": 794366,
  "timeoffset": 0,
  "connections": 46,
  "proxy": "",
  "difficulty": 77273910.25391859,
  "testnet": false,
  "keypoololdest": 1413557305,
  "keypoolsize": 1001,
  "paytxfee": 0.00000000,
  "relayfee": 0.00001000,
  "errors": "Warning: unknown new rules activated (versionbit 1)"
}

After update (more than 10 hours of work):

dash-cli getpeerinfo | grep inbound | sort -n | uniq -c
     15     "inbound": false,
      1     "inbound": true,

Server 2
Before update:

dash-cli getinfo
{
  "version": 120201,
  "protocolversion": 70208,
  "walletversion": 61000,
  "balance": 0.00000000,
  "privatesend_balance": 0.00000000,
  "blocks": 794382,
  "timeoffset": 0,
  "connections": 70,
  "proxy": "",
  "difficulty": 83651633.484871,
  "testnet": false,
  "keypoololdest": 1509862047,
  "keypoolsize": 1001,
  "paytxfee": 0.00000000,
  "relayfee": 0.00001000,
  "errors": ""
}

After update (more than 10 hours of work):

dash-cli getpeerinfo | grep inbound | sort -n | uniq -c
      8     "inbound": false,
      8     "inbound": true,

[ilsawa]

To increase the connection, you can run the following command in the Linux terminal:

dash-cli addnode 82.200.205.30 onetry && \
dash-cli addnode 185.25.60.199 onetry && \
dash-cli addnode 109.87.19.9 onetry && \
dash-cli addnode 108.61.229.222 onetry && \
dash-cli addnode 164.132.200.123 onetry && \
dash-cli addnode 69.162.67.3 onetry && \
dash-cli addnode 76.102.55.12 onetry && \
dash-cli addnode 104.225.220.19 onetry && \
dash-cli addnode 146.66.207.128 onetry && \
dash-cli addnode 93.184.160.140 onetry && \
dash-cli addnode 45.32.154.91 onetry && \
dash-cli addnode 185.60.135.146 onetry && \
dash-cli addnode 91.197.223.88 onetry && \
dash-cli addnode 104.225.217.12 onetry && \
dash-cli addnode 193.169.129.29 onetry && \
dash-cli addnode 185.17.134.38 onetry && \
dash-cli addnode 213.174.7.18 onetry && \
dash-cli addnode 5.9.143.40 onetry && \
dash-cli addnode 95.213.242.194 onetry && \
dash-cli addnode 83.239.114.42 onetry && \
dash-cli addnode 213.170.103.10 onetry && \
dash-cli addnode 18.195.20.0 onetry && \
dash-cli addnode 178.64.174.58 onetry && \
dash-cli addnode 94.242.222.27 onetry && \
dash-cli addnode 27.68.131.10 onetry && \
dash-cli addnode 104.207.156.161 onetry && \
dash-cli addnode 43.229.77.46 onetry && \
dash-cli addnode 88.98.228.14 onetry && \
dash-cli addnode 222.185.244.242 onetry && \
dash-cli addnode 37.29.116.122 onetry && \
dash-cli addnode 46.191.226.84 onetry && \
dash-cli addnode 77.75.14.8 onetry && \
dash-cli addnode 83.143.51.50 onetry && \
dash-cli addnode 94.228.255.132 onetry && \
dash-cli addnode 46.188.44.20 onetry && \
dash-cli addnode 185.136.89.182 onetry && \
dash-cli addnode 62.152.54.44 onetry && \
dash-cli addnode 185.38.163.63 onetry && \
dash-cli addnode 31.131.23.68 onetry && \
dash-cli addnode 95.79.102.209 onetry && \
dash-cli addnode 163.172.216.135 onetry && \
dash-cli addnode 5.19.171.173 onetry && \
dash-cli addnode 194.67.204.96 onetry && \
dash-cli addnode 173.212.233.248 onetry && \
dash-cli addnode 82.117.166.77 onetry && \
dash-cli addnode 213.174.7.26 onetry && \
dash-cli addnode 178.238.236.130 onetry && \
dash-cli addnode 85.192.167.85 onetry && \
dash-cli addnode 83.136.86.133 onetry && \
dash-cli addnode 78.107.58.201 onetry && \
dash-cli addnode 89.223.27.250 onetry && \
dash-cli addnode 85.214.73.33 onetry && \
dash-cli addnode 78.94.185.115 onetry && \
dash-cli addnode 77.249.213.25 onetry && \
dash-cli addnode 90.188.4.86 onetry && \
dash-cli addnode 195.154.39.159 onetry && \
dash-cli addnode 80.64.104.164 onetry && \
dash-cli addnode 188.40.110.58 onetry && \
dash-cli addnode 176.56.24.155 onetry && \
dash-cli addnode 5.134.216.210 onetry && \
dash-cli addnode 78.46.177.112 onetry && \
dash-cli addnode 188.0.182.10 onetry && \
dash-cli addnode 193.108.117.202 onetry && \
dash-cli addnode 31.13.217.61 onetry && \
dash-cli addnode 95.47.137.96 onetry && \
dash-cli addnode 213.110.128.16 onetry && \
dash-cli addnode 185.49.69.160 onetry && \
dash-cli addnode 81.169.139.77 onetry && \
dash-cli addnode 82.199.145.145 onetry && \
dash-cli addnode 5.8.101.60 onetry && \
dash-cli addnode 63.135.175.51 onetry && \
dash-cli addnode 104.207.147.90 onetry && \
dash-cli addnode 62.210.69.84 onetry && \
dash-cli addnode 18.221.229.215 onetry && \
dash-cli addnode 89.212.142.123 onetry && \
dash-cli addnode 94.156.218.28 onetry && \
dash-cli addnode 198.91.88.177 onetry && \
dash-cli addnode 89.191.157.237 onetry && \
dash-cli addnode 85.238.102.203 onetry && \
dash-cli addnode 24.96.71.8 onetry && \
dash-cli addnode 198.40.48.24 onetry && \
dash-cli addnode 194.204.21.105 onetry && \
dash-cli addnode 45.63.7.168 onetry && \
dash-cli addnode 194.186.241.98 onetry && \
dash-cli addnode 81.169.157.154 onetry && \
dash-cli addnode 66.168.24.71 onetry && \
dash-cli addnode 212.58.112.196 onetry && \
dash-cli addnode 173.44.45.108 onetry && \
dash-cli addnode 122.11.58.176 onetry && \
dash-cli addnode 45.32.3.69 onetry && \
dash-cli addnode 186.31.33.251 onetry && \
dash-cli addnode 208.113.166.44 onetry && \
dash-cli addnode 128.199.76.158 onetry && \
dash-cli addnode 100.38.97.189 onetry && \
dash-cli addnode 123.207.101.189 onetry && \
dash-cli addnode 72.74.85.38 onetry && \
dash-cli addnode 97.77.64.18 onetry && \
dash-cli addnode 104.156.229.181 onetry && \
dash-cli addnode 96.231.104.39 onetry && \
dash-cli addnode 208.84.223.121 onetry && \
dash-cli addnode 222.185.244.244 onetry && \
dash-cli addnode 169.239.181.107 onetry && \
dash-cli addnode 154.0.172.35 onetry && \
dash-cli addnode 163.44.193.160 onetry && \
dash-cli addnode 104.238.141.83 onetry && \
dash-cli addnode 103.82.242.56 onetry && \
dash-cli addnode 69.27.173.228 onetry && \
dash-cli addnode 41.87.209.199 onetry && \
dash-cli addnode 175.138.41.27 onetry && \
dash-cli addnode 219.89.81.149 onetry && \
dash-cli addnode 52.67.66.249 onetry && \
dash-cli addnode 67.242.212.128 onetry && \
dash-cli addnode 101.100.174.138 onetry && \
dash-cli addnode 31.13.217.62 onetry && \
dash-cli addnode 84.95.250.20 onetry && \
dash-cli addnode 190.52.135.28 onetry && \
dash-cli addnode 104.207.156.59 onetry && \
dash-cli addnode 108.61.164.240 onetry && \
dash-cli addnode 111.252.241.74 onetry && \
dash-cli addnode 114.215.167.86 onetry && \
dash-cli addnode 123.180.19.182 onetry && \
dash-cli addnode 128.199.104.133 onetry && \
dash-cli addnode 128.199.50.108 onetry && \
dash-cli addnode 130.240.22.202 onetry && \
dash-cli addnode 137.74.17.9 onetry && \
dash-cli addnode 142.44.229.62 onetry && \
dash-cli addnode 145.255.172.241 onetry && \
dash-cli addnode 146.185.149.28 onetry && \
dash-cli addnode 149.56.66.239 onetry && \
dash-cli addnode 163.44.169.29 onetry && \
dash-cli addnode 172.245.62.23 onetry && \
dash-cli addnode 176.74.220.141 onetry && \
dash-cli addnode 178.62.236.77 onetry && \
dash-cli addnode 178.62.250.133 onetry && \
dash-cli addnode 178.62.252.187 onetry && \
dash-cli addnode 18.220.101.59 onetry && \
dash-cli addnode 18.220.87.231 onetry && \
dash-cli addnode 18.221.211.126 onetry && \
dash-cli addnode 18.221.217.223 onetry && \
dash-cli addnode 18.221.94.213 onetry && \
dash-cli addnode 18.221.94.98 onetry && \
dash-cli addnode 185.100.85.169 onetry && \
dash-cli addnode 185.106.121.28 onetry && \
dash-cli addnode 185.156.178.125 onetry && \
dash-cli addnode 185.156.178.216 onetry && \
dash-cli addnode 185.158.153.232 onetry && \
dash-cli addnode 185.217.1.100 onetry && \
dash-cli addnode 185.81.166.138 onetry && \
dash-cli addnode 185.81.167.118 onetry && \
dash-cli addnode 186.93.247.197 onetry && \
dash-cli addnode 188.166.121.39 onetry && \
dash-cli addnode 188.166.48.64 onetry && \
dash-cli addnode 188.166.54.223 onetry && \
dash-cli addnode 188.166.64.85 onetry && \
dash-cli addnode 188.213.173.171 onetry && \
dash-cli addnode 188.226.156.13 onetry && \
dash-cli addnode 188.226.164.49 onetry && \
dash-cli addnode 188.226.192.105 onetry && \
dash-cli addnode 188.227.19.66 onetry && \
dash-cli addnode 188.227.72.88 onetry && \
dash-cli addnode 188.227.74.139 onetry && \
dash-cli addnode 188.227.74.154 onetry && \
dash-cli addnode 188.227.75.148 onetry && \
dash-cli addnode 192.99.214.221 onetry && \
dash-cli addnode 194.135.82.238 onetry && \
dash-cli addnode 194.135.85.194 onetry && \
dash-cli addnode 194.135.89.16 onetry && \
dash-cli addnode 194.135.91.174 onetry && \
dash-cli addnode 197.87.109.127 onetry && \
dash-cli addnode 34.197.111.167 onetry && \
dash-cli addnode 37.139.0.155 onetry && \
dash-cli addnode 41.40.9.193 onetry && \
dash-cli addnode 45.32.200.85 onetry && \
dash-cli addnode 45.32.62.81 onetry && \
dash-cli addnode 45.63.116.62 onetry && \
dash-cli addnode 45.63.26.147 onetry && \
dash-cli addnode 45.63.68.85 onetry && \
dash-cli addnode 45.76.113.4 onetry && \
dash-cli addnode 45.76.146.246 onetry && \
dash-cli addnode 45.76.212.121 onetry && \
dash-cli addnode 45.76.32.195 onetry && \
dash-cli addnode 45.77.139.30 onetry && \
dash-cli addnode 45.77.92.202 onetry && \
dash-cli addnode 5.132.191.109 onetry && \
dash-cli addnode 5.135.104.32 onetry && \
dash-cli addnode 51.15.86.18 onetry && \
dash-cli addnode 52.59.235.63 onetry && \
dash-cli addnode 62.45.129.195 onetry && \
dash-cli addnode 69.90.186.12 onetry && \
dash-cli addnode 74.207.228.180 onetry && \
dash-cli addnode 76.182.75.127 onetry && \
dash-cli addnode 80.209.234.209 onetry && \
dash-cli addnode 82.211.25.64 onetry && \
dash-cli addnode 82.211.25.78 onetry && \
dash-cli addnode 91.138.73.96 onetry && \
dash-cli addnode 93.158.216.153 onetry && \
dash-cli addnode 94.176.237.1 onetry && \
dash-cli addnode 95.85.57.198 onetry

[oliverw]

@ilsawa If there was a way to filter the nodes on the "bad" segment perhaps we could stop the daemon, delete peers.dat and just add back the nodes on the "good" segment.

[ilsawa]

@oliverw I do not know the IP addresses of the demons of these pools, which get normal blocks, not orphans. If someone can find these IP addresses, I can try to do as you said

[UdjinM6]

The issue is not p2pool specific, see dashpay/dash#1822
Closing here.