api: add hole punching info and data transfer stats

[ivan4th]

Motivation

Current AdminService.PeerInfoStream doesn't provide enough information about whether hole punching was used while connecting to a particular peer, and also doesn't reflect actual utilization of each peer, which makes it harder to see whether the peer connections are being used efficiently.
Also, importantly, in case if go-spacemesh causes network overload it may be hard to find out what kind of traffic is causing the problems. It would be nice to have some overview of the traffic even in case if the metrics are disabled. The current data transfer rates should be easy to view even without any Prometheus to do rate calculations.

Description

AdminService.PeerInfoStream response is extended with new fields.
This includes connection kind (relay / hole punching information, with HPInbound and HPOutbound meaning inbound and out-bound hole-punching, respectively) and client/server request and transfer statistics. Also, it includes the data transfer rates (bytes/s) calculated over 10s and 1m periods.
The peer records look like this (not using actual IPs or peer IDs):

{
  "id": "...",
  "connections": [
    {
      "address": "/ip4/123.45.67.89/tcp/15816",
      "uptime": "1336.521022291s",
      "outbound": true,
      "kind": "Outbound"
    }
  ],
  "clientStats": {
    "successCount": "1",
    "latency": "0.062496709s"
  },
  "bytesSent": "4635446",
  "bytesReceived": "29709306",
  "sendRate": [
    "0",
    "71"
  ],
  "recvRate": [
    "0",
    "193"
  ]
}
{
  "id": "...",
  "connections": [
    {
      "address": "/ip4/23.45.67.89/tcp/1024",
      "uptime": "449.661134625s",
      "outbound": true,
      "kind": "HPInbound"
    }
  ],
  "clientStats": {
    "successCount": "1",
    "latency": "0.070969208s"
  },
  "bytesSent": "6110842",
  "bytesReceived": "39615132",
  "sendRate": [
    "0",
    "257"
  ],
  "recvRate": [
    "0",
    "121"
  ]
}

The output of spacemesh.v1.DebugService.NetworkInfo now looks like:

{
  "id": "...",
  "listenAddresses": [
    "/ip4/0.0.0.0/tcp/7513",
    "/ip4/0.0.0.0/udp/7513/quic-v1",
    "/p2p-circuit"
  ],
  "knownAddresses": [
    "/ip4/127.0.0.1/tcp/7513",
    "/ip4/127.0.0.1/udp/7513/quic-v1",
    "/ip4/123.45.67.89/tcp/7513",
    "/ip4/192.168.40.159/tcp/7513",
    "/ip4/192.168.40.159/udp/7513/quic-v1",
    "/ip4/192.168.64.1/tcp/7513",
    "/ip4/192.168.64.1/udp/7513/quic-v1",
    "/ip4/...tcp/5001/p2p/....../p2p-circuit",
    "/ip4/.../tcp/5001/p2p/....../p2p-circuit"
  ],
  "natTypeTcp": "Cone",
  "reachability": "Private",
  "stats": {
    "/ipfs/id/1.0.0": {
      "bytesSent": "1702673",
      "bytesReceived": "1461222",
      "sendRate": [
        "0",
        "25"
      ],
      "recvRate": [
        "0",
        "18"
      ]
    },
    "/ipfs/id/push/1.0.0": {
      "bytesSent": "207189",
      "bytesReceived": "8977845",
      "sendRate": [
        "0",
        "237"
      ],
      "recvRate": [
        "0",
        "251"
      ]
    },
    "/libp2p/autonat/1.0.0": {
      "bytesSent": "7151",
      "bytesReceived": "3164"
    },
    "/libp2p/circuit/relay/0.2.0/hop": {
      "bytesSent": "2589492",
      "bytesReceived": "2461896"
    },
    "/libp2p/circuit/relay/0.2.0/stop": {
      "bytesSent": "88239",
      "bytesReceived": "85382",
      "sendRate": [
        "0",
        "5"
      ],
      "recvRate": [
        "0",
        "5"
      ]
    },
    "/libp2p/dcutr": {
      "bytesSent": "9139",
      "bytesReceived": "12671"
    },
    "/meshsub/1.1.0": {
      "bytesSent": "5265121597",
      "bytesReceived": "3980876238",
      "sendRate": [
        "456745",
        "185721"
      ],
      "recvRate": [
        "541102",
        "68351"
      ]
    },
    "/peersync/1.0/": {
      "bytesSent": "46385",
      "bytesReceived": "43979"
    },
    "/spacekad/kad/1.0.0": {
      "bytesSent": "644931",
      "bytesReceived": "15350810",
      "sendRate": [
        "0",
        "41"
      ],
      "recvRate": [
        "0",
        "1038"
      ]
    },
    "__other__": {
      "bytesSent": "2861993",
      "bytesReceived": "2857514",
      "sendRate": [
        "0",
        "33"
      ],
      "recvRate": [
        "0",
        "33"
      ]
    },
    "__total__": {
      "bytesSent": "5708199745",
      "bytesReceived": "4558225401",
      "sendRate": [
        "456745",
        "186550"
      ],
      "recvRate": [
        "541102",
        "69955"
      ]
    },
    "ax/1": {
      "bytesSent": "86797553",
      "bytesReceived": "520784404"
    },
    "hs/1": {
      "bytesSent": "338030798",
      "bytesReceived": "17237601",
      "sendRate": [
        "0",
        "251"
      ],
      "recvRate": [
        "0",
        "25"
      ]
    },
    "ld/1": {
      "bytesSent": "8693168",
      "bytesReceived": "6800320",
      "sendRate": [
        "0",
        "222"
      ],
      "recvRate": [
        "0",
        "222"
      ]
    },
    "lp/2": {
      "bytesSent": "320056",
      "bytesReceived": "256960",
      "sendRate": [
        "0",
        "8"
      ],
      "recvRate": [
        "0",
        "7"
      ]
    },
    "ml/1": {
      "bytesSent": "1079381",
      "bytesReceived": "1015395"
    }
  }
}

Test Plan

Tested on a mainnet node

TODO

• Explain motivation or link existing issue(s)
• Test changes and document test plan
• Merge Add extended peer info to AdminService.PeerInfoStream api#322 and update api dependency
• Update changelog as needed

[codecov]

Codecov Report

Attention: Patch coverage is 87.84195% with 40 lines in your changes are missing coverage. Please review.

Project coverage is 80.6%. Comparing base (4a3d808) to head (88ed2e8).
Report is 1 commits behind head on develop.

Files	Patch %	Lines
p2p/peerinfo/peerinfo.go	89.9%	11 Missing and 8 partials ⚠️
api/grpcserver/admin_service.go	82.2%	8 Missing ⚠️
p2p/upgrade.go	74.0%	6 Missing and 1 partial ⚠️
p2p/server/server.go	80.6%	5 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff            @@
##           develop   #5792     +/-   ##
=========================================
+ Coverage     80.5%   80.6%   +0.1%     
=========================================
  Files          286     287      +1     
  Lines        29537   29838    +301     
=========================================
+ Hits         23784   24066    +282     
- Misses        4155    4166     +11     
- Partials      1598    1606      +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[ivan4th]

bors try

[spacemesh-bors]

try

Build failed:

• systest-status

[ivan4th]

Partition test flake:

    partition_test.go:175: 
        	Error Trace:	/src/systest/tests/partition_test.go:175
        	            				/src/systest/tests/partition_test.go:204
        	Error:      	Should be true
        	Test:       	TestPartition_50_50

[ivan4th]

bors try

[spacemesh-bors]

try

Build succeeded:

• ci-status
• systest-status