rpc: support new p2p infrastructure (#6820)

CHANGELOG_PENDING.md

@@ -25,6 +25,7 @@ Friendly reminder: We have a [bug bounty program](https://hackerone.com/tendermi
   - [rpc/grpc] \#6725 Mark gRPC in the RPC layer as deprecated.
   - [blockchain/v2] \#6730 Fast Sync v2 is deprecated, please use v0
   - [rpc] Add genesis_chunked method to support paginated and parallel fetching of large genesis documents.
+  - [rpc] \#6820 Update RPC methods to reflect changes in the p2p layer, disabling support for `UnsafeDialPeers` and `UnsafeDialPeers` when used with the new p2p layer, and changing the response format of the peer list in `NetInfo` for all users.
 
 - Apps
   - [ABCI] \#6408 Change the `key` and `value` fields from `[]byte` to `string` in the `EventAttribute` type. (@alexanderbez)
@@ -33,7 +34,7 @@ Friendly reminder: We have a [bug bounty program](https://hackerone.com/tendermi
   - [ABCI] \#5818 Use protoio for msg length delimitation. Migrates from int64 to uint64 length delimiters.
   - [ABCI] \#3546 Add `mempool_error` field to `ResponseCheckTx`. This field will contain an error string if Tendermint encountered an error while adding a transaction to the mempool. (@williambanfield)
   - [Version] \#6494 `TMCoreSemVer` has been renamed to `TMVersion`.
-    - It is not required any longer to set ldflags to set version strings
+	- It is not required any longer to set ldflags to set version strings
   - [abci/counter] \#6684 Delete counter example app
 
 - P2P Protocol
@@ -56,25 +57,25 @@ Friendly reminder: We have a [bug bounty program](https://hackerone.com/tendermi
   - [store] \#5848 Remove block store state in favor of using the db iterators directly (@cmwaters)
   - [state] \#5864 Use an iterator when pruning state (@cmwaters)
   - [types] \#6023 Remove `tm2pb.Header`, `tm2pb.BlockID`, `tm2pb.PartSetHeader` and `tm2pb.NewValidatorUpdate`.
-    - Each of the above types has a `ToProto` and `FromProto` method or function which replaced this logic.
+	- Each of the above types has a `ToProto` and `FromProto` method or function which replaced this logic.
   - [light] \#6054 Move `MaxRetryAttempt` option from client to provider.
-    - `NewWithOptions` now sets the max retry attempts and timeouts (@cmwaters)
+	- `NewWithOptions` now sets the max retry attempts and timeouts (@cmwaters)
   - [all] \#6077 Change spelling from British English to American (@cmwaters)
-    - Rename "Subscription.Cancelled()" to "Subscription.Canceled()" in libs/pubsub
-    - Rename "behaviour" pkg to "behavior" and internalized it in blockchain v2
+	- Rename "Subscription.Cancelled()" to "Subscription.Canceled()" in libs/pubsub
+	- Rename "behaviour" pkg to "behavior" and internalized it in blockchain v2
   - [rpc/client/http] \#6176 Remove `endpoint` arg from `New`, `NewWithTimeout` and `NewWithClient` (@melekes)
   - [rpc/client/http] \#6176 Unexpose `WSEvents` (@melekes)
   - [rpc/jsonrpc/client/ws_client] \#6176 `NewWS` no longer accepts options (use `NewWSWithOptions` and `OnReconnect` funcs to configure the client) (@melekes)
   - [internal/libs] \#6366 Move `autofile`, `clist`,`fail`,`flowrate`, `protoio`, `sync`, `tempfile`, `test` and `timer` lib packages to an internal folder
   - [libs/rand] \#6364 Remove most of libs/rand in favour of standard lib's `math/rand` (@liamsi)
   - [mempool] \#6466 The original mempool reactor has been versioned as `v0` and moved to a sub-package under the root `mempool` package.
-    Some core types have been kept in the `mempool` package such as `TxCache` and it's implementations, the `Mempool` interface itself
-    and `TxInfo`. (@alexanderbez)
+	Some core types have been kept in the `mempool` package such as `TxCache` and it's implementations, the `Mempool` interface itself
+	and `TxInfo`. (@alexanderbez)
   - [crypto/sr25519] \#6526 Do not re-execute the Ed25519-style key derivation step when doing signing and verification.  The derivation is now done once and only once.  This breaks `sr25519.GenPrivKeyFromSecret` output compatibility. (@Yawning)
-  - [types] \#6627 Move `NodeKey` to types to make the type public. 
+  - [types] \#6627 Move `NodeKey` to types to make the type public.
   - [config] \#6627 Extend `config` to contain methods `LoadNodeKeyID` and `LoadorGenNodeKeyID`
   - [blocksync] \#6755 Rename `FastSync` and `Blockchain` package to `BlockSync`
-    (@cmwaters)
+	(@cmwaters)
 
 - Blockchain Protocol
 

UPGRADING.md

@@ -100,8 +100,22 @@ will need to change to accommodate these changes. Most notably:
 
 ### RPC changes
 
+#### gRPC Support
+
 Mark gRPC in the RPC layer as deprecated and to be removed in 0.36.
 
+#### Peer Management Interface
+
+When running with the new P2P Layer, the methods `UnsafeDialSeeds` and
+`UnsafeDialPeers` RPC methods will always return an error. They are
+deprecated and will be removed in 0.36 when the legacy peer stack is
+removed.
+
+Additionally the format of the Peer list returned in the `NetInfo`
+method changes in this release to accommodate the different way that
+the new stack tracks data about peers. This change affects users of
+both stacks.
+
 ### Support for Custom Reactor and Mempool Implementations
 
 The changes to p2p layer removed existing support for custom
@@ -110,7 +124,7 @@ used, the introduction of the prioritized mempool covers nearly all of
 the use cases for custom reactors. If you are currently running custom
 reactors and mempools and are having trouble seeing the migration path
 for your project please feel free to reach out to the Tendermint Core
-development team directly. 
+development team directly.
 
 ## v0.34.0
 

crypto/secp256k1/secp256k1.go

@@ -13,7 +13,7 @@ import (
 	tmjson "github.com/tendermint/tendermint/libs/json"
 
 	// necessary for Bitcoin address format
-	"golang.org/x/crypto/ripemd160" // nolint: staticcheck
+	"golang.org/x/crypto/ripemd160" // nolint
 )
 
 //-------------------------------------

node/node.go

@@ -445,9 +445,11 @@ func makeNode(config *cfg.Config,
 			BlockStore:       blockStore,
 			EvidencePool:     evPool,
 			ConsensusState:   csState,
-			P2PPeers:         sw,
 			BlockSyncReactor: bcReactor.(cs.BlockSyncReactor),
 
+			P2PPeers:    sw,
+			PeerManager: peerManager,
+
 			GenDoc:           genDoc,
 			EventSinks:       eventSinks,
 			ConsensusReactor: csReactor,

rpc/core/consensus.go

@@ -1,6 +1,8 @@
 package core
 
 import (
+	"errors"
+
 	cm "github.com/tendermint/tendermint/internal/consensus"
 	tmmath "github.com/tendermint/tendermint/libs/math"
 	ctypes "github.com/tendermint/tendermint/rpc/core/types"
@@ -54,24 +56,56 @@ func (env *Environment) Validators(
 // More: https://docs.tendermint.com/master/rpc/#/Info/dump_consensus_state
 func (env *Environment) DumpConsensusState(ctx *rpctypes.Context) (*ctypes.ResultDumpConsensusState, error) {
 	// Get Peer consensus states.
-	peers := env.P2PPeers.Peers().List()
-	peerStates := make([]ctypes.PeerStateInfo, len(peers))
-	for i, peer := range peers {
-		peerState, ok := peer.Get(types.PeerStateKey).(*cm.PeerState)
-		if !ok { // peer does not have a state yet
-			continue
-		}
-		peerStateJSON, err := peerState.ToJSON()
-		if err != nil {
-			return nil, err
+
+	var peerStates []ctypes.PeerStateInfo
+	switch {
+	case env.P2PPeers != nil:
+		peers := env.P2PPeers.Peers().List()
+		peerStates = make([]ctypes.PeerStateInfo, 0, len(peers))
+		for _, peer := range peers {
+			peerState, ok := peer.Get(types.PeerStateKey).(*cm.PeerState)
+			if !ok { // peer does not have a state yet
+				continue
+			}
+			peerStateJSON, err := peerState.ToJSON()
+			if err != nil {
+				return nil, err
+			}
+			peerStates = append(peerStates, ctypes.PeerStateInfo{
+				// Peer basic info.
+				NodeAddress: peer.SocketAddr().String(),
+				// Peer consensus state.
+				PeerState: peerStateJSON,
+			})
 		}
-		peerStates[i] = ctypes.PeerStateInfo{
-			// Peer basic info.
-			NodeAddress: peer.SocketAddr().String(),
-			// Peer consensus state.
-			PeerState: peerStateJSON,
+	case env.PeerManager != nil:
+		peers := env.PeerManager.Peers()
+		peerStates = make([]ctypes.PeerStateInfo, 0, len(peers))
+		for _, pid := range peers {
+			peerState, ok := env.ConsensusReactor.GetPeerState(pid)
+			if !ok {
+				continue
+			}
+
+			peerStateJSON, err := peerState.ToJSON()
+			if err != nil {
+				return nil, err
+			}
+
+			addr := env.PeerManager.Addresses(pid)
+			if len(addr) >= 1 {
+				peerStates = append(peerStates, ctypes.PeerStateInfo{
+					// Peer basic info.
+					NodeAddress: addr[0].String(),
+					// Peer consensus state.
+					PeerState: peerStateJSON,
+				})
+			}
 		}
+	default:
+		return nil, errors.New("no peer system configured")
 	}
+
 	// Get self round state.
 	roundState, err := env.ConsensusState.GetRoundStateJSON()
 	if err != nil {

rpc/core/env.go

@@ -36,7 +36,7 @@ const (
 //----------------------------------------------
 // These interfaces are used by RPC and must be thread safe
 
-type Consensus interface {
+type consensusState interface {
 	GetState() sm.State
 	GetValidators() (int64, []*types.Validator)
 	GetLastHeight() int64
@@ -58,6 +58,11 @@ type peers interface {
 	Peers() p2p.IPeerSet
 }
 
+type peerManager interface {
+	Peers() []types.NodeID
+	Addresses(types.NodeID) []p2p.NodeAddress
+}
+
 //----------------------------------------------
 // Environment contains objects and interfaces used by the RPC. It is expected
 // to be setup once during startup.
@@ -70,9 +75,14 @@ type Environment struct {
 	StateStore     sm.Store
 	BlockStore     sm.BlockStore
 	EvidencePool   sm.EvidencePool
-	ConsensusState Consensus
+	ConsensusState consensusState
 	P2PPeers       peers
-	P2PTransport   transport
+
+	// Legacy p2p stack
+	P2PTransport transport
+
+	// interfaces for new p2p interfaces
+	PeerManager peerManager
 
 	// objects
 	PubKey           crypto.PubKey

rpc/core/net.go

@@ -13,19 +13,35 @@ import (
 // NetInfo returns network info.
 // More: https://docs.tendermint.com/master/rpc/#/Info/net_info
 func (env *Environment) NetInfo(ctx *rpctypes.Context) (*ctypes.ResultNetInfo, error) {
-	peersList := env.P2PPeers.Peers().List()
-	peers := make([]ctypes.Peer, 0, len(peersList))
-	for _, peer := range peersList {
-		peers = append(peers, ctypes.Peer{
-			NodeInfo:         peer.NodeInfo(),
-			IsOutbound:       peer.IsOutbound(),
-			ConnectionStatus: peer.Status(),
-			RemoteIP:         peer.RemoteIP().String(),
-		})
-	}
-	// TODO: Should we include PersistentPeers and Seeds in here?
-	// PRO: useful info
-	// CON: privacy
+	var peers []ctypes.Peer
+
+	switch {
+	case env.P2PPeers != nil:
+		peersList := env.P2PPeers.Peers().List()
+		peers = make([]ctypes.Peer, 0, len(peersList))
+		for _, peer := range peersList {
+			peers = append(peers, ctypes.Peer{
+				ID:  peer.ID(),
+				URL: peer.SocketAddr().String(),
+			})
+		}
+	case env.PeerManager != nil:
+		peerList := env.PeerManager.Peers()
+		for _, peer := range peerList {
+			addrs := env.PeerManager.Addresses(peer)
+			if len(addrs) == 0 {
+				continue
+			}
+
+			peers = append(peers, ctypes.Peer{
+				ID:  peer,
+				URL: addrs[0].String(),
+			})
+		}
+	default:
+		return nil, errors.New("peer management system does not support NetInfo responses")
+	}
+
 	return &ctypes.ResultNetInfo{
 		Listening: env.P2PTransport.IsListening(),
 		Listeners: env.P2PTransport.Listeners(),

rpc/core/types/responses.go

@@ -7,7 +7,6 @@ import (
 
 	abci "github.com/tendermint/tendermint/abci/types"
 	"github.com/tendermint/tendermint/crypto"
-	"github.com/tendermint/tendermint/internal/p2p"
 	"github.com/tendermint/tendermint/libs/bytes"
 	tmproto "github.com/tendermint/tendermint/proto/tendermint/types"
 	"github.com/tendermint/tendermint/types"
@@ -145,10 +144,8 @@ type ResultDialPeers struct {
 
 // A peer
 type Peer struct {
-	NodeInfo         types.NodeInfo       `json:"node_info"`
-	IsOutbound       bool                 `json:"is_outbound"`
-	ConnectionStatus p2p.ConnectionStatus `json:"connection_status"`
-	RemoteIP         string               `json:"remote_ip"`
+	ID  types.NodeID `json:"node_id"`
+	URL string       `json:"url"`
 }
 
 // Validators for a height.

rpc/openapi/openapi.yaml

@@ -1476,16 +1476,12 @@ components:
     Peer:
       type: object
       properties:
-        node_info:
-          $ref: "#/components/schemas/NodeInfo"
-        is_outbound:
-          type: boolean
-          example: true
-        connection_status:
-          $ref: "#/components/schemas/ConnectionStatus"
-        remote_ip:
+        node_id:
+          type: string
+          example: ""
+        url:
           type: string
-          example: "95.179.155.35"
+          example: "<id>@95.179.155.35:2385>"
     NetInfo:
       type: object
       properties:

test/e2e/tests/net_test.go

@@ -32,11 +32,12 @@ func TestNet_Peers(t *testing.T) {
 			seen[n.Name] = (n.Name == node.Name) // we've clearly seen ourself
 		}
 		for _, peerInfo := range netInfo.Peers {
-			peer := node.Testnet.LookupNode(peerInfo.NodeInfo.Moniker)
-			require.NotNil(t, peer, "unknown node %v", peerInfo.NodeInfo.Moniker)
-			require.Equal(t, peer.IP.String(), peerInfo.RemoteIP,
-				"unexpected IP address for peer %v", peer.Name)
-			seen[peerInfo.NodeInfo.Moniker] = true
+			id := peerInfo.ID
+			peer := node.Testnet.LookupNode(string(id))
+			require.NotNil(t, peer, "unknown node %v", id)
+			require.Contains(t, peerInfo.URL, peer.IP.String(),
+				"unexpected IP address for peer %v", id)
+			seen[string(id)] = true
 		}
 
 		for name := range seen {