adr-012-peer-transport.md

ADR 012: PeerTransport

Context

One of the more apparent problems with the current architecture in the p2p package is that there is no clear separation of concerns between different components. Most notably the Switch is currently doing physical connection handling. An artifact is the dependency of the Switch on [config.P2PConfig](

tendermint/config/config.go

Lines 272 to 339 in 05a76fb

	// P2PConfig defines the configuration options for the Tendermint peer-to-peer networking layer
	type P2PConfig struct {
	RootDir string `mapstructure:"home"`
	// Address to listen for incoming connections
	ListenAddress string `mapstructure:"laddr"`
	// Address to advertise to peers for them to dial
	ExternalAddress string `mapstructure:"external_address"`
	// Comma separated list of seed nodes to connect to
	// We only use these if we can’t connect to peers in the addrbook
	Seeds string `mapstructure:"seeds"`
	// Comma separated list of nodes to keep persistent connections to
	PersistentPeers string `mapstructure:"persistent_peers"`
	// UPNP port forwarding
	UPNP bool `mapstructure:"upnp"`
	// Path to address book
	AddrBook string `mapstructure:"addr_book_file"`
	// Set true for strict address routability rules
	AddrBookStrict bool `mapstructure:"addr_book_strict"`
	// Maximum number of peers to connect to
	MaxNumPeers int `mapstructure:"max_num_peers"`
	// Time to wait before flushing messages out on the connection, in ms
	FlushThrottleTimeout int `mapstructure:"flush_throttle_timeout"`
	// Maximum size of a message packet payload, in bytes
	MaxPacketMsgPayloadSize int `mapstructure:"max_packet_msg_payload_size"`
	// Rate at which packets can be sent, in bytes/second
	SendRate int64 `mapstructure:"send_rate"`
	// Rate at which packets can be received, in bytes/second
	RecvRate int64 `mapstructure:"recv_rate"`
	// Set true to enable the peer-exchange reactor
	PexReactor bool `mapstructure:"pex"`
	// Seed mode, in which node constantly crawls the network and looks for
	// peers. If another node asks it for addresses, it responds and disconnects.
	//
	// Does not work if the peer-exchange reactor is disabled.
	SeedMode bool `mapstructure:"seed_mode"`
	// Comma separated list of peer IDs to keep private (will not be gossiped to
	// other peers)
	PrivatePeerIDs string `mapstructure:"private_peer_ids"`
	// Toggle to disable guard against peers connecting from the same ip.
	AllowDuplicateIP bool `mapstructure:"allow_duplicate_ip"`
	// Peer connection configuration.
	HandshakeTimeout time.Duration `mapstructure:"handshake_timeout"`
	DialTimeout time.Duration `mapstructure:"dial_timeout"`
	// Testing params.
	// Force dial to fail
	TestDialFail bool `mapstructure:"test_dial_fail"`
	// FUzz connection
	TestFuzz bool `mapstructure:"test_fuzz"`
	TestFuzzConfig *FuzzConnConfig `mapstructure:"test_fuzz_config"`
	}

).

Addresses:

• #2046
• #2047

First iteraton in #2067

Decision

Transport concerns will be handled by a new component (PeerTransport) which will provide Peers at its boundary to the caller. In turn Switch will use this new component accept new Peers and dial them based on NetAddress.

PeerTransport

Responsible for emitting and connecting to Peers. The implementation of Peer is left to the transport, which implies that the chosen transport dictates the characteristics of the implementation handed back to the Switch. Each transport implementation is responsible to filter establishing peers specific to its domain, for the default multiplexed implementation the following will apply:

• connections from our own node
• handshake fails
• upgrade to secret connection fails
• prevent duplicate ip
• prevent duplicate id
• nodeinfo incompatibility

// PeerTransport proxies incoming and outgoing peer connections.
type PeerTransport interface {
	// Accept returns a newly connected Peer.
	Accept() (Peer, error)

	// Dial connects to a Peer.
	Dial(NetAddress) (Peer, error)
}

// EXAMPLE OF DEFAULT IMPLEMENTATION

// multiplexTransport accepts tcp connections and upgrades to multiplexted
// peers.
type multiplexTransport struct {
	listener net.Listener

	acceptc chan accept
	closec  <-chan struct{}
	listenc <-chan struct{}

	dialTimeout      time.Duration
	handshakeTimeout time.Duration
	nodeAddr         NetAddress
	nodeInfo         NodeInfo
	nodeKey          NodeKey

	// TODO(xla): Remove when MConnection is refactored into mPeer.
	mConfig conn.MConnConfig
}

var _ PeerTransport = (*multiplexTransport)(nil)

// NewMTransport returns network connected multiplexed peers.
func NewMTransport(
	nodeAddr NetAddress,
	nodeInfo NodeInfo,
	nodeKey NodeKey,
) *multiplexTransport

Switch

From now the Switch will depend on a fully setup PeerTransport to retrieve/reach out to its peers. As the more low-level concerns are pushed to the transport, we can omit passing the config.P2PConfig to the Switch.

func NewSwitch(transport PeerTransport, opts ...SwitchOption) *Switch

Status

In Review.

Consequences

Positive

• free Switch from transport concerns - simpler implementation
• pluggable transport implementation - simpler test setup
• remove Switch dependency on P2PConfig - easier to test

Negative

• more setup for tests which depend on Switches

Neutral

• multiplexed will be the default implementation

[0] These guards could be potentially extended to be pluggable much like middlewares to express different concerns required by differentally configured environments.