3.0.0

This release closes the remaining gaps with go-libp2p-pubsub v0.15.0 found by the 2.0.0 review: GossipSub v1.3, topic discovery, Go's trace formats, the peer gater and the seen-message cache. It also adds two options applications asked for: setting a topic's score parameters at runtime, and the outbound queue size.

Upgrading from 2.x

• Custom routers must implement Router.enoughPeers(topic, suggested), which discovery and publish(ready:) use. The built-in routers implement it. This is the only breaking API change.

• Trace files change format to Go's (varint framing for PbEventTracer, Go's JSON for JsonEventTracer). Tools that read the old formats need updating.

• GossipSubRouter negotiates /meshsub/1.3.0 with peers that speak it (including Go v0.15.0 nodes). Older peers still negotiate 1.2.0 or lower.

• Messages for topics the node does not subscribe to are now dropped, as in Go. A node that relied on validating or forwarding them must subscribe to the topic.

• The tracers write go-libp2p-pubsub's formats, so Go's trace tools can read their files. PbEventTracer prefixes each event with its length as a varint, where it wrote 4 bytes. JsonEventTracer writes one JSON object per line with the fields named as in trace.proto, bytes in base64, and the type and timestamp as numbers, where it wrote field numbers as keys.

• Every event carries the local peer ID and a timestamp (nanoseconds since the Unix epoch), as in Go. The router used to set peerID to the remote peer, and no event had a timestamp. Routers now send events through the new PubSub.traceEvent, which stamps them.

• RECV_RPC events carry the summary of the RPC (meta), and the RPC summaries list the subscriptions, as in Go.

• Tracing after dispose() no longer throws: the event is dropped. stop() and dispose() can be called again.

Added for applications

• GossipSubRouter.setTopicScoreParams(topic, params) sets or changes a topic's score parameters after the router is created, as go-libp2p-pubsub's Topic.SetScoreParams: a topic created later can use the built-in penalties (such as P4 for invalid messages). Lowering the delivery caps caps the peers' counters, as in Go. PeerScore.topicScoreParams(topic) reads them. The scorer is now created with the router rather than when it is attached.
• PubSub(peerOutboundQueueSize: n) sets the number of RPCs queued per peer before more are dropped, as go-libp2p-pubsub's WithPeerOutboundQueueSize (default 32).

Discovery

• Topic discovery works, as go-libp2p-pubsub's WithDiscovery. PubSub(discovery: ...) takes a dart_libp2p Discovery service. PubSub advertises each subscribed topic, every second looks up the peers of the subscribed topics that the router has not enough peers on, and dials them through a backoff connector (discoveryConnector, default Go's). discoveryOptions are passed to the service. Before, the discovery helper was not connected to PubSub.
• Topics are advertised under Go's namespace, floodsub:<topic>, so Dart and Go nodes find each other. The helper used gossipsub_topic:<topic>.
• publish(topic, data, ready: minTopicSize(n), readyTimeout: ...) waits until the router has enough peers on the topic before publishing, looking them up while it waits, as Go's WithReadiness.
• Breaking for custom routers: Router has a new method, enoughPeers(topic, suggested), as Go's EnoughPeers. The built-in routers implement it with Go's rules.
• Removed PubSubDiscovery's old API (addDiscoveryListener, discoverTopic, DEFAULT_GENERAL_PUBSUB_SERVICE_TAG, TOPIC_DISCOVERY_PREFIX). The class was not exported.

Messages

• Messages for topics the node does not subscribe to are ignored, before validation, as in go-libp2p-pubsub. They used to be validated, scored, cached and forwarded, which cost validation work on topics the node had no interest in.

Peer gater

• Added go-libp2p-pubsub's peer gater (WithPeerGater): GossipSubRouter(peerGaterParams: PeerGaterParams()). When validation is throttled, it handles only the control messages of peers whose messages are mostly not delivered, at random. Off by default, as in Go.
• PubSub.validateMessage takes onReject, called with the reason a message was dropped.
• Removed lib/src/core/peer_gater.dart, a connection gater that nothing used and was not exported.

Seen messages

• The seen-message cache has no size limit, as in go-libp2p-pubsub. It held at most 2^17 IDs and evicted the oldest beyond that, so under a high message rate a message could be validated and forwarded again within seenMessagesTTL.
• The cache uses a monotonic clock, so a change to the system clock no longer expires IDs early or keeps them too long.
• seenMessagesStrategy (GossipSubParams, FloodSubRouter, RandomSubRouter) chooses Go's first-seen (default) or last-seen expiry, as WithSeenMessagesStrategy.
• Removed lib/src/util/backoff.dart, which nothing used and was not exported.

GossipSub v1.3

• GossipSubRouter speaks /meshsub/1.3.0 (gossipSubIDv13), first, as go-libp2p-pubsub v0.15.0, and negotiates it with Go nodes. With a v1.3 peer, each side announces its extensions in its first RPC; a peer that announces them again gets a behaviour penalty of 10, as in Go. v1.3 peers get IDONTWANT and PX as v1.2 peers.
• The experimental test extension, as Go's WithTestExtension: GossipSubRouter(testExtension: TestExtensionConfig(...)). Unlike Go, the callback runs only for RPCs that carry a TestExtension message.
• The protobuf gains ControlMessage.extensions, ControlExtensions and RPC.testExtension, encoded as Go encodes them.
• PubSubProtocol.onFirstRpc lets a router rewrite the first RPC on each new stream.

Tests

• Peer Exchange is tested on a real network: peers pruned by a bootstrapper with PX connect to each other, build a mesh and exchange messages. The test stores the signed peer records in the bootstrapper's address book itself, as dart_libp2p 4.0.1's identify does not.

Known limitations

• The PRUNEs we send rarely carry signed peer records: dart_libp2p 4.0.1's identify checks the records of remote peers but does not store them.

v2.0.0

Breaking release. A review against go-libp2p-pubsub v0.15.0 found remote denial-of-service holes, a crash, message IDs that did not interoperate with Go, and a peer-scoring model that did not work. 2.0.0 fixes them and aligns the GossipSub router with Go.

Upgrading from 1.x

• Peer scoring moved to the router and is opt-in. Pass scoreParams: and scoreThresholds: (both, together) to GossipSubRouter. PubSub no longer has scoreParams, getPeerScore, getPeerScoreObject, peerScores or refreshScores; use router.score.
• Message IDs changed to Go's default (from bytes followed by seqno). Nodes on 1.x and 2.0.0 compute different IDs for the same message, so upgrade a network together.
• GossipSubParams uses Go's defaults and validation rules (see Router below).
• Peer Exchange is off by default (GossipSubRouter(doPX: true)).
• PubSub.stop() no longer disposes the tracer (dispose it yourself), keeps subscriptions, and can be followed by start().
• Router gained protocols and acceptFrom; custom routers must implement them.
• dart_libp2p >=4.0.1 <5.0.0 is required (was >=1.0.0); this release is tested only against 4.0.1.

Security

• A failed stream open crashed the process. A peer that connected without speaking pubsub made the per-peer stream lock complete with an error nobody listened to, which terminated the isolate. It no longer does.
• Inbound RPC size was unbounded. A peer could announce a huge frame and make the node buffer it. Frames above maxMessageSize (default 1 MiB, PubSub(maxMessageSize:)) now reset the stream before being read. Outgoing RPCs are split to fit (as Go's RPC.split), and an oversized message is dropped.
• IWANT replies could be amplified. Requested IDs were not de-duplicated and could be requested without limit. Each message is now sent once per IWANT, at most gossipRetransmission (3) times per peer, and only to peers above the gossip threshold.
• A PRUNE with a huge backoff aborted RPC handling. Backoffs are now capped to maxPruneBackoff (24 h).
• Messages were marked seen before their signature was checked, so a forged copy blocked the genuine message and got honest forwarders penalised. As in Go, a message is marked seen once its signature is verified, and messages failing signature checks are not tracked.
• IHAVE handling had no limits. maxIHaveMessages, maxIHaveLength and IWANT promises (a peer that does not deliver an advertised message is penalised) now apply, and IHAVEs for topics not joined are ignored.
• Graylisted peers (score below graylistThreshold) have their RPCs ignored (Router.acceptFrom).
• The outgoing RPC queue was unbounded and could keep failed RPCs forever. It now holds 32 RPCs per peer with a priority lane, drops RPCs when full (traced as DROP_RPC; dropped GRAFT/PRUNEs are retried with the next RPC), and drops an RPC that fails to send.
• Concurrent writes could interleave frames on a stream; writes are now serialised per stream.
• A message that claims to come from this node but arrives from a peer is rejected.
• The blacklist is enforced, as go-libp2p-pubsub's WithBlacklist and BlacklistPeer. PubSub(blacklist:) takes a Blacklist (now exported) and PubSub.blacklistPeer adds a peer, removing it from the router if connected. A blacklisted peer is not added or greeted, its RPCs are ignored, and messages it forwards or wrote are dropped before validation (traced as blacklisted peer / blacklisted source) without being marked seen. Before, Blacklist was not used by anything.

Interop

• Message IDs now match Go. The default ID is the bytes of from followed by seqno (Go's DefaultMsgIdFn), and messageIDs in IHAVE/IWANT/IDONTWANT are bytes (wire-compatible with Go's string). IDs are Dart strings with one byte per code unit (messageIdFromBytes, messageIdToBytes). PubSub(messageIdFn:) sets a custom ID function.
• Protocols. GossipSub speaks /meshsub/1.2.0, 1.1.0, 1.0.0 and /floodsub/1.0.0 and negotiates one per peer (Router.protocols); features follow the protocol (PX from v1.1, IDONTWANT from v1.2). FloodSub peers get every message of their topics.
• IDONTWANT (v1.2) is sent for large messages and honoured.
• Signature policies: PubSub(signaturePolicy:) takes strictSign (default), strictNoSign, laxSign or laxNoSign, and noAuthor omits from/seqno, as Go's WithMessageSignaturePolicy and WithNoAuthor.
• Heartbeat gossip: the heartbeat now emits IHAVE gossip for recent messages to max(DLazy, gossipFactor * peers) peers, so lost messages are repaired; IHAVE is no longer sent at publish time. The message cache keeps historyLength heartbeats and gossips the last historyGossip, as Go's.
• FloodSub and RandomSub work: they register their protocols, keep a seen cache, validate messages, and skip the source and the author. RandomSub sends to max(RandomSubD, sqrt(networkSize)) peers.

Peer scoring (breaking)

• Scoring was rebuilt on go-libp2p-pubsub's score.go: decaying counters per peer and topic, a score computed on demand from P1–P7, delivery records, IP colocation and the behaviour penalty. Before, the score accumulated on each refresh and one penalty pinned a peer near -1000.
• Scoring moved from PubSub to the router and is opt-in, as in Go: GossipSubRouter(scoreParams:, scoreThresholds:), both required together. PubSub no longer has scoreParams, getPeerScore, getPeerScoreObject, peerScores or refreshScores; use router.score.
• PeerScoreParams, TopicScoreParams and PeerScoreThresholds have Go's fields and validate() rules (topics, topicScoreCap, appSpecificWeight, ipColocationFactorWeight, behaviourPenaltyThreshold, gossipThreshold, publishThreshold, graylistThreshold, acceptPXThreshold, opportunisticGraftThreshold, ...). scoreParameterDecay is Go's ScoreParameterDecay.

Peer Exchange and lifecycle

• The router now connects to the peers offered in a received PRUNE (Peer Exchange), as Go's pxConnect: when the sender's score is at least acceptPXThreshold, up to prunePeers of them, through connectors (8) concurrent attempts with at most maxPendingConnections (128) queued and a connectionTimeout (30 s). A signed peer record must be about the peer and signed by its key; its addresses are stored in the certified address book. Our PX now carries the signed records the address book has (dart_libp2p 4.0.1's identify does not store remote records, so these are mostly records learned from PX).
• stop() then start() works, and a peer that restarts its pubsub is picked up again:
  • PubSub.stop() removes all peers from the router and closes all pubsub streams; the router forgets its meshes, fanouts, backoffs and queues. start() registers the protocol handlers again, rejoins the subscribed topics and greets the connected peers again. Subscriptions and validators are kept across a restart.
  • stop() no longer disposes the tracer, which it could not reopen (as in Go, the owner of the tracer closes it).
  • As Go's handlePeerDead, a peer that ends our stream to it (it stopped its pubsub) is removed from the router; if it is still connected, it is greeted again with Go's dead-peer backoff (at most 4 attempts in 10 minutes), so it is added back when it restarts.
• PubSub.stop() could hang forever. Closing a stream waits for its FIN to be sent behind any data queued for a peer, and PubSubProtocol.close() waited for every stream with no limit, so one slow or stalled peer could block stop() indefinitely (reported on 1.5.0 after a burst of 100 messages). Each stream close is now bounded by streamCloseTimeout (default 2 s) and the stream is reset when it expires. This applies to close(), closePeerStream and the stream cleanup in sendRpc, whose wait stalled that peer's send queue. close() now also closes inbound streams, ending their read loops.

Router (breaking)

• The heartbeat follows Go: prunes mesh peers with a negative score, keeps DOut outbound peers, keeps the DScore best peers when pruning, grafts opportunistically only when the mesh median is below opportunisticGraftThreshold, filters fanout by publishThreshold, and keeps backoffs two heartbeats past their end.
• GRAFTs from peers with a negative score, or from inbound peers when the mesh has DHigh peers, are refused with a PRUNE.
• Flood publish is on by default (floodPublish), and forwarded messages skip their author.
• Peer Exchange is off by default (GossipSubRouter(doPX: true)).
• GossipSubParams: DScore is now a peer count (default 4), DLow defaults to 5, new DOut, gossipFactor, historyLength, historyGossip, maxIHaveLength, maxIHaveMessages, iwantFollowupTime, opportunisticGraftPeers, IDONTWANT parameters; opportunisticGraftScoreThreshold was removed; prunePeers defaults to 16. validate() uses Go's rules.
• Router has protocols and acceptFrom; PubSub now calls Router.addPeer for each pubsub peer, sends its hello to every connected peer (including peers connected before start()), and announces subscription changes to peers being greeted.
• Validation no longer blocks reading a peer's stream.

Also

• The public library exports the routers, Router, Topic, the protocol IDs, the message ID helpers and the tracers.
• README links point to doc/.

Known limitations

• The PRUNEs we send rarely carry signed peer records: dart_libp2p 4.0.1's identify checks the records of remote peers but does not store them, so we offer mostly records learned through PX. Peer Exchange is tested with mocks, not yet on a real network.
• The tracers do not yet write Go's format (the protobuf tracer frames events with 4 bytes where Go uses a varint; the JSON tracer writes field numbers, no timestamps, and the remote peer as peerID).
• Topic discovery uses the name...