UPDATE operations not propagated between peers - breaks collaborative applications like River

[sanity]

Summary

UPDATE operations are not propagating correctly between peers in many network configurations, causing divergent contract states and breaking collaborative applications like River. Investigation suggests the issue stems from gaps in the subscription network that prevent updates from reaching all contract holders.

Problem Description

Current Behavior

When multiple peers cache the same contract and make updates:

• Updates only propagate when there's an existing subscription path between peers
• Many peers end up isolated with no subscription connections
• Contract creators often have no subscribers
• Result: Each peer maintains divergent state unless connected by subscriptions

Test Evidence

Using multi-machine-test with River:

1. Alice creates a room → stores contract locally (no subscriptions)
2. Bob joins room → retrieves contract, subscribes to provider
3. Alice sends message → updates her local state
4. Bob sends message → updates his local state
5. Result: Neither sees the other's messages (no subscription path between them)

Key Question: Do Updates Travel Up AND Down Subscription Trees?

The code suggests subscriptions might be bidirectional:

• When peer A subscribes to peer B, B adds A to its subscriber list (line 277 in subscribe.rs)
• When A receives confirmation, A also adds B to its subscriber list (line 380)
• This would create bidirectional connections: A ↔ B

Need to verify: Are updates actually propagating in both directions through these connections? Or is there still a unidirectional limitation?

When Updates DO Work

Updates successfully propagate when:

• There's a subscription path between the updater and receiver
• Example: If C subscribes to B, and B subscribes to A, updates from A reach C

When Updates FAIL

Updates fail to propagate when:

• No subscription path exists between peers (most common)
• Original creator has no subscribers (isolated node)
• Peers cache from different sources (disconnected islands)

Root Causes Identified

1. Disconnected Subscription Networks

   • Contract creators (like Alice) often have no subscribers
   • Peers that cache independently don't form subscription connections
   • Creates isolated islands with no update paths between them
   • No automatic discovery between peers holding the same contract

2. Possible Unidirectional Flow (Needs Verification)

   • Code analysis suggests subscriptions should be bidirectional
   • But need to confirm updates actually flow both up and down
   • get_broadcast_targets_update() only returns subscribers - is this complete?

3. Incomplete Implementation

   • TODO comment in update.rs: // TODO: complete update logic in the network

Proposed Solution

A two-part approach to create robust update propagation:

Part 1: Verify and Fix Bidirectional Subscriptions

First, confirm whether subscriptions are truly bidirectional:

• Test if updates flow both up and down subscription trees
• If not, fix the implementation to match the intended design
• Ensure the delta-sync "viral spread" model works as intended

Part 2: Proximity Propagation (Essential)

Add a neighbor-based propagation mechanism to connect peers that aren't in subscription relationships:

Concept:

• Peers periodically exchange bloom filters of cached contracts with immediate neighbors
• When receiving an UPDATE, forward to neighbors who likely cache the same contract
• Creates automatic mesh topology based on actual cache state

Benefits:

• Connects all contract holders regardless of how they obtained the contract
• Self-organizing network topology
• Naturally bidirectional
• Fills gaps in subscription network

Implementation approach:

// Periodic neighbor discovery
fn exchange_cache_filters(neighbor: Peer) {
    neighbor.send(bloom_filter_of_cached_contracts())
}

// On UPDATE received
fn propagate_update(contract_key: Key, update: Update) {
    // First: Send through subscription channels
    for peer in subscription_peers(contract_key) {
        peer.send(update)
    }
    
    // Second: Send to neighbors with matching contracts
    for neighbor in immediate_neighbors() {
        if neighbor.likely_has_contract(contract_key) {  // via bloom filter
            neighbor.send(update)
        }
    }
}

Part 3: Enhanced Subscription Formation (Optional)

Consider improving how subscriptions are formed:

• Contract creators should subscribe to first retrievers
• More sophisticated logic for automatic subscriptions
• Ensure connected topology from the start

Why This Solution Works

1. Connected subscription networks ensure updates can reach all participants
2. Proximity propagation creates redundant paths for update delivery
3. Together they form a robust mesh network matching the delta-sync vision
4. Gradual rollout possible: Can verify/fix bidirectional flow first, then add proximity propagation

Technical Considerations

• Loop prevention: Need message IDs or TTL to prevent infinite propagation
• Bloom filter tuning: Balance between false positive rate and filter size
• Rate limiting: Prevent update storms in highly connected networks

Test Verification

The solution should pass these scenarios:

1. Original creator receives updates from all participants
2. New joiners receive historical updates
3. Updates propagate even with partial network connectivity
4. Network converges to consistent state
5. Updates flow both up and down subscription trees

Related Context

• Delta-sync design: https://freenet.org/news/summary-delta-sync/
• Current implementation: crates/core/src/operations/update.rs
• Integration tests missing: crates/core/tests/operations.rs only tests single client

This architectural investigation and fix is essential for Freenet to deliver on its promise of distributed, eventually-consistent state synchronization.

[AI-assisted debugging and analysis]

[iduartgomez]

First, confirm whether subscriptions are truly bidirectional:

Yeah, this should be working, if not there is a bug. Subscription trees should form naturally as peers cache the contract so we should get updates spreading across the network, unless buggy.

[sanity]

@iduartgomez What do you think of the "proximity propagation" proposal - I think that's the most significant change that should hopefully discourage disconnected subscription trees like we seem to be seeing.

[sanity]

Closing as this is now consolidated into #1848 which provides a comprehensive solution to the update propagation architecture issues. [AI-assisted debugging and comment]