Subgraph Versioning operational features

[hopeyen]

Problem statement

Motivated by forum post, Initial design doc

Radios can already send and receive subgraph versioning messages. Indexers running the radio with notifications should receive version upgrade message from verifiable subgraph owners. Currently we expect them to handle the message manually.

We can consider further handling of the messages and interactions between indexers and subgraph developers. This should close the gap to achieve seamless subgraph version upgrades. For developers, the subgraph can be published and upgraded with deterministic data availability and service quality. For indexers, the subgraph traffic becomes predictable during the upgrade and subgraphs can be pre-synced so query services do not get disrupted.

General workflow

sequenceDiagram
    actor DR as Subgraph A Owner
    participant GN as Graphcast Network
    participant SIR as Subscribed Indexer Radios
    participant IMS as Indexer Management Server
    SIR-->>GN: Periodic Public PoI messages (topic Subgraph A-0)
    Note over DR: Deploy new Subgraph A version 1
    DR-->>GN: Send version upgrade message (A-0)
    GN-->>SIR: Boardcast version upgrade message (A-0)
    activate SIR
    SIR->>SIR: Trusted source identity verification
    deactivate SIR
    opt Sender identity as Subgraph owner verified
        opt Auto sync management
            SIR->>IMS: POST request to initiate off-chain syncing A-1
            IMS->>SIR: Response from Graph Node (Success/Error)
            alt Success
                SIR->>SIR: Update topics
                SIR-->>GN: Subscribe to A-1
                SIR-->>GN: Broadcast public status endpoint (A-1)
            else Error
                SIR-->>GN: Broadcast error message (A-0)
            end
        end
        opt Notifications
            activate SIR
            SIR->>SIR: Notify events to human
            deactivate SIR
        end
    end
    opt Continuous Radio
        activate DR
        DR-->GN: Collect Public Status messages (A-1)
        Note over DR: Monitors for updatedness threshold 
        deactivate DR
    end
    DR-->SIR: Switch service from A-0 to A1, deprecate A-0
    SIR-->>GN: Unsubscribe to A-0

Loading

Expectation proposal

For indexers:

• Automatically manage the indexer to start syncing the new subgraph deployment.
  • Optional, adds dependency to indexer management server
  • If the topic coverage is set to comprehensive, the radio will subscribe to the new deployment hash during the next content topic refresh iteration.
  • If the topic coverage is set to on-chain, the radio will neither publish nor subscribe to this topic until the indexer operator allocates resources to the new version.
• While syncing the new subgraph, indexers could send indexing_statuses messages to the new hash channel. This could create a competitive race to chainhead even though unverified and no rewards.

For subgraph developers:

• Developers are not expected to run the radio continuously, but there could be a benefit to do so anyway
  • Developers can subscribe to the subgraph deployments they own and monitor the public PoI consensus
  • If the indexers send indexing_statuses messages, then subgraph owners can immediately triage and respond to the problem with well defined error information, before or even after the version gets published to the network.
  • Developers can upgrade the service deterministically, ensuring stability.

Issue breakdown

Minimal

• Add indexer_management_server_url: Option<String> to configs. (issue [Feat.Req] Add indexer management server url #23)
• In VersionUpgradeMessage handler, optionally send POST request indexer offchain new_hash to the url. topics should be automatically updated according to topic_coverage (issue [Feat.Req] Auto-upgrade by version upgrade message #24)
• Ratelimit subgraph deployment

Can extend on the developer + indexer continuous interactions later on

• Always persisting VersionUpgradeMessages as part of persisted_state (issue [Feat.Req] Store, serve, and clean version upgrade messages #25)

Alternative considerations

• indexers to send DeploymentHealth or IndexingStatus messages to the new hash channel while syncing the new subgraph. or Indexers to send their public indexing status API for more active checking
• developers can continue to run the radio until they receive one or more PublicPoiMessages, the radio notifies them that there are indexers synced up-to chainhead and then shuts down

Additional context
From old repo: graphops/poi-radio#211