Summary
Let’s write a doublezero RFC (request for comments) that we will publish into https://github.com/malbeclabs/doublezero/tree/main/rfcs to reach the goal set out below. See the RFC template here - https://github.com/malbeclabs/doublezero/blob/main/rfcs/rfc0-template.md. See the other RFCs in [https://github.com/malbeclabs/doublezero/blob/main/rfcs/](https://github.com/malbeclabs/doublezero/blob/main/rfcs/rfc0-template.md)rfc-*.md to understand the level of detail that is appropriate for a doublezero RFC.
Rialo wants to guarantee users that their data is stored and processed only in specified jurisdictions (namely Europe, for GDPR protections, at launch). This requires a system that can verify that certain network nodes, known as REX devices (”Clients” in our RFC), are physically located where they claim to be. We want to design a system that builds on https://github.com/malbeclabs/doublezero/, which already has a system that measures latency between doublezero devices (DZDs) using the doublezero telemetry agent (https://github.com/malbeclabs/doublezero/tree/main/controlplane/telemetry), which runs on DZDs and writes data to the telemetry smartcontract program (https://github.com/malbeclabs/doublezero/tree/main/smartcontract/programs/doublezero-telemetry, design details are in https://github.com/malbeclabs/doublezero/blob/main/rfcs/rfc4-telemetry-for-contributor-rewards.md).
We know exactly what building each DZD is in, so we have a highly accurate location, withing 10s of meters, for the DZD. We will use this as the basis for determining the approximate location of a nearby probe, but we will express that location as being “z milliseconds away from lat x / lon y”. From there we can establish the approximate location of Rialo REX devices (”Clients”) by pinging them from the probe.
Our goal in this phase is to design a minimum viable version of the system, write the code needed, and deploy it in doublezero testnet to a single probe that we will build. The RFC should provide details for a POC and a MVP.
Please ask clarifying questions if there is any information missing from this issue.
Requirements
- Proof-of-concept requirements
- DZD can pull mated probe IPs from DZ Ledger for which it should be pinging
- Requires a new probe object onchain. Should that go in the telemetry program, serviceability program, or a new program? Info about the existing programs is in https://github.com/malbeclabs/doublezero/tree/main/smartcontract. See the README there for more info.
- We need a place to store measurements onchain. Should this be in a new data structure similar to the existing DZLatencySamples in the telemetry program? Or perhaps in a new program?
- DZD posts the “Offset” for each mated probe on-chain, in addition to sending the offset to the probe directly along with the lat/lon of the DZD & DZD Pubkey
- Probes will add the “Offset” received from a DZD to their measured offset. In the MVP the probe will use one reference. It will add the measured latency between it and the client, to the latency in its reference offset. It will then use the DZD’s lat/lon.
- Full requirements for MVP - not required for current POC phase:
- Probes.
- We (we being some combination of Malbec Labs, DoubleZero Foundation, and DoubleZero contributors) need to deploy a bare metal probe server in every doublezero exchange (you can get the list with
doublezero --env mainnet-beta exchange list --json.
- We need to decide where to deploy each probe depends on establishing a latency threshold for probes, where a probe must to be within milliseconds of a device.
Design guidance
Please use the following design guidance points. Ask for user feedback if one of these has a critical issue.
Design diagram: 
Core Concepts:
-
A Probe↔DZD Relationship is called an “Offset”. Offsets are stored on-chain and sent to the probe. MVP has probe using one offset and adding it to its measured latency.
💡
A future improvement will have probes using multiple offsets to compute a more accurate starting point.
A DZD will send an offset to a probe that contains no references. A DZD doesn’t require references to prove its location.
A Probe will send an offset to a client that includes its references. This is a vector of offset structs that were used to establish the lat/lon used in the offset sent by the Probe. A receiver could recreate the lat/lon from the references.
An example of what data is include in an Offset:
type Offset struct {
Signature [64]byte
Pubkey [64]byte
Lat float64
Lon float64
OffsetNS uint64,
NumReferences byte
References []Offset
}A probe will have an open UDP port on which it is listening to receive references. For POC and MVP, a probe will only accept Offsets from a DZD pubkey specified at startup with a command line argument. It will always use the most recent offset it receives.
Open questions
How should DZDs Sign Latencies to their mated Probe(s)?
- Let’s create a version of the telemetry agent to deploy on the probes. We may need to refactor the telemetry agent to make this possible.
- Concern: IP spoof, fake probe close to DZD, actual probe far away. DZD thinks its pinging actual probe, but it’s actually far away. We are not solving for this in the PoC but should discuss it in the Security Considerations section
- Infrastructure planning: we need to find a vendor in each exchange where we can run a bare metal probe in the same data center as one of our devices.
- Does Offset data structure above need to identify what type of pubkey the Pubkey is, like is it a device in the serviceability devices table, or a probe in the serviceability probes table (or telemetry probes table)?
Proposed Solution: Telemetry agent should also ping mated probes as an additional link for TWAMP probes and post the probe’s latency from the DZDs to DZ Ledger.
PoC: Sidecar uses same private key json file
Probe device will have its own separate private key.
The dual posting of results: Leaving a source of truth on the DZ ledger for Rialo to verify against adds a perceived trust, and crates further DZ integrations. It also helps enable Nihar’s extension with multi-probe verification.
Summary
Let’s write a doublezero RFC (request for comments) that we will publish into https://github.com/malbeclabs/doublezero/tree/main/rfcs to reach the goal set out below. See the RFC template here - https://github.com/malbeclabs/doublezero/blob/main/rfcs/rfc0-template.md. See the other RFCs in [https://github.com/malbeclabs/doublezero/blob/main/rfcs/](https://github.com/malbeclabs/doublezero/blob/main/rfcs/rfc0-template.md)rfc-*.md to understand the level of detail that is appropriate for a doublezero RFC.
Rialo wants to guarantee users that their data is stored and processed only in specified jurisdictions (namely Europe, for GDPR protections, at launch). This requires a system that can verify that certain network nodes, known as REX devices (”Clients” in our RFC), are physically located where they claim to be. We want to design a system that builds on https://github.com/malbeclabs/doublezero/, which already has a system that measures latency between doublezero devices (DZDs) using the doublezero telemetry agent (https://github.com/malbeclabs/doublezero/tree/main/controlplane/telemetry), which runs on DZDs and writes data to the telemetry smartcontract program (https://github.com/malbeclabs/doublezero/tree/main/smartcontract/programs/doublezero-telemetry, design details are in https://github.com/malbeclabs/doublezero/blob/main/rfcs/rfc4-telemetry-for-contributor-rewards.md).
We know exactly what building each DZD is in, so we have a highly accurate location, withing 10s of meters, for the DZD. We will use this as the basis for determining the approximate location of a nearby probe, but we will express that location as being “z milliseconds away from lat x / lon y”. From there we can establish the approximate location of Rialo REX devices (”Clients”) by pinging them from the probe.
Our goal in this phase is to design a minimum viable version of the system, write the code needed, and deploy it in doublezero testnet to a single probe that we will build. The RFC should provide details for a POC and a MVP.
Please ask clarifying questions if there is any information missing from this issue.
Requirements
doublezero --env mainnet-beta exchange list --json.Design guidance
Please use the following design guidance points. Ask for user feedback if one of these has a critical issue.
Design diagram:
Core Concepts:
A Probe↔DZD Relationship is called an “Offset”. Offsets are stored on-chain and sent to the probe. MVP has probe using one offset and adding it to its measured latency.
💡A future improvement will have probes using multiple offsets to compute a more accurate starting point.
A DZD will send an offset to a probe that contains no references. A DZD doesn’t require references to prove its location.
A Probe will send an offset to a client that includes its references. This is a vector of offset structs that were used to establish the lat/lon used in the offset sent by the Probe. A receiver could recreate the lat/lon from the references.
An example of what data is include in an Offset:
A probe will have an open UDP port on which it is listening to receive references. For POC and MVP, a probe will only accept Offsets from a DZD pubkey specified at startup with a command line argument. It will always use the most recent offset it receives.
Open questions
How should DZDs Sign Latencies to their mated Probe(s)?
Proposed Solution: Telemetry agent should also ping mated probes as an additional link for TWAMP probes and post the probe’s latency from the DZDs to DZ Ledger.
PoC: Sidecar uses same private key json file
Probe device will have its own separate private key.
The dual posting of results: Leaving a source of truth on the DZ ledger for Rialo to verify against adds a perceived trust, and crates further DZ integrations. It also helps enable Nihar’s extension with multi-probe verification.