Skip to content

Latest commit

 

History

History
244 lines (207 loc) · 9.26 KB

pdp-0002.mediawiki

File metadata and controls

244 lines (207 loc) · 9.26 KB
Raw

PDP: 2
Layer: Applications
Title: PAI Data Storage and Sharing
Author: Alex Waters <alex@waters.nyc>
        Mark Harvilla <mark@oben.com>
        Patrick Gerzanics <patrick.gerzanics@upandrunningsoftware.com>
Status: Proposed
Type: Informational
Created: 2018-05-15
License: MIT

Table of Contents

Abstract

This document describes an initial specification for extending the standard PAI Coin protocol to include a method of submitting and provisioning access to arbitrary data. This is achieved by utilizing a segmented network of torrent nodes, a custom OP_RETURN data protocol and elliptical encryption.

The specification consists of several parts. In the first part, the protocol for the OP_RETURN format for submitters and recipients is detailed. The second part discusses how the data is stored and distributed in v1 of the implementation. The third part covers further expansion of ideas for indexing, storage and participant rewards.

Copyright

This PDP is licensed under the MIT license.

Motivation

Securely sharing data, such as on a public website, has been well-covered for some time using TLS/SSL. However, what happens to that data once it has been shared and is at rest is undefined. One of the classical uses of blockchain is to act as a distributed ledger. We believe that this ledger functionality can be leveraged for tracking grants and revocations to allow end users full control over what happens to their data in transit and at rest.

Specification

Data Storage Flow Summary

Glossar

Submitter-- User that is providing data to a Recipient

Provider-- Data storage provider as tracked in the PAI blockchain

Recipient-- Entity that is to receive the data from the Submitter

Flow

Submitter and Recipient: Monitor the PAI blockchain for data storage provider OP_RETURN messages to have an index of available storage providers; adding and removing as necessary. Submitter: Encrypts required data using the intended recipient’s public key. Submitter: Submits encrypted data to a PAI data storage provider using the Provider API and receives the unique data ID. Provider: Stores data within their storage network. Submitter: Creates a transaction on the blockchain according to the OP_RETURN protocol. This grants access to the data to the intended recipient as well as pays the storage provider, if necessary. Recipient: Notified via PAI transaction with OP_RETURN and their wallet address that data has been submitted. Retrieves ID from OP_RETURN. Recipient: Calls Provider API get-torrent to retrieve the torrent file for accessing data Recipient: Retrieves data using the torrent protocol Recipient: Monitors for ‘revoke’ operations. When such is received immediately destroys all copies of provided data and discontinues any usage.

OP_RETURN Protocol

In order to achieve our goals with using the PAI blockchain to track provisioning, revocation and storage requests we need to embed this data into transactions. To do so, we have chosen to use the standard approach of embedding data in the chain utilizing the op code OP_RETURN.

Glossary

Header: The initial 8 bytes after the OP_RETURN code that indicate that it is a PAI data storage txout and the version currently being used. Payload: The operation and associated parameters for the request Operations: Types of requests that can be made via the PAI Coin data storage implementation Storage Method: This is the storage method being utilized. In the initial implementation we have 0x01 which is the segmented bittorrent based solution as detailed below.

Protocol Definition

PAI PROTOCOL Purpose Length (bits) Value (hex)
PAI HEADER PAI delimiter 8 0x92 = crc8("PAI")
Protocol Version 8 0x10 = 1.0
Reserved 48 0xFFFFFFFFFFFF
PAI PAYLOAD Operation 8 0x00..0xFE + encoded NOP(0xFF)
Storage method 8 0x00..0xFE + encoded NST(0xFF)
First operand size 8 0x0..0x41 (max 65 bytes)
First operand 256
Second operand size 8 0x0..0x40 (max 64 bytes)
Second operand 256
PAI Checksum DSHA256 32 SUBSTR(0, 8, DSHA256(CONCAT(header, payload)))

Protocol Operations and Operands

Operation ID Operation Operand 1 Operand 2
0x00 grant data hash reference (sha-256 of encrypted data) txout index of wallet address
0x01 revoke data hash reference (sha-256 of encrypted data)
0x02 add_provider ip address/hostname (max 32 bytes) port
0x03 remove_provider ip address/hostname (max 32 bytes) port

Operation Details

  • Grant - Method for granting access of arbitrary data to a recipient. If the storage provider requires a payment for this operation, it should be paid for with the Grant request by one of the outputs of the transaction.
  • Revoke - Method used to inform the original recipient of the encrypted data that all copies of the provided data should be destroyed and that further usage is prohibited.
  • Add Provider - Method used with the initial PAI Data Storage method to add entrypoints for submitting data to the PAI Data Storage system per the API definition defined below.
  • Remove Provider - Method used with the initial PAI Data Storage method to indicate that an existing entrypoint should no longer be used and has been retired.

Storage Provider API and Behavior

In order to standardize interactions with PAI data storage providers, below is the initial API paths, parameters and responses. This is only storage method 0x01 per the OP return protocol.

Path Parameters Success Response Error Codes
/info N/A { "provider": "ACME Data Store", "retention_period": "30d", "cost":0, "wallet": "[Payment]" } N/A
/make-torrent file: [encrypted] key: [public] { "result": "Success", "id": "[Data]", "cost": 0 } 405 -- Invalid params
/get-torrent id: [hash] { "result": "Success", "url": "[torrent]" } 404 -- Unknown data hash
/remove id: [hash] signature: [signed] { "result": "Success", } 403 -- Invalid signature

  • Info - Used to retrieve high level information about a given provider. This can be used to determine information about an unknown provider that has been added via the add-entrypoint operation. Response data is currently limited to the provider name, retention period and cost (per MB) stored.
  • Make Torrent - This is used to submit an encrypted data file to be stored on the provider along with a public key to use for later operations. The data provider should create the torrent file, associate it with the public key, add it to its index and return the ID to be used by other operations
  • Get Torrent -- This is used by the recipient of the data file to get the torrent file to be used for retrieval.
  • Remove - This is used by the original submitter to remove the data from the storage provider at which time it is no longer required. It requires a signed version of the file ID using the private side of the keypair submitted in the Make Torrent request.
In addition to these operations, the storage provider is free to use the ‘retention_period’ as returned in the ‘info’ request in order to cleanup old files. Any files older than the retention period can be removed. Likewise, if there is a cost for the data provider than the file can be removed after 24 hours if a transaction is not received to pay for storage for the length of the retention period to the ‘wallet’ specified by the ‘info’ endpoint.

Compatibility

This PDP on its own does not cause any backwards incompatibility.

Implementation

An initial implementation of this PDP is included in the PAI Coin core in the contrib/data-store directory. This includes a Python API implementation of the Data Provider specification and Dockerfiles for launching torrent nodes using OpenTracker. It also includes sample code for submitting OP_RETURN transactions and exercising the protocol for Submitters and Recipients.