[WIP][DOC] High level specs of transaction model

[diminator]

Transaction Model

TL;DR

Update and spec out the transaction model such that users can:

• understand intuitively how the ownership layer works
• understand that crypto-conditions is one of potential ownership languages
• not be annoyed by redundant information like embedded ID's, owners_*, details, ...
• enjoy consistent ownership indexing
• have a clean and resolvable addressing schema
• remove ambiguity on implicit fulfillments in a CREATE transaction
• re-use or update assets
• get a grip on the validation hierarchy

Context and definitions

In BigchainDB, Transactions are used to store both data and ownership Policy of. A somewhat fair representation is shown below.

        +-------------+
        | Transaction |
        |             |
        |+-----------+|
        ||           ||
    o---||           ||---o
    In  ||           || Out
    o---||  Policy   ||---o
     .  ||           || .
     .  ||           ||---o
    o---||           ||
        |+-----------+|
        |+-----------+|
        ||           ||
        ||   Data    ||
        ||           ||
        |+-----------+|
        +-------------+

Each state update to the replicated state machine (RSM) is embedded in a transaction. A validation node in the RSM needs to check the integrity of a transaction and apply all policies. When all checks are valid, then the node will vote the transaction (or it's surrounding block) VALID otherwise INVALID.

Transactions interact with the nodes as follows:

1. node receives tx
2. node validates tx
  - check the integrity of the data structure
  - check the policy of the server
  - check the policy of the client
3. node approves/declines tx

Remember: each step in the validation phase costs computation and directly affects throughput!

Policies

You might have noticed a new notion of policy. A policy is a set of rules that a node can validate.

Some policy are inherent to the RSM: "Rules to create a valid chain". Let's call it Server Policy for now, but it could also be name chain policy, node policy, consistency rules, etc. Some examples include avoiding double spends, making sure the input amounts match the outputs or . This policy is hard coded in each validation node. Clients cannot alter existing or impose their own server policy. It's up to the federation of server nodes to decide and impose the server policy.

Other policies are on the client level: "Rules for a client to spend a transaction output". Let's call it Client Policy for now, but it could also be named transaction policy, user policy, etc. Some examples include providing the right fulfillment at the input to match a condition at the output. Each time a new transaction spends an output it allows the user to create a new policy in the output (eg. setting a new output condition). Note that client policies are not necessarily limited to the insides of a transactions. It could also inherit or import policies from other places such as the previous transaction or somewhere else. This last note is important when you want to create reusable policies. More about that in a later section.

Server policy

In a UTXO model, transactions are chained by linking a transactions and follow a Server Policy. Such policy defines the rules to construct a chain or DAG. Examples are:

• double spent prevention
• conservation of input and output amounts
• checking the integrity of the bytecode - like matching conditions and fulfillments by the URI, and so on.

Server policies are shared among the nodes in the system. Possible implementations use a configuration file, load plugins or fork of the code.

Client policy

To create a transaction BigchainDB@v0.8.0, you need to comply to the CREATE policy. There are many paths to implement such a policy (see #327 #347 #794). In practice the nodes will decide whether or not to use an implicit policy, hence such create policy is part of server policy.

Client policy really restricts itself to rules that are applied by clients to the system. A typical placeholder for these rules is the output of a transaction - but not need to be. A generic representation of a CREATE and TRANSFER transaction using the notion of policies is given below (policy0 and policy1 are client policies).

            tx0                              tx1
 +---------------------+            +-------------------+          
 | inputs to | policy0 |            | inputs  | policy1 |
 | create    |  byte   |---o----o---| to      |   byte  |---o
 | policy    |  code   |            | policy0 |   code  |         
 +---------------------+            +-------------------+         

When you own something then you have control to whom you would give it next. In transaction world this means that a user can specify how to spend the output of a transaction. This means that on each transaction, the client policy can change for that transaction. Typically we store the new policy in the output of a transaction - but more generally (parts of) the policy could exist in other locations (think assets and contract addresses).

How does client policy look like? Policy is mathematically enforced by some cryptographic validation like checking signatures or maybe even executing a stateful smart contract. Client policy is bytecode. One could also think of it as a lock and key pair, where unlocking a transaction requires to provide a key - ( in general: the input to a script ) - and try to fit it on the lock - (in general: evaluating the bytecode of the script with the input, possibly in a VM ).

          tx0                              tx1
 +-------------------+            +-------------------+          
 | key to  |         |            |         |         |
 | create  |  lock0  |---o----o---|  key0   |  lock1  |---o
 | policy  |         |            |         |         |         
 +-------------------+            +-------------------+       

In BigchainDB@v0.8.0, the lock and key pairs are implemented using crypto-conditions with conditions and fulfillments. Every time a transaction arrives at a node, all policies are executed and verified by each node. Only if the execution of the code is valid by consensus, the state of the ledger is updated by appending the transaction to the chain.

Assets & Tokens vs Policy?

The above context doesn't really speak about Assets nor Tokens. It's framed in terms of Transactions and Policy. Tokens and Transactions have a similar meaning. Each Transaction contains a Token. You could think of Assets as a specific implementation of a reusable policy: a pointer to a set of rules.

Multiple inputs and outputs & divisible assets

A transaction can have m inputs and n outputs, each with a specific amount. This way you can implement divisible assets like currency or kilograms of gold. To do so, a transaction combines the amount at each input and splits the outputs different amounts and a conservation policy (subset of the server policy, as it is enforced by the nodes, not the clients) makes sure that the amount at the input match the output. A simplistic representation is shown below.

                                                           +-------------+ 
                                                           |     tx3     |
                                                       o---|             |---o
     +-------------+                                  /    |  amount=1   |
     |     tx0     |                                 /     +-------------+   
     |             |---o-\                          /
     |  amount=4   |      \     +-------------+    /             +-------------+
     +-------------+       -o---|     tx2     |---o              |     tx4     |
                                |             |---o--------------|             |---o
 +-------------+           -o---|  amount=6   |---o--            |  amount=3   |
 |     tx1     |          /     +-------------+      \           +-------------+
 |             |-------o-/                            \
 |  amount=2   |                                       \      +-------------+       
 +-------------+                                        \     |     tx5     |
                                                         -o---|             |---o
                                                              |  amount=2   |
                                                              +-------------+            

Proposed changes and path forward

Transaction

The transaction interface needs to be human understandable, compact and generic.

Transactions is a container that fits in a graph: It contains data, policy code, references and so on and the server policy decides the rules to append a transaction to the graph (block-chain).

The model should reflect these dynamics and have a clear and concise representation

Updatable and Reusable Policies

The client and server policy layers discussed above do not take into account the following:

• "How can I control future spending of my Transactions?"
  A creator of an asset might want to make sure that some checks or signatures are always provided upon every future transfer of this asset - even when the policy creator is not directly involved in the act of asset transfer. You might even want to only enforce a Policy for a limited amount of future transactions, or until eternity
• "How can I reuse existing policies?"
  Every asset is tied to the issuance of the underlying token. Creating an asset policy implies creating a token. How to create new tokens with the same policy? How to update the policies? Is there a BigchainDB address where I can store my asset policy?

The above implies to add an Asset policy layer that can be stored in a Transaction and treated as such:

• Asset policy: validate checks (code or schema) for each transaction related to an asset

Validation

Terminology:

key and lock or fulfillment and condition - albeit in the ownership layer of a transaction or embedded in an asset policy, there must be a clear definition on locks and keys. This means:

• a clear specification on the scope of locks and keys, including terminology

• abstraction of the language that is used to generate and validate locks and keys, this also implies clean versioning of the language (eg cc-v0 to point to a version of the crypto-conditions RFC and corresponding implementation and bdb-cc-v0 to shim crypto-conditions with non-RFC features)

• if locks or keys need to be queryable, make sure that they can be indexed in a way that is consistent with the provided scripts. The system should be able to validate that keywords-to-be-indexed (such as public_keys to query unspents) actually match the signature of the lock. This signature will be required

What not to expect

Policy validation

Things like

Status Quo

{
    "id": "<hash of transaction, excluding signatures>",
    "version": "<version number of the transaction model>",
    "transaction": {
       "fulfillments": ["<list of fulfillment>"],
       "conditions": ["<list of condition>"],
       "operation": "<string>",
       "asset": "<asset JSON>",
       "metadata": "<metadata JSON>"
   }
}

Or in expanded form

{
    "id": "<hash of transaction, excluding signatures>",
    "version": "<version number of the transaction model>",
    "transaction": {
        "fulfillments": ["<list of "{
            "owners_before": ["<public key of the owner before the transaction happened>"],
            "fid": 0,
            "fulfillment": "string",
             "input": {
                 "cid": "<input condition index>",
                 "txid": "<input transaction index>"
             }
         }">"],
        "conditions": ["<list of "{
            "cid": "<condition index>",
            "condition": {
                "details": "<json document>",
                "uri": "<string>"
            },
            "owners_after": ["<new owner public key>"],
            "amount": "<int>"
        }">"],
        "operation": "<string>",
        "asset": {
           "id": "<uuid>",
           "divisible": "<true | false>",
           "updatable": "<true | false>",
           "refillable": "<true | false>",
           "data": "<json document>"
        },
        "metadata": {
            "id": "<uuid>",
            "data": "<any JSON document>"
        }
    }
}

Proposed changes

{
    "id": "<hash of transaction, excluding signatures (see explanation)>",
    "version": "<version number of the transaction model>",
    "inputs": ["<list of inputs>"],
    "outputs": ["<list of outputs>"],
    "operation": "<string>",
    "asset": "<asset JSON>",
    "metadata": "<metadata JSON>"
}

Assets

In BigchainDB@v0.8.0, a digital asset entry can be defined upon a CREATE transaction. Here, the future of that asset can be controlled somewhat by specifying the amount of division, whether the asset or the amount can be updated in the future. A great start - and we love to keep going down that road. A few improvement suggestions for the Asset feature include:

1. Separate the definition of the asset and it's tokenization

Problem statement

The definition of an asset is currently bound to the minting of the asset. This means that if you transfer your asset, you lose the control over the asset. Moreover, it becomes tedious to reuse or re-issue new assets with the same definition (and policy).

Improvement Proposal

First of all definitions are in order:

• Asset: a set of Tokens that follow a specific Policy
• Token: a chain (or DAG) of Transactions initiated by a CREATE Transaction.
• Transaction: a Token
• Policy: rules for Tokens

Being able to store

In practice this means that:

• a user wants to issue a digital asset (Token)
• a user wants to specify rules (Policy) for issuing and spending these Tokens
• the System enforces the Policy upon every future Transaction that involves this Token.

In this definition:

• Policy : a set of rules to comply with.
• Token: an instance of a Policy.
• Transactions: a container that allows you to issue and transfer Policies or Tokens
• System: A set of independent auditors that maintain a replicated state machine using consensus.

Some analogies:

• OO-programming: a Policy is a class or a type, whilst a Token is an instance or an object.
• Databases: a Policy is a schema and a Token is an entry (a row) in that schema.

Motivation (vs. BigchainDB@0.8.0 and below)

Renaming Assets to Policy

Separate Asset/Policy definition from issuance

Description and enforcement of Policy rules

Some examples:

ISIN Trading

An exchange platform might want to trade securities, which are defined by a spec called ISIN. The ISIN definition is a schema with mandatory and optional fields. The ISIN could thus be converted to a Policy:

policy_ISIN = Policy(schema=schema_ISIN)

Each security can now be issued as a Token on the blockchain using the ISIN Policy:

security_apple = _Token_(policy=policy_ISIN, )

Tokens as well as the :

• Policy that controls the ownership:
• Policy payload could be an schema of a specification like CoalaIP, ISIN securities, ISBN for books, VIN for cars, an identity schema
• Tokens:

Transactions are a general structure (like a container) that allow you make append-only changes to the truth (ie the blockchain or the transactions in approved blocks). Each transaction has an Ownership layer and a Data layer.

• Ownership layer: provide means to cryptographically Open and Lock a transaction.
• Data layer: Defines or instantiates a Policy

kk

For example:

Transaction

[TimDaub]

@diminator and I just had a quick chat about asset use cases:

Composable Assets or Asset assembly

TL;DR: Different assets (asset1.id !== asset2.id) should be mergable/composable into a new asset (asset3) by applying a conflict-resolving policy to the assets.

Description

WrigleyMate Org. found an exciting and delicious new receipt for mate. All the cool kids love mate. Even more so when all the ingredients are sourced from fair trade and organic dealers. Since WrigleyMate Org. believes in transparency, they decide to track their new mate's supply chain on an immutable and open ledger.

The receipt goes like this:

1. Take a glass bottle
2. 0.5l of water
3. 3g of cinnamon
4. Boil the mixture and infuse 10g mate tea
5. Fill the cooled down mixture in the bottle

On BigchainDB, each of these ingredients is registered separately as a divisible asset by the respective dealers/manufacturers:

bottle_tx = {
    condition: {
        amount: 1
    }
    asset: {
        id: "Qaaabbbccc",
        volume: "0.5",
        manufacturer: "FairTrade Bottle Inc."
    }
}


water_tx = {
    condition: {
        amount: 500
    }
    asset: {
        id: "Qcccdddeee",
        manufacturer: "Organic Water GmbH"
    }
}


cinnamon_tx = {
    condition: {
        amount: 3
    }
    asset: {
        id: "Qdddeeefff",
        manufacturer: "Cinnamondo Europe"
    }
}

mate_tx = {
    condition: {
        amount: 10
    }
    asset: {
        id: "Qeeefffggg",
        manufacturer: "WrigleyMate Org."
    }
}

The following describes WrigleyMate's manufacturing process:

Cinnamondo Europe receives FairTrade Bottle Inc.'s 0.5l bottles as well as water from Organic Water GmbH's. To each bottle, it adds 3g of cinnamon, as well as 0.45l of water.

To represent this in BigchainDB, it needs to support the assembly of assets:

    Qaaabbbccc --------------\ (0.5l bottle)
                              \
        Qdddeeefff ------------\ (3g cinnamon)
                                \
            Qcccdddeee ----------\ (0.45l water)
                                  \
                                  -------- Qfffggghhh (new asset: "Cinnamon mixture")

By mixing together ingredients, we create a new, non-final asset on the fly: a mixture of cinnamon, water and a bottle.

cinnamon_mixture_tx = {
    condition: {
        amount: 1
    }
    asset: {
        id: "Qfffgggyhhh",
        manufacturer: "Cinnamondo Europe"
        composed_from: [
            {
                id: "Qaaabbbccc",
                amount: 1
            },
            {
                id: "Qdddeeefff",
                amount: 3
            },
            {
                id: "Qcccdddeee",
                amount: 450
            }
        ]
}

To proceed, Cinnamondo Europe transfers the mixture (both physically and digitally) to WrigleyMate Org. WrigleyMate Org. boils the mixture and adds 10g of their organic mate tea to it.

    Qaaabbbccc --------------\ (0.5l bottle)
                              \
        Qdddeeefff ------------\ (3g cinnamon)
                                \
            Qcccdddeee ----------\ (0.45l water)
                                  \
                                  -------- Qfffggghhh (new asset: "Cinnamon mixture") -------\
                                                                                              \
                                                                                               \
                                                                            Qeeefffggg ---------\ (10g of mate tea)
                                                                                                 \
                                                                                                 -------- Qggghhhiii (new asset: "Wrigley Mate")

wrigley_mate = {
    condition: {
        amount: 1
    }
    asset: {
        id: "Qggghhhiii",
        manufacturer: "WrigleyMate Org."
        composed_from: [
            {
                id: "Qfffggghhh",
                amount: 1
            },
            {
                id: "Qeeefffggg",
                amount: 10
            }
        ]
}

A lit is put on the bottle. It's ready to be sold to the cool kids on the Block ;).

Edit: This use case is also relevant for partners that want to use COALA IP with SourceRights.
Edit2:: Please note that in this example we made it look like the composed asset can only be exist through composing/spending multiple asset in a new TRANSFER. There is however alternative ways to represent a composable asset, by for example CREATEing a new asset of type=composable (not literally), that links and unlinks to other assets throughout its life-cycle.
Here is a sketch of what was discussed:

[diminator]

Looks great already!

At first sight, it seems that for some cases there should be a conservation of amounts (all in same units ofc). This can be done through summing the input amounts and making sure the output amounts to the same. In this case the amount would be in the input rather than in the asset

cinnamon_mixture_tx = {
    outputs: [{
        amount: 454
    }],
    asset: {
        id: "Qfffgggyhhh",
        manufacturer: "Cinnamondo Europe"
        composed_from: [
            {
                id: "Qaaabbbccc",
            },
            {
                id: "Qdddeeefff",
            },
            {
                id: "Qcccdddeee",
            }
        ]
   }
   inputs: [
        {
             amount: 1
        },
        {
             amount: 3
        },
        {
             amount: 450
        }
   ]
}

[diminator]

Here is a sketch of what was discussed:

👍