Skip to content

Interacting with the demo

Codegnosis edited this page Nov 15, 2018 · 34 revisions

Please ensure you have run through Getting Started - Initialising the Demo System first - none of the commands below will work unless the Docker containers are up and running!

Contents

  1. The Babel, Haiku and MOTHER CLI Tools
  2. Babel CLI Tool Overview
    1. Running babel
    2. Viewing Metadata Schemas
    3. Viewing permissions
    4. Viewing UND Balance
    5. Changing Uapp access permissions
    6. Proving permissions and change requests
    7. Transferring UNDs
  3. Haiku CLI Tool Overview
    1. Processing Permission Change requests
    2. Requesting data
    3. Decrypting and Viewing Data
  4. MOTHER CLI Tool Overview
    1. List valid apps according to MOTHER
    2. List invalid apps according to MOTHER
    3. Invalidate an app
    4. Validate an app

The Babel, Haiku and MOTHER CLI Tools

The babel, haiku and mother CLI tools can be used to simulate the behaviour of an end user, and an app's Haiku Server, and Unification respectively.

It's probably a good idea to open a terminal window for each CLI tool (in addition to the one (hopefully!) already open and running the Docker containers). Don't quit the terminal currently running the Docker Composition from the previous steps!

If you have't already, bring up the composition. In a terminal, from the root of the project, run:

./run_docker.sh

Babel CLI Tool Overview

As an end user, babel can be used to view information, such as what data an App is making available and from where it's sourcing and requesting data (e.g. internally from its own database, or from another app); grant or deny access to data (for example, user1 wishes to deny app3 access to any data about them held in app1); view UND balance; and transfer UNDs to another user account.

Running babel

For any of the following commands to work, it's necessary to "log in" to the babel Docker container. In a second terminal, run:

docker exec -it babel /bin/bash

Once inside the container, we can run any of the following commands:

Viewing metadata schemas

It is possible to view the metadata schema templates for the data an app is publishing by running the schemas command:

babel schemas [app_account]

Where [app_account] is the blockchain account name of the app's UApp Smart Contract

for example:

babel schemas app1

will return information about the data app1 is making available in the Unification ecosystem (from its own sources). In this case, app1 is publishing some data from its fitness database.

Sample output:

app1 has the following Schemas:

Schema ID 0:
{
  "fields": [
    {
      "name": "account_name",
      "type": "varchar",
      "is-null": "false",
      "table": "unification_lookup"
    },
    {
      "name": "Heartrate",
      "type": "int",
      "is-null": "true",
      "table": "data_1"
    },
    {
      "name": "GeoLocation",
      "type": "int",
      "is-null": "true",
      "table": "data_1"
    },
    {
      "name": "TimeStamp",
      "type": "int",
      "is-null": "true",
      "table": "data_1"
    },
    {
      "name": "Pulse",
      "type": "int",
      "is-null": "true",
      "table": "data_1"
    }
  ]
}

All of this information is being held in the app's Uapp Smart Contract - babel is querying the Smart Contract. The information is therefore transparent, and a user can see what kind of data (if any) an app is making available to the Unification ecosystem. For example, app1 is making Heart rate, Pulse, Geo-location and an associated Timestamp available.

IMPORTANT: THIS IS NOT THE ACTUAL DATA - IT IS ONLY METADATA, DESCRIBING WHAT TYPE OF DATA CAN BE REQUESTED. REAL DATA IS NOT STORED ON THE BLOCKCHAIN!

Viewing permissions

There are currently three users in the demo environment, imaginatively called user1, user2 and user3, and three apps equally imaginatively called app1, app2, and app3. We can simulate a user viewing the permissions they have granted/revoked for the apps with the permissions command:

babel permissions [user_account]

Where [user_account] is the blockchain account name for the user (for example, user1). This will return the current state of permissions the user has applied to all apps in the demo environment as recorded on the blockchain. For example:

babel permissions user1

simulates user1 viewing their permissions in the ecosystem.

Sample output:

Provider: app1
  Consumer: app2
    Schema ID: 0
      Granted: True
      Fields: Heartrate,GeoLocation,TimeStamp,Pulse
  Consumer: app3
    Schema ID: 0
      Granted: True
      Fields: Heartrate,GeoLocation,TimeStamp,Pulse
Provider: app2
  Consumer: app1
    Schema ID: 0
      Granted: True
      Fields: DataBlob,BlobSize
  Consumer: app3
    Schema ID: 0
      Granted: True
      Fields: DataBlob,BlobSize
Provider: app3
  Consumer: app1
    Schema ID: 0
      Granted: True
      Fields: Image
  Consumer: app2
    Schema ID: 0
      Granted: False

Viewing UND Balance

The balance command can be used to simulate a user viewing their UND Token balance:

babel balance [user_account] [wallet_password]

Where [user_account] is the blockchain account name for the user (for example, user1), and [wallet_password] is the, well, Wallet Password. Here's a list of wallet passwords configured for the demo environment:

user1: PW5KZ2g5KuwVw2QhjNGn9aBbiSGsf3uq5HTigWohM6P7H767kw3dx
user2: PW5JRx3hihCz33R9Tm9uTQGPJj7UFA54nUjcgGkq3PYBy5iQXTsCG
user3: PW5KfhcoCs5yV7wLTWWh97fZbf9jshHZL7vD9tQARfpCGVnDyA95t

For example,

babel balance user2 PW5JRx3hihCz33R9Tm9uTQGPJj7UFA54nUjcgGkq3PYBy5iQXTsCG

will return the UND balance for user2

Sample output:

user2 Balance: 2.0000 UND

Changing UApp access permissions

We can simulate users within the ecosystem granting and revoking access to their data held within apps. Granting and Revoking permission data is stored in a JSON object in IPFS, and the hash along with a Merkle Root hash is stored within the Data Provider App's Smart Contract. The Haiku Node software checks its own permissions lookup with every data request, and both historical and current state of permissions can be queried, traced and proven at any time.

An End User has fine-grained contol over their permissions, meaning that they can grant and revoke access to each individual field specified in a Data Provider's metadata schema.

Permissions can be contolled using either the modify_permissions_direct command, or the interactive modify_permissions command. Both commands generate a permission change request, in the form of a JSON Web Token, which is sent to the Data Provider. The Data Provider's Haiku Node verifies the JWT, validates the fields in the change request payload, and if everything is valid, queues the change request, ready to processes the queue in batches.

Grant Access

The modify_permissions_direct command can be used to simulate a user granting access for a Data Consumer app to access their data in a Data Provider app:

 babel modify_permissions_direct [user_account] [wallet_password] [provider_account] [consumer_Account] [schema_id] [-f field_list]

Note: the schema_id can be found via the babel schemas [provider_account] command

For example, user1 now wishes to grant app3 access to their data held in app2:

 babel modify_permissions_direct user1 PW5KZ2g5KuwVw2QhjNGn9aBbiSGsf3uq5HTigWohM6P7H767kw3dx app2 app3 0 -f DataBlob,BlobSize

Sample output:

user1 is Requesting permission change: Granting access to app3 in Provider app2 for fields DataBlob,BlobSizein Schema 0

Success. Process ID 2 Queued by app2
Request ID: 1
Check status, run:
babel check_change_request user1 1

This permission change request has been sent to app2's Haiku Node for processing. The change request information is also saved in a local BABEL database, so that we can use the babel check_change_request command to prove that the change request has been processed by app2 and is being honoured.

Revoke Access

Similarly, the modify_permissions_direct command can be used to simulate a user revoking access to their data. To revoke access, we just need to omit the -f option, and not pass any fields data:

babel modify_permissions_direct [user_account] [wallet_password] [provider_account] [consumer_account] [schema_id]

For example, user1 wishes to revoke access for app3 to their data held in app1:

 babel modify_permissions_direct user1 PW5KZ2g5KuwVw2QhjNGn9aBbiSGsf3uq5HTigWohM6P7H767kw3dx app1 app3 0

Sample output:

user1 is Requesting permission change: Revoking access to app3 in Provider app1 for Schema 0
Success. Process ID 7 Queued by app1
Request ID: 2
Check status, run:
babel check_change_request user1 2

Interative Changes

Permission change requests can be initiated interactively using the modify_permissions command:

babel modify_permissions [user_account] [wallet_password] [provider_account] [consumer_account]

This allows Schema IDs to be selected, and field level permissions to be toggled before automatically compiling and sending the request.

Proving permissions and change requests

Both the historical and current state of an End User's permissions are recorded on the blockchain, and as such, we can prove whether or not a Data Provider has processed and is honouring an End User's change request, usig the babel check_change_request command:

babel check_change_request [user_account] [request_id]

Where the [request_id] is the ID of the change request stored locally within BABEL. For example, the check whether or not app2 has processed the Grant Access command fomr above, we can run:

babel check_change_request user1 1

Depending on whether or not the batch queue has been processed, we may see either:

Verifying granting access to fields DataBlob,BlobSize in Schema 0 was processed by Provider app2 for Consumer app3...
Permission change request ID 7 has not been processed yet

which indicates that app2 hasn't yet processed the request queue. If the request has been processed, and the permissions are being honoured, the output should be:

Verifying granting access to fields DataBlob,BlobSize in Schema 0 was processed by Provider app2 for Consumer app3...
Request processed in blockchain Tx ID 5826292ff749a06aec943d58b706c44487c48efa0578994a2dadde35cbf9e91d. Checking:
IPFS Hash on blockchain at time of change request: QmUMpsJWRcUL1iDbaWU9bTbBbeAdqKPEGXQyNZ8miaV4YW
Merkle Root on blockchain at time of change request: 12d729e58d588b0cd361b6be7be89584cf1a5f3bca379c2b89c82dfbd9fde6c7
Requesting proof chain from app2
Tx proof verified: True
Verifying granting access to fields DataBlob,BlobSize in Schema 0 is currently honoured by Provider app2 for Consumer app3...
Current IPFS Hash on blockchain: QmUMpsJWRcUL1iDbaWU9bTbBbeAdqKPEGXQyNZ8miaV4YW
Current Merkle Root on blockchain: 12d729e58d588b0cd361b6be7be89584cf1a5f3bca379c2b89c82dfbd9fde6c7
Requesting proof chain from app2
Current proof verified: True

The command queries app2's Haiku Node to see if it has processed the request. If it has, the Node responds with the blockchain transaction ID in which the permission state was modified. BABEL then discovers the IPFS Hash and Merkle Root that was written in this transaction, and sends a second request to the Haiku Node for the Merkle tree proof chain. BABEL then receates the Merkle leaf locally, and verifies it is part of he tree using the root hash obtained from the blockchain, and the proof chain obtained form app2's Haiku Node. If the request was processed and honoured, the output for this stage should be Tx proof verified: True.

The process is then repeated for the current state (i.e. the IPFS Hash and Merkle Root currently stored within app2's UApp smart contract), to check that app2 is currently honouring the premission state. If so, we should see the output Current proof verified: True

Transferring UNDs

Assuming a user has enough UNDs, they can transfer them to another user using babel

babel transfer [from] [to] [amount] [wallet_password]

[amount] can be an integer, or a float (up to 4 decimal places).

For example, user1 transfers a whopping 1 UND to user2

babel transfer user1 user2 1 PW5KZ2g5KuwVw2QhjNGn9aBbiSGsf3uq5HTigWohM6P7H767kw3dx

Sample output:

user1 is transferring 1.0000 UNDto user2:
user1 Old Balance: 4.0000 UND
user2 Old Balance: 2.0000 UND
Transfer result: #    unif.token <= unif.token::transfer         {"from":"user1","to":"user2","quantity":"1.0000 UND","memo":"UND transfer"}
#         user1 <= unif.token::transfer         {"from":"user1","to":"user2","quantity":"1.0000 UND","memo":"UND transfer"}
#         user2 <= unif.token::transfer         {"from":"user1","to":"user2","quantity":"1.0000 UND","memo":"UND transfer"}
user1 New Balance: 3.0000 UND
user2 New Balance: 3.0000 UND

Haiku CLI Tool Overview

The haiku CLI tool can be used to simulate the actions a Haiku Server will undertake in order to request data from another app's Haiku Node, decrypt and view the data, and automatically transfer UND rewards to users/apps when data has been received.

In order to run any of the haiku commands, we need to be within one of the Haiku App Docker containers. Open a new terminal, and run:

docker exec -it haiku-[app_name] /bin/bash

Where [app_name] is app1, app2 or app3, depending on which app is making data requests to other apps, for example:

docker exec -it haiku-app3 /bin/bash

will place us in app3's Haiku Docker container (which is emulating a real App server somewhere)

Processing Permission Change requests

When an End User grants or revokes access to data, the request is received by the Haiku Node and is queued. The process_batch command can be used to simulate a Haiku Node processing the queue of change requests initialised by End Users.

haiku process_batch

This process would normally be run via cron or a similar scheduling tool.

When run, the permission requests queue will be processed. Each change request is verified and the new permissions written to the blockchain.

Requesting data

Requesting data is one of the fundamental aspects of the system. The haiku CLI tool can be used to simulate an app requesting data. Obviously, in the production version, the process of acquiring and validating data will be fully automated once the Haiku Node Server Software is installed and configured, and us running. However, for the prototype/demo, we need to run this process manually.

Fetching a single user's data

We can request data from an app for a single user:

haiku fetch [providing_app] [data_dump_name] --user [user_account]

where [data_dump_name] is any string, which will be used as the filename to save the data to

For example:

haiku fetch app2 app2-user1-data1.dat --user user1

will request data about user1 from app2. If user1 has granted permission for app3 to access data in app2, then the request will be fulfilled: app2 will process the request, check their Smart Contract to see if user1 has granted permission to app3 to access their data, export and encrypt the data, and forward it to app3, who will then reward both user1 and app2 with the amount of UNDs they have specified in their Smart Contract rewards table.

Sample output:

App app3 is requesting data from https://haiku-app2:8050 for user1
Data computationally valid
users_to_pay
['user1']
Pay 1 users
pay user1
Data written to: /tmp/app2-request-app2-user1-data1.dat
View using: haiku view app2 app2-user1-data1.dat

In addition to checking user permissions, the Haiku Node will also call MOTHER (the Unification controlled "central" Smart Contract, deployed on the unif.mother blockchain account in the demo) to see if the requesting app is both valid (according to Unification), and their Smart Contract code hash is valid. Any invalid apps, or apps whose deployed Smart Contract code hash does not match that held in Mother automatically have their data requests rejected, before any user permission query even takes place

Fetching all available data

It is also possible to fetch all available data - for example, app3 can request data from app2 for all users who have granted it permission. The production version will have more fine-grained querying, which will allow data to be fetched, for example, for a certain time period, or country (if that data has been made available by the providing app). For the purposes of the prototype, we're just grabbing all available data:

haiku fetch [providing_app] [data_dump_name]

for example

haiku fetch app2 app2-data1.dat

Sample output:

App app3 is requesting data from https://haiku-app2:8050 for all users
Data computationally valid
users_to_pay
['user1', 'user2', 'user3']
Pay 3 users
pay user1
pay user2
pay user3
Data written to: /tmp/app2-request-app2-data1.dat
View using: haiku view app2 app2-data1.dat

If an End User has completely revoked access to their data (i.e. no permission on any fields), then they will not be included in the data export at all.

Note: An app can't request data from itself - the request will always fail.

Decrypting and Viewing Data

Once data has been received, it's stored locally in its encrypted format. In order to actually view that data, the Haiku must use its private key to decrypt the data (well, technically, it's using its private key to decrypt the random key used to encrypt the actual data). This can be simulated using the view command:

haiku view [providing_app] [data_dump_name]

where [data_dump_name] matches what was passed to the previous fetch command

For example:

haiku view app2 app2-user1-data1.dat

will decrypt and output the data we requested in the previous fetch command example, to the console.

Sample output:

App app3 is reading ingested data from app2
{'data': [{'DataBlob': '4D2AFF4D2AFF4D2AFF4D2AFF', 'BlobSize': 40, 'account_name': 'user1'}, {'DataBlob': '4D2AFF4DFF4D2AFF', 'BlobSize': 50, 'account_name': 'user1'}, {'DataBlob': '4D2AFF4D2AFF4D2AFF4D2AFFBDC2F21', 'BlobSize': 89, 'account_name': 'user1'}, {'DataBlob': '2AFF2AFF2AFF4D2AFF4D2AFF4D2AFF4D2AFF', 'BlobSize': 101, 'account_name': 'user1'}], 'schema': {'fields': [{'name': 'account_name', 'type': 'varchar', 'is-null': 'false', 'table': 'BlobData'}, {'name': 'DataBlob', 'type': 'binarydata', 'is-null': 'true', 'table': 'BlobData'}, {'name': 'BlobSize', 'type': 'int', 'is-null': 'true', 'table': 'BlobData'}]}, 'unification_users': ['user1']}

If an End User has completely revoked access to their data, then there will be no data included for that user. If they have granted access to some fields, and revoked access to others, then the revoked fields will contain null values.

MOTHER CLI Tool Overview

The mother CLI tool can be used to simulate Unification validating and invalidating apps, and can be used to get the current status of apps. For brevity, this CLI tool can be run within the babel Docker container:

docker exec -it babel /bin/bash

List valid apps according to MOTHER

We can quickly query the MOTHER Smart Contract to get a list of valid apps within the Unification ecosystem, and their associated metadata:

mother validapps

Sample output

Valid apps according to MOTHER:
app1
Contract Hash: d151d05c06cb93f1a56e75c3a177e009f1df8f758c6e28739a95382737ca6bdf
RPC Server: http://haiku-app1:8050
MOTHER Signature: SIG_K1_KkbxZpLhZva447QWLLFQ5eA5CiyvXJhu3JDCQsBkvRigxfhEGZFNZK9A2TVRyaA4VKX5uvF3eE3ZyMQStawYUZJUpmmHn2
is valid: true
app2
Contract Hash: d151d05c06cb93f1a56e75c3a177e009f1df8f758c6e28739a95382737ca6bdf
RPC Server: http://haiku-app2:8050
MOTHER Signature: SIG_K1_KA72yTcuVrp8qUi1jw2hLsB8WDyuQsCrirrUnJ7MpfN1jzD2wfdCFRjavAoZ27ocF56PxdDvNKkVjcBWBvewoLV2u9TgM9
is valid: true
app3
Contract Hash: d151d05c06cb93f1a56e75c3a177e009f1df8f758c6e28739a95382737ca6bdf
RPC Server: http://haiku-app3:8050
MOTHER Signature: SIG_K1_KAH3VNPzp2kSmueyNuAr1Ko4ReKugRr3xPSzcPeaHP8QEppYzf2Uh46NKVpSzNiRfgqw9PU6w9dVhGZLSi1ALxeZjSf7Qp
is valid: true

List Invalid apps according to MOTHER

Similarly, we can also quickly query the MOTHER Smart Contract to get a list of invalid apps within the Unification ecosystem, and their associated metadata:

mother invalidapps

Sample output:

Valid apps according to MOTHER:
app3
Contract Hash: d151d05c06cb93f1a56e75c3a177e009f1df8f758c6e28739a95382737ca6bdf
RPC Server: http://haiku-app3:8050
MOTHER Signature: SIG_K1_KAH3VNPzp2kSmueyNuAr1Ko4ReKugRr3xPSzcPeaHP8QEppYzf2Uh46NKVpSzNiRfgqw9PU6w9dVhGZLSi1ALxeZjSf7Qp
is valid: false

Invalidate an app

We can simulate Unification invalidating an app in the ecosystem, thereby preventing the app from both providing and requesting any data:

mother invalidate [app_name] [mother_wallet_password]

For example:

mother invalidate app3 PW5JN14rVAQ4oL19PREuJbDjbde6QfrRmWCXsoBq8866KdxvWYJSH

will invalidate app3 in the ecosystem. It will no longer be able to provide data to, or request data from other apps in the ecosystem.

Validate an app

We can also simulate Unification validating an existing app (which was previously invalidated)

mother validate [app_name] [mother_wallet_password]

for example

mother validate app3 PW5JN14rVAQ4oL19PREuJbDjbde6QfrRmWCXsoBq8866KdxvWYJSH

Will re-validate app3, therefore allowing it to both provide, and request data from other apps

Now we know how to interact with the system, on to a Full Demo Scenario

You can’t perform that action at this time.