Skip to content

Turning Tabular Data into Blocks in a Merkle DAG

flyingzumwalt edited this page Feb 26, 2016 · 1 revision

Converting Tabular Data to JSON in Block Chains (Hypercore feeds)

Imagine we have data like this from a survey of workshop participants:

Type of Experience Little or No Experience Some Experience Very Familiar
Writing software in any programming language 1 5 4
Frontend Web Development 4 3 3
Server-side (“backend”) Web Development 4 4 2
Using Git to track changes and share code (add, commit, push, pull) 2 5 3

If we convert each row into JSON and add each row as a block in a hypercore feed, we get a feed whose content looks like this: (Each block has both a value and a cryptographic hash that is calculated based on the value. These hashes allow us to validate the content of each block. It also allows us to optimize the process of replicating data across the network)

Hash Value
6aa7c09 {"Type of Experience":"Writing software in any programming language","Little/No Experience":1,"Some Experience":5,"Very Familiar":4}
a509f60a {"Type of Experience":"Frontend Web Development","Little/No Experience":4,"Some Experience":3,"Very Familiar":3}
0db6ef7 {"Type of Experience":"Server-side (\“backend\”) Web Development","Little/No Experience":4,"Some Experience":4,"Very Familiar":2}
c3e75b0 {"Type of Experience":"Using Git to track changes and share code (add, commit, push, pull)","Little/No Experience":2,"Some Experience":5,"Very Familiar":3}

The feed itself also gets a hash, say 9cb5a83, so feed 9cb5a83 contains a block chain that looks like this:

 6aa7c09
    |
 a509f60a
    |
 0db6ef7
    |
 c3e75b0 
       \
       9cb5a83 (the feed id)

Adding More Rows

Adding rows is relatively simple. We simply open a new feed based on the original 9cb5a83 feed and then append new blocks. When we're done, our new feed gets its own identifier. Now we have two feeds, which are roughly equivalent to git commits - they each represent a historical checkpoint when changes were applied to the data.

 6aa7c09
    |
 a509f60a
    |
 0db6ef7
    |
 c3e75b0 
       \
       9cb5a83 (the original feed id)
       /
 35844d9
    |
 8e198a1
    |
 d2fe4ed
       \
       ee55467 (the new feed id)

Updating and Deleting Rows

What about updating the values of an existing row, or deleting an existing row? How do we represent that in a feed?