Metadata

Oded Leiba edited this page Nov 9, 2016 · 87 revisions

Metadata and Torrents

The new colored coins protocol supports adding (potentially arbitrary amounts of) metadata to any colored transaction (issuance or transfer). The metadata is in plain JSON format and it's inclusion is optional.

When metadata is included it is not stored directly on the blockchain but rather using the BitTorrent Protocol which gives a decentralized way to share and store data. We use the new DHT trackerless torrents that remove the need for a centralized host for torrent file trackers, further supporting decentralization of our protocol.

The BitTorrent protocol uses a SHA1 hash to reference the file that stores the actual data. This 20 byte SHA1 info_hash is recorded on the blockchain.

BitTorrent, SHA1 and Security

Because SHA1 hashes are no longer considered cryptographically secure this opens up the (somewhat theoretical) possibility that a preimage attack on the torrent info_hash can cause us to load the wrong data from torrents. We provide a simple counter measure to deal with this possibility. The protocol supports recording an additional SHA-256 hash of the metadata on the blockchain. This way we can verify the torrent data integrity before using it. Including the SHA-256 hash of the metadata is optional and metadata recorded on the blockchain without the additional SHA2 for verification can be considered to be of lower security.

Torrent Blockchain Record

The torrent 20 byte SHA1 info_hash referencing the metadata in a colored transaction is currently recorded on the blockchain using a single OP_RETURN output (if 80 bytes are enough) or potentially with one additional (1|2) or (1|3) multisignature output, depending on the coloring scheme and whether the SHA-256 metadata hash is included for additional security.

Torrents and 80 bytes OP_RETURN

Now that the vast majority of the Bitcoin network is accepting 80 bytes OP_RETURN, we may include both the SHA1 torrent info_hash and the SHA-256 metadata hash after the OP_RETURN. In that case it may be beneficial to further compress the SHA-256 using an additional RIPEMD-160 so that also the SHA2 blockchain record size is limited to 20 bytes.

Metadata Structure

Metadata support is one of the new novelties introduced by the new colored coins protocol. There are two basic parts: Static Data and Rules, each discussed in detail in the linked sections. Generally, the data section consists of various static data fields while the rules section encodes an extra layer of logic supported by our rule engine that unlocks smart contracts functionality to colored coins.

Syntax

Static Data goes under the metadata key, Rules go under the Rules key.

{
  metadata: {...Static data goes here...},
  rules: {...Rule definitions go here...}
}

The general metadata syntax is the following:

{
  metadata: {
    assetId: String,
    assetName: String,
    assetGenesis: String,
    issuer: String,
    description: String,
    urls: [
       {name: String, url: String, mimeType: String, dataHash: String },
       {name: String, url: String, mimeType: String, dataHash: String },
       ...
    ], 
    userData : {
      meta: [
        {key: String, value: String, type: String},
        {key: String, value: String, type: String},
        ...
      ],
      user_key: user_value,
      user_key: user_value,
      ...
     },
    encryptions: [
        {key: "user_key", pubKey: 'RSA Public Key',format:'pem|der',type:'pkcs1|pkcs8' },
        ...
    ],
    verifications: {
      social:{
        network_1:{ 
          ....
        },
        network_2:{ 
          ....
        },
       ....
     },
      domain:{
        url:"https://www.example.com/path/to/file/filename.txt"
      },
      signed:{
        message: "We at ... verifying issuance of colored coins asset with asset ID [...].",
        signed_message: "-----BEGIN CMS-----...-----END CMS-----",
        cert: "-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----"
      }
    }
  },
  rules: {
    fees: [{}],
    expiration: {},
    minters: [{}],
    holders: [{}]    
  }
}

All the above keys are optional (except for assetGenesis in case of re-issuance by minters).

Currently, only the hyperlinked keys are recognized by the colored coins code. However, even though the basic colored coins code will ignore other keys, any one can fork the colored coins opensource code and add supporting logic for more keys.

Example

metadata: {
  assetId:'LDJMbzwCBWhrrXpKS7TrCfoAWYgXQhwZg1G6R',
  assetName: "Time Machine",        
  issuer: "Dr. Emmet Brown", 
  description: "The flux capacitor will send us back to the future",
  urls: [
    {
      name: "imdb",
      url: "http://www.imdb.com/title/tt0088763/",
      mimeType: "text/html",
      dataHash: "249e3e3c77d07d8fe8984a47bbbab8c89aeb8b1dadf4e2ff47db42a3e5a1c126"
    }
  ],
  encryptions: [
    {
      key: "Undelrying Physics",
      pubKey: "-----BEGIN RSA PUBLIC KEY-----\nMIIBCgKCAQEA0hw6PRO9RpHRf/pdpEMfD01odzBTaheuA1JxunVTq+/X1hGSUrpRWMIM/tp8\n9DQod6K+6Bo/2CmoZxkWPOk45tbU9QE4Cb532n+MIkzsmbvmM+i49UXSqC8v44MGKTVLb7X2\nPogItSM3lqH4KpZR3cM/JDarfS1R77U/OMDZ/YECDPbcwKPdSLQHhWJ1c9cX5+0lCSDt1WXY\n4XX+hH64C+L/Ss4dMP2kpyHvbsBYpGdLu7AmcDmHtCOl2rXR1z4E0asYGiojw3PI56ATOndS\n30ABKKgQTAExjPQ24BtJYhfJ+zD5zHhztizPPfOwrID2HTfGwVTwfXinV4bpoFfwhwIDAQAB\n-----END RSA PUBLIC KEY-----\n",
      format: "pem",
      type: "pkcs1"
    }
  ],
  verifications: {
    social:{
      twitter:{
        username: 'my_username'
      },
      facebook:{
        page_id: '1232952150'
      },
      github:{       
        gist_id: '6c704f5759927212e714'
      }
    },
    domain:{
      url:"https://www.example.com/digital_assets/assets.txt"
    },
    signed:{
      message: "We at example.com verifying issuance of colored coins asset with asset ID [LJEC6Q2h9JKNvZqEC87TbEXvxm4br1uivb2QX].",
      signed_message: "-----BEGIN CMS-----
MIIFawYJKoZIhvcNAQcCoIIFXDCCBVgCAQExDzANBglghkgBZQMEAgEFADCBgQYJ
KoZIhvcNAQcBoHQEcldlIGF0IGV4YW1wbGUuY29tIHZlcmlmeWluZyBpc3N1YW5j
ZSBvZiBjb2xvcmVkIGNvaW5zIGFzc2V0IHdpdGggYXNzZXQgSUQgW0xKRUM2UTJo
OUpLTnZacUVDODdUYkVYdnhtNGJyMXVpdmIyUVhdLqCCAwMwggL/MIIB56ADAgEC
AgEBMA0GCSqGSIb3DQEBBQUAMBoxCzAJBgNVBAYTAlVTMQswCQYDVQQKDAJaNDAe
Fw0xMzA4MjgxODI4MzRaFw0yMzA4MjgxODI4MzRaMBoxCzAJBgNVBAYTAlVTMQsw
CQYDVQQKDAJaNDCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAN906qi0
d3nlJg7R0vPahd6eDD+1n6rdmY495WYy44whg78K/XCTZTQ4rim6Dg4FIk+GXT1m
zojAHw3A85TsQNOAw5xBRgH/pA0IjUntmbagu25yHPPikhv6jhkCFWmsuFHK+qLw
5MpDuR8Z+zlO7CafUz/R9CR/MzEAOmM4o2B/S7LqU3q62N1Os8ooVRA11zr9PLwR
5OdATBSWxPYsBcJ8QrFOflGVGWMPO1LtJ+CzxUvQU/LVCLwH50VyBFgoWtymxVpn
WUGZcqxcNG7lBH9GDd/0NGrgZHWw0fxEtj24CkyXZI3P6xHjNE8bzlH+x7lDToa7
QFRDdQ+brsRNVksCAwEAAaNQME4wHQYDVR0OBBYEFAdk9MTJBW7/Qj/dBaGVPJbt
ahPNMB8GA1UdIwQYMBaAFAdk9MTJBW7/Qj/dBaGVPJbtahPNMAwGA1UdEwQFMAMB
Af8wDQYJKoZIhvcNAQEFBQADggEBAMfGon3LxqVMftqBUxZl+Jz9Fut28fDKr4g6
uANOirKjTD4hnoEKf2+O/mK6Gq7gWis5YMAWlAplbQMsEkRDL/v/T72mTYM+ErVf
N06i+VKuRG34ZwSkadRefMAJcMFs7T5auT6FyMSRBFErVtbauScBEgQkb0ZL6lKE
/0Gr/QGY2wx6l01wFrTPdrkHR/MXIJChSVfpFOrFHDWHVC3kXTSl+yntKNdUi9hx
7Ado0BJu2jWfmGtLPVVj2EttrXWK8vU3hahZubMAvoFZpuwLYP8x6dJPb9fqFMrI
ZemDcSDaHeIu9S3Bw2fLSFEqy1zvcmX3FHepd9cokfROFOWrSCoxggG1MIIBsQIB
ATAfMBoxCzAJBgNVBAYTAlVTMQswCQYDVQQKDAJaNAIBATANBglghkgBZQMEAgEF
AKBpMBgGCSqGSIb3DQEJAzELBgkqhkiG9w0BBwEwHAYJKoZIhvcNAQkFMQ8XDTE1
MTAxMzA4MjM0N1owLwYJKoZIhvcNAQkEMSIEIEmnhXUqo76ePeBF1RGvV0oCDsGZ
3kF+Z6RvD3eC8lSXMA0GCSqGSIb3DQEBCwUABIIBADWkvjI5Adtlj7MKZ6m8Q20Y
uonGMQBmOJNBIfG2X+nfRFO7FGKT7hGT7Hx5Hkx2oStljdMC1C/orNQUzC7BORuF
JKLktO79a4nSpegEN5X1JK2ZLsifUONup3EgLGNj9NR5AIGlLM7fLVKAAPrOLnlf
mXt5awcgLEztzM+/W7HMPDS+neKGwFtDdjZgNOQjGKehnjWv8vDeslIxVq+FHlAy
0GYd2y14RwTxejIU1mpXRP6I1J8/9OwUNmW8J1oELi9qgx/WdeppXwZma9QLWBU7
qZT5uAGldHG1UJl+sJ266J96wB5TyvCXzdT+G5Pw7as3tn1GIXel/hXIMu0Nq00=
-----END CMS-----",
     cert: "-----BEGIN CERTIFICATE-----
MIIC/zCCAeegAwIBAgIBATANBgkqhkiG9w0BAQUFADAaMQswCQYDVQQGEwJVUzEL
MAkGA1UECgwCWjQwHhcNMTMwODI4MTgyODM0WhcNMjMwODI4MTgyODM0WjAaMQsw
CQYDVQQGEwJVUzELMAkGA1UECgwCWjQwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQDfdOqotHd55SYO0dLz2oXengw/tZ+q3ZmOPeVmMuOMIYO/Cv1wk2U0
OK4pug4OBSJPhl09Zs6IwB8NwPOU7EDTgMOcQUYB/6QNCI1J7Zm2oLtuchzz4pIb
+o4ZAhVprLhRyvqi8OTKQ7kfGfs5Tuwmn1M/0fQkfzMxADpjOKNgf0uy6lN6utjd
TrPKKFUQNdc6/Ty8EeTnQEwUlsT2LAXCfEKxTn5RlRljDztS7Sfgs8VL0FPy1Qi8
B+dFcgRYKFrcpsVaZ1lBmXKsXDRu5QR/Rg3f9DRq4GR1sNH8RLY9uApMl2SNz+sR
4zRPG85R/se5Q06Gu0BUQ3UPm67ETVZLAgMBAAGjUDBOMB0GA1UdDgQWBBQHZPTE
yQVu/0I/3QWhlTyW7WoTzTAfBgNVHSMEGDAWgBQHZPTEyQVu/0I/3QWhlTyW7WoT
zTAMBgNVHRMEBTADAQH/MA0GCSqGSIb3DQEBBQUAA4IBAQDHxqJ9y8alTH7agVMW
Zfic/RbrdvHwyq+IOrgDToqyo0w+IZ6BCn9vjv5iuhqu4ForOWDAFpQKZW0DLBJE
Qy/7/0+9pk2DPhK1XzdOovlSrkRt+GcEpGnUXnzACXDBbO0+Wrk+hcjEkQRRK1bW
2rknARIEJG9GS+pShP9Bq/0BmNsMepdNcBa0z3a5B0fzFyCQoUlX6RTqxRw1h1Qt
5F00pfsp7SjXVIvYcewHaNASbto1n5hrSz1VY9hLba11ivL1N4WoWbmzAL6BWabs
C2D/MenST2/X6hTKyGXpg3Eg2h3iLvUtwcNny0hRKstc73Jl9xR3qXfXKJH0ThTl
q0gq
-----END CERTIFICATE-----"
    }
  },               
  userData :{
    meta: [
      {key: 'Weight', value: 50000,      type: 'Number'},
      {key: 'Model',  value: "Delorean", type: 'String'},       
    ],
    technology: 'flux capacitor 666',
    "Undelrying Physics": 'This magnetic flux calculator calculates the magnetic flux of an object based on the magnitude of the magnetic field which the object emanates and the area of the object, according to the formula, Φ=BA, if the magnetic field is at a 90° angle (perpendicular) to the area of the object. If the magnetic field is not perpendicular to the object, then use the calculator below, which computes the magnetic flux at non-perpendicular angles. The magnetic flux is directly proportional to the magnitude of the magnetic field emanating from the object and the area of the object. The greater the magnetic field, the greater the magnetic flux. Conversely, the smaller the magnetic field, the smaller the flux. The area of the object has the same direct relationship. The greater the area of an object, the greater the flux. Conversely, the smaller the area, the smaller the magnetic flux.'
  } 
}