zArchive~Protocol Documentation

NeuralMiner edited this page Oct 18, 2018 · 1 revision

This section is under construction. It is heavily based on https://en.bitcoin.it/wiki/Protocol_documentation. This page describes the behavior of the Original Gridcoin wallet. The Gridcoin protocol is specified by the behavior of the reference client, not by this page. In particular, while this page is quite complete in describing the network protocol, it does not attempt to list all of the rules for block or transaction validity.

Type names used in this documentation are from the C99 standard.

Common standards

Hashes

Usually, when a hash is computed within Gridcoin, it is computed twice. Most of the time SHA-256 hashes are used, however RIPEMD-160 is also used when a shorter hash is desirable (for example when creating a bitcoin address).

Example of double-SHA-256 encoding of string "hello":

hello
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 (first round of sha-256)
9595c9df90075148eb06860365df33584b75bff782a510c6cd4883a419833d50 (second round of sha-256)

For Gridcoin addresses (RIPEMD-160) this would give:

hello
2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 (first round is sha-256)
b6a9c8c230722b7c748331a8b450f05566dc7d0f (with ripemd-160)

Merkle Trees

Merkle trees are binary trees of hashes. Merkle trees in Gridcoin use a double SHA-256, the SHA-256 hash of the SHA-256 hash of something.

If, when forming a row in the tree (other than the root of the tree), it would have an odd number of elements, the final double-hash is duplicated to ensure that the row has an even number of hashes.

First form the bottom row of the tree with the ordered double-SHA-256 hashes of the byte streams of the transactions in the block.

Then the row above it consists of half that number of hashes. Each entry is the double-SHA-256 of the 64-byte concatenation of the corresponding two hashes below it in the tree.

This procedure repeats recursively until we reach a row consisting of just a single double-hash. This is the Merkle root of the tree.

For example, imagine a block with three transactions a, b and c. The Merkle tree is:

d1 = dhash(a)
d2 = dhash(b)
d3 = dhash(c)
d4 = dhash(c)            # a, b, c are 3. that's an odd number, so we take the c twice

d5 = dhash(d1 concat d2)
d6 = dhash(d3 concat d4)

d7 = dhash(d5 concat d6)

where

dhash(a) = sha256(sha256(a))

d7 is the Merkle root of the 3 transactions in this block.

Note: Hashes in Merkle Tree displayed in the block explorer are of little-endian notation. For some implementations and calculations, the bytes need to be reversed before they are hashed, and again after the hashing operation.

Signatures

Gridcoin uses Elliptic Curve Digital Signature Algorithm (ECDSA) to sign transactions.

For ECDSA the secp256k1 curve from http://www.secg.org/sec2-v2.pdf is used.

Public keys (in scripts) are given as 04 where x and y are 32 byte big-endian integers representing the coordinates of a point on the curve or in compressed form given as where is 0x02 if y is even and 0x03 if y is odd.

Signatures use DER encoding to pack the r and s components into a single byte stream (this is also what OpenSSL produces by default).

Transaction Verification

Transactions are cryptographically signed records that reassign ownership of Gridcoins to new addresses. Transactions have inputs - records which reference the funds from other previous transactions - and outputs - records which determine the new owner of the transferred Gridcoins, and which will be referenced as inputs in future transactions as those funds are respent.

Each input must have a cryptographic digital signature that unlocks the funds from the prior transaction. Only the person possessing the appropriate private key is able to create a satisfactory signature; this in effect ensures that funds can only be spent by their owners.

Each output determines which Gridcoin address (or other criteria) is the recipient of the funds.

In a transaction, the sum of all inputs must be equal to or greater than the sum of all outputs. If the inputs exceed the outputs, the difference is considered a transaction fee, and is redeemable by whoever first includes the transaction into the block chain.

A special kind of transaction, called a coinbase transaction, has no inputs. It is created by minters, and there is one coinbase transaction per block. Because each block comes with a reward of newly created Gridcoins, the first transaction of a block is, with few exceptions, the transaction that grants those coins to their recipient (the minter). In addition to the newly created Gridcoins, the coinbase transaction is also used for assigning the recipient of any transaction fees that were paid within the other transactions being included in the same block. The coinbase transaction can assign the entire reward to a single Gridcoin address, or split it in portions among multiple addresses, just like any other transaction. Coinbase transactions always contain outputs totalling the sum of the block reward plus all transaction fees collected from the other transactions in the same block.

Most Gridcoin outputs encumber the newly transferred coins with a single ECDSA private key. The actual record saved with inputs and outputs isn't necessarily a key, but a script. Gridcoin uses an interpreted scripting system to determine whether an output's criteria have been satisfied, with which more complex operations are possible, such as outputs that require two ECDSA signatures, or two-of-three-signature schemes. An output that references a single Gridcoin address is a typical output; an output actually contains this information in the form of a script that requires a single ECDSA signature. The output script specifies what must be provided to unlock the funds later, and when the time comes in the future to spend the transaction in another input, that input must provide all of the thing(s) that satisfy the requirements defined by the original output script.

Addresses

A Gridcoin address is in fact the hash of a ECDSA public key, computed this way:

Version = Mainnet: 0x3e; Testnet: #TODO
Key hash = Version concatenated with RIPEMD-160(SHA-256(public key))
Checksum = 1st 4 bytes of SHA-256(SHA-256(Key hash))
Gridcoin Address = Base58Encode(Key hash concatenated with Checksum)

The Base58 encoding used is home made, and has some differences. Especially, leading zeroes are kept as single zeroes when conversion happens.

Common structures

Almost all integers are encoded in little endian. Only IP or port number are encoded big endian.

Message structure

See 1.

Field Size Description Data type Comments
4 magic uint32_t Magic value indicating message origin network, and used to seek to next message when stream state is unknown
12 command char[12] ASCII string identifying the packet content, NULL padded (non-NULL padding results in packet rejected)
4 size uint32_t Size of payload in number of bytes
4 checksum uint32_t First 4 bytes of sha256(sha256(payload))
? payload uchar[] The actual data

Known magic values:

Network Magic value Sent over wire as
main 0x70352205 2 70 35 22 05
testnet 0xCDF2C0EF 3 CD F2 C0 EF

Variable length integer

Integer can be encoded depending on the represented value to save space. Variable length integers always precede an array/vector of a type of data that may vary in length. Longer numbers are encoded in little endian.

Value Storage length Format
< 0xFD 1 uint8_t
<= 0xFFFF 3 0xFD followed by the length as uint16_t
<= 0xFFFF FFFF 5 0xFE followed by the length as uint32_t
- 9 0xFF followed by the length as uint64_t

Variable length string

Variable length string can be stored using a variable length integer followed by the string itself.

Field Size Description Data type Comments
? length var_int Length of the string
? string char[] The string itself (can be empty)

Network address

When a network address is needed somewhere, this structure is used. Network addresses are not prefixed with a timestamp in the version message.

Field Size Description Data type Comments
4 time uint32 the Time (version >= 31402). Not present in version message.
8 services uint64_t same service(s) listed in version
16 IPv6/4 char[16] IPv6 address. Network byte order. The original client only supported IPv4 and only read the last 4 bytes to get the IPv4 address. However, the IPv4 address is written into the message as a 16 byte IPv4-mapped IPv6 address (12 bytes 00 00 00 00 00 00 00 00 00 00 FF FF, followed by the 4 bytes of the IPv4 address).
2 port uint16_t port number, network byte order

Hexdump example of Network address structure

0000   01 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ................
0010   00 00 FF FF 0A 00 00 01  20 8D                    ........ .

Network address:
 01 00 00 00 00 00 00 00                         - 1 (NODE_NETWORK: see services listed under version command)
 00 00 00 00 00 00 00 00 00 00 FF FF 0A 00 00 01 - IPv6: ::ffff:a00:1 or IPv4: 10.0.0.1
 20 8D                                           - Port 8333

Inventory Vectors

Inventory vectors are used for notifying other nodes about objects they have or data which is being requested.

Inventory vectors consist of the following data format:

Field Size Description Data type Comments
4 type uint32_t Identifies the object type linked to this inventory
32 hash char[32] Hash of the object

The object type is currently defined as one of the following possibilities:

Value Name Description
0 ERROR Any data of with this number may be ignored
1 MSG_TX Hash is related to a transaction
2 MSG_BLOCK Hash is related to a data block
3 MSG_FILTERED_BLOCK Hash of a block header; identical to MSG_BLOCK. Only to be used in getdata message. Indicates the reply should be a merkleblock message rather than a block message; this only works if a bloom filter has been set.
4 MSG_CMPCT_BLOCK Hash of a block header; identical to MSG_BLOCK. Only to be used in getdata message. Indicates the reply should be a cmpctblock message. See BIP 152 for more info.

Other Data Type values are considered reserved for future implementations.

Block Headers

Block headers are sent in a headers packet in response to a getheaders message.

Field Size Description Data type Comments
4 version int32_t Block version information (note, this is signed)
32 prev_block char[32] The hash value of the previous block this particular block references
32 merkle_root char[32] The reference to a Merkle tree collection which is a hash of all transactions related to this block
4 timestamp uint32_t A timestamp recording when this block was created (Will overflow in 2106[1])
4 bits uint32_t The calculated difficulty target being used for this block
4 nonce uint32_t The nonce used to generate this block… to allow variations of the header and compute different hashes
1 txn_count var_int Number of transaction entries, this value is always 0

Differential encoding

Several uses of CompactSize below are "differentially encoded". For these, instead of using raw indexes, the number encoded is the difference between the current index and the previous index, minus one. For example, a first index of 0 implies a real index of 0, a second index of 0 thereafter refers to a real index of 1, etc.

PrefilledTransaction

A PrefilledTransaction structure is used in HeaderAndShortIDs to provide a list of a few transactions explicitly.

Field Name Type Size Encoding Purpose
index CompactSize 1, 3 bytes Compact Size, differentially encoded since the last PrefilledTransaction in a list The index into the block at which this transaction is
tx Transaction variable As encoded in tx messages The transaction which is in the block at index index.

See BIP 152 for more information.

HeaderAndShortIDs

A HeaderAndShortIDs structure is used to relay a block header, the short transactions IDs used for matching already-available transactions, and a select few transactions which we expect a peer may be missing.

Field Name Type Size Encoding Purpose
header Block header 80 bytes First 80 bytes of the block as defined by the encoding used by "block" messages The header of the block being provided
nonce uint64_t 8 bytes Little Endian A nonce for use in short transaction ID calculations
shortids_length CompactSize 1 or 3 bytes As used to encode array lengths elsewhere The number of short transaction IDs in shortids (ie block tx count - prefilledtxn_length)
shortids List of 6-byte integers 6*shortids_length bytes Little Endian The short transaction IDs calculated from the transactions which were not provided explicitly in prefilledtxn
prefilledtxn_length CompactSize 1 or 3 bytes As used to encode array lengths elsewhere The number of prefilled transactions in prefilledtxn (ie block tx count - shortids_length)
prefilledtxn List of PrefilledTransactions variable size*prefilledtxn_length As defined by PrefilledTransaction definition, above Used to provide the coinbase transaction and a select few which we expect a peer may be missing

See BIP 152 for more information.

BlockTransactionsRequest

A BlockTransactionsRequest structure is used to list transaction indexes in a block being requested.

Field Name Type Size Encoding Purpose
blockhash Binary blob 32 bytes The output from a double-SHA256 of the block header, as used elsewhere The blockhash of the block which the transactions being requested are in
indexes_length CompactSize 1 or 3 bytes As used to encode array lengths elsewhere The number of transactions being requested
indexes List of CompactSizes 1 or 3 bytes*indexes_length Differentially encoded The indexes of the transactions being requested in the block

See BIP 152 for more information.

BlockTransactions

A BlockTransactions structure is used to provide some of the transactions in a block, as requested.

Field Name Type Size Encoding Purpose
blockhash Binary blob 32 bytes The output from a double-SHA256 of the block header, as used elsewhere The blockhash of the block which the transactions being provided are in
transactions_length CompactSize 1 or 3 bytes As used to encode array lengths elsewhere The number of transactions provided
transactions List of Transactions variable As encoded in tx messages The transactions provided

See BIP 152 for more information.

Short transaction ID

Short transaction IDs are used to represent a transaction without sending a full 256-bit hash. They are calculated by:

  1. single-SHA256 hashing the block header with the nonce appended (in little-endian)
  2. Running SipHash-2-4 with the input being the transaction ID and the keys (k0/k1) set to the first two little-endian 64-bit integers from the above hash, respectively.
  3. Dropping the 2 most significant bytes from the SipHash output to make it 6 bytes.

See BIP 152 for more information.

Message types

version / aries

When a node creates an outgoing connection, it will immediately advertise its version. The remote node will respond with its version. No further communication is possible until both peers have exchanged their version.

Payload 4:

Field Size Description Data type Comments
4 version int32_t Identifies protocol version being used by the node
32 nonce string 256 random bytes encoded as hex string
16 pw1 string md5(nonce+","+sboinchashargs)
16 mycpid string The CPID of the node sending this message
16 mycpid string The CPID of the node sending this message
TODO acid string Command nonce
8 nLocalServices uint64_t (1 << 0)
8 nTime uint64_t UNIX Timestamp of current time
TODO addrYou TODO TODO
TODO addrMe TODO TODO
8 nLocalHostNonce uint64_t 8 random bytes
? FormatSubVersion string Clientversion 5
4 nBestHeight int Current blockheight
34 sGRCAddress string Your Gridcoin address

A "verack" packet shall be sent if the version packet was accepted.

  1. https://en.wikipedia.org/wiki/Unix_time#Notable_events_in_Unix_time
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.