SMIP: Spacemesh Reference Wallets - File Format and Accounts Derivation

[avive]

Motivation

We'd like to formalize how the reference Spacemesh wallets create accounts, sign transactions, manage wallets and accounts and persist wallet and accounts data to wallet file. This will allow different wallets such as CLIWallet and Smapp to use wallets created by any of these apps. Currently, the CLIWallet file format is different than Smapp's plus we don't encrypt the data at rest in CLIWallet. This spec will also allow the community to write their own Spacemesh wallets. It is also important for building hardware wallets support for Spacemesh coins.

Implementation Notes

Smapp already implements this SMIP.

Spacemesh app wallet structure

Wallet File Name

Wallet file name pattern is: my_wallet__<creation_timestamp>.json.

• Creation timestamp - An ISO date string with : chars replaced with -.

Example file name: my_wallet_2020-05-26T09-53-06.105Z.json.

---

Wallet File Contents - Json Syntax

{
  "meta": {
    "displayName": <string>,
    "created": <string>,
    "netId": <integer>,
    "remoteApi": <string>,
    "hd_id": <integer>,
    "meta": {
        "salt": "Spacemesh blockmesh"
    }
  },
  "crypto": {
    "cipher": "AES-128-CTR",
    "cipherText": <string>
  },
  "contacts": [
    {
        "nickname": "Joe Doe",
        "address": "0afef01..."
    },
    ...
  ]
}

• displayName - Wallet's display name, up to 50 chars.
• created - Creation timestamp - same format as the timestamp in the wallet's file name.
• netId - Network id that this wallet is designated to work with. Currently unused. It will be used later to match a wallet file to a specific Spacemesh network. Legal values are 0-255.
• hd_id - 0: a hot wallet. 1: a ledger wallet. 2: tezos wallet. When field is excluded, assume value is 0 - a whot wallet.
• cipher - The name of the encryption algorithm used to encrypt and to decrypt the secret part of the wallet (cipher text). Currently only AES-128-CTR is supported.
• cipherText - Encrypted wallet's data. Hex string.
• contacts - array of contact objects. Contact object syntax:
  • nickname - Contact friendly name. String up to 50 characters.
  • address - 40 hex chars representing contact's 20 bytes address.

---

A decrypted cipherText data has the following json syntax:

{
    mnemonic: <string>,
    accounts: [
        {
            displayName: <string>,
            created: <string>,
            path: <string>,
            publicKey: <string>,
            secretKey: <string>
        },
        ...
    ]
}

• mnemonic - 12 words concatenated in single string with single space between each word. This field should be excluded or value set to the empty string when wallet is a hardware wallet. e.g. "hd_id" != 0.
• accounts - array of account objects. Each object consists of:
  • displayName - account name up to 50 chars.
  • created - creation timestamp in the same format as in file name and the wallet's created timestamp field.
  • path - string in the form 0/0/<account number>, where account number is integer representing serial number of accounts created in this wallet.
  • publicKey - hex string of the account's public key byte array.
  • secretKey - hex string of the account's secret key byte array. This field should be excluded or value set to the empty string when wallet is a hardware wallet. e.g. "hd_id" != 0 as the secretKey is only available in the hardware wallet.

---

Minimal Wallet Json Example

{
    "meta":  {
        "displayName":"Main Wallet",
        "created": "2020-05-24T12-56-44.897Z",
        "netId": 0,
        "meta":{
            "salt": "Spacemesh blockmesh"
        }
    },
    "crypto": {
        "cipher":"AES-128-CTR",
        "cipherText": "0fbd900ce9cc02682..."
    },
    "contacts":[]
}

TODO: add example of decrypted cipher text with 2 accounts here...

---

Mnemonic and accounts key-pair generation

• The nemonic is created using the [bip39] spec.
• The library used in Smapp is bip39.
• The mnemonic is generated by calling bip39.generateMnemonic. Result is 12 words string separated by white spaces.
• Use bip39.validateMnemonic to validate a mnemonic.

Account Generation

• Call bip39.mnemonicToSeedSync with a mnemonic to generate a seed.

• The signature scheme used to generate signing keys and to derive new pairs from the seed is ed25519. Note that this is a custom ed25519 signature scheme developed by Spacemesh and not the standard ed25519 signature scheme.

• To create user accounts in the wallet, use ed25519.NewDerivedKeyFromSeed to create a new key pair, with the following params:

  1. seed - From calling bip39.mnemonicToSeedSync.
  2. index - Account's serial number. e.g 0, 1, 2...
  3. salt - String "Spacemesh blockmesh" encoding using UTF-8 to a byte array.

The first wallet's account should be the derived account index 0, the second at 1, etc... Do not use the a master key-pair generated from the seed directly for user accounts. User accounts should always be used using ed25519.NewDerivedKeyFromSeed.

Spacemesh BIP32 paths

m / 44' / coin_type / [account] / 0 / [address_index]`

• Account should be constant 0 for now. May be used in the future for additional functionality.
• Coin type is 1 for a Spacemesh Testnet and 540 for Spacemesh mainent.
• The first mainent account should be at path m / 44' / 540' / 0' / 0 / 0'.
• The second mainent account should be at path m / 44' / 540' / 0' / 0 / 1'.
• The first testnet account should be at path `m / 44' / 1' / 0' / 0 / 0'.
• The second testnet account should be at path `m / 44' / 1' / 0' / 0 / 1'.
• We have registered to have 540 for Spacemesh. see: Add Spacemesh coin to slip 0044 satoshilabs/slips#943
• address_index is int32 so there can be up to 2^32 addresses per wallet.
• All path components should be hardened, indicated by an apostrophe char ' per bip32.

Wallet Creation Flows

Obtain Encryption / Decryption Key from User's Password

1. User provides a password - at least 8 unicode characters.
2. We use pbkdf2 to derive an encryption key using the following method:

pbkdf2.pbkdf2Sync(password, "Spacemesh blockmesh", 1000000, 32, "sha512")

This key is used to encrypt and decrypt the wallet's file cipher-text binary data.

Encrypt and Decrypt Wallet data

Only AES is currently supported and we use aes-js in the following way:

Encryption

1. Call aes.utils.utf8.toBytes on stringified json data to be encrypted to get the byte array representation of the data.
2. Call 'aes.ModeOfOperation.ctr(key, new aes.Counter(5))wherekey` is the encryption key we have created.
3. Call aes.encrypt() with byte array from step 1.
4. Call aes.utils.hex.fromBytes on the encrypted data result to get hex string representation to be saved in the wallet's file.

Decryption

1. Call aes.ModeOfOperation.ctr(key, new aes.Counter(5))wherekey` is the encryption key we created before.
2. Call aes.utils.hex.toBytes on the encrypted hex string to get bytes array representation of the encrypted data.
3. Call aes.decrypt to decrypt the bytes array.
4. Call aes.utils.utf8.fromBytes to get a json stringified representation of decrypted wallet data.
5. Get the json data from the json string.

[avive]

@IlyaVi - we may want to move the contacts to inside ciphertext for increased privacy. The contacts is linking an address to a potential personally identifiable information.

[avive]

Hardware Wallet Support

A spacemesh wallet can be backed by a hardware wallet (e.g. Ledger wallet) or be a hot wallet where the mnemonic is stored in the wallet file.
When a wallet is created for a hardware wallet, the following is changed in the wallet file format:

Meta field

"meta": {
    ...
    "hd_id": [integer]
  },

We add thehd_id filed to indicate the type of the backing hardware wallet. e.g. 0 - None (hot wallet). 1 - Ledger. 2 - Tezos. All wallets should have this field, if it is omitted we assume the wallet file is for a hot wallet.

The wallet's ciphertext syntax for a hardware wallet doesn't include the mnemonic field as it is only stored in the hardware wallet and we don't store specific accounts secret keys. e.g:

{
    accounts: [
        {
            displayName: <string>,
            created: <string>,
            path: <string>,
            publicKey: <string>,
        },
        ...
    ]
}

So we have an entry for each account created by the user which is backed by the hardware wallet, as well the public key of that account (obtained from the wallet). The path field specifies what account to use for signing when user the hardware wallet's api.

[DaveAppleton]

Can we have a reference wallet with expected accounts for testing purposes?

[avive]

I think we can use smapp as the reference wallet as it implements support for this file format. In go-spacemesh we have 2 hard-coded accounts with balances for tests which are not used in public testnet. We can create a reference wallet file with these account - good idea. @IlyaVi

[avive]

@IlyaVi - we need to update this spec with the contacts now in cyphertext instead of cleartext in the current smapp implementation. Also - please review that this spec accurately describes the smapp implementation..

[avive]

@IlyaVi - please see my last comment - this becomes a priority as we would like to have CLIWallet compatibility with smapp for release 0.2

[DaveAppleton]

So for clarity : cipherText now contains

{
    mnemonic: <string>,
    accounts: [
        {
            displayName: <string>,
            created: <string>,
            path: <string>,
            publicKey: <string>,
            secretKey: <string>
        },
        ...
    ],
    contacts:[]
}

[DaveAppleton]

And the Minimum Wallet Example is

{
    "meta":  {
        "displayName":"Main Wallet",
        "created": "2020-05-24T12-56-44.897Z",
        "netId": 0,
        "meta":{
            "salt": "Spacemesh blockmesh"
        }
    },
    "crypto": {
        "cipher":"AES-128-CTR",
        "cipherText": "0fbd900ce9cc02682..."
    }
}

[avive]

I've updated the main smip to consider the hardware-wallet related fields - this info was only in a comment.

[brusherru]

Regarding spacemeshos/smapp#781 and discussion with @avive I have to update the wallet structure presented in this SMIP. Added:

{
  ...
  "meta": {
    ...
    "remoteApi": <string>,
  }
}

Since it is a newly added property we had to assume that this property may not persist in the wallet file, so the developers should default it to the empty string for the local node or "host:port" (E.G. "192.168.1.1:30310") for the wallet only mode.

[brusherru]

Due to the brand-new transactions and accounts, we need to tweak the wallet file structure.

So in the new version (v0.3), we will have keypairs (that were stored under the "accounts" key, but let's name them keypairs to avoid misunderstanding since these keypairs will be used only for signing/verifying instead of having their own balances), which may be used in accounts ("standard" ones: singlesig / multisig / vesting / vault, and 3rd-party ones in post genesis).

So I propose to rename the current account into keychain and keep it the same and introduce the brand-new accounts:

keychain: [
     {
            displayName: <string>,
            created: <string>,
            path: <string>,
            publicKey: <string>,
            secretKey: <string>
     }
],
accounts: [
     {
            displayName: <string>,
            address: <bech32string>,
            templateAddress?: <bech32string>,
            spawnArguments?: <any>
     }
]

keys with the question mark are optional

Meanwhile, templateAddress and spawnArguments are required to allow the User to sign transactions (his public key should be listed in the spawnArguments).
Since accounts could be in a "not spawned" state — then it might be impossible to retrieve the template addresses and spawn arguments from the network. That's why it might be important to store these data in the wallet file.

In addition to it, spawn arguments can be retrieved from the network only in the case the account represents a standard type (uses a built-in template, and this is valid for genesis, but we'd like to have custom templates in the future). As a consequence, it might be impossible to decode spawn arguments retrieved from the network without additional User interaction or other services. In other words, the user will need to provide a codec for the template, or the Template developer should register it in some public registry.

@dshulyak @countvonzero @pigmej @lrettig @noamnelke, please check out my proposal and tell me if I miss or overcomplicate something :) In case it seems good — please let me know as well (thumbs up emoji will be enough) ;)

[jonZlotnik]

So, to clarify:
The keychain list will contain all keypairs? (those which have balances associated with them and those that do not)
And the accounts list will contain all addresses (encoded public keys) which have balances?

Do you think we could remove the "display name" from either the keychain or the accounts?
It seems redundant as the identifiers that allow you to correlate the templateAddress and spawnArguments with the appropriate keypair are the address, and publicKey

This JSON structure won't be read by humans anyways because it should always be encrypted at rest.

[brusherru]

Yes, that's right.
Just in case, I'll add this:
The key pairs themselves do not have their own balances. Only accounts may have balances.
And one keypair may be used in a few accounts (associated with). It is not a best practice, but it is possible :)
E.G. I can use one key pair for my single-sig account, and for a couple of multi-sig accounts.

Do you think we could remove the "display name" from either the keychain or the accounts?

I'd like to get rid of all redundant data there, but I'm not sure it will be handy for Users if we will get rid of one of displayName. But probably we can get rid of created field without losing usability :)

This JSON structure won't be read by humans anyways because it should always be encrypted at rest.

That's correct. But this JSON structure is also used in the client apps (which decrypts it), and the client app shows such metadata to the User ;)

[brusherru]

Btw, I don't how the hardware wallet keypairs will be integrated there.
Probably we'll have a bit different kind of keypair shape for it (aka there will be no secret key, but something else).
But I believe we can leave these details outside of this discussion for a while :)

[jonZlotnik]

The key pairs themselves do not have their own balances. Only accounts may have balances.
And one keypair may be used in a few accounts (associated with). It is not a best practice, but it is possible :)
E.G. I can use one key pair for my single-sig account, and for a couple of multi-sig accounts.

Btw, I don't how the hardware wallet keypairs will be integrated there.
Probably we'll have a bit different kind of keypair shape for it (aka there will be no secret key, but something else).

In relation to these two points, I think the best thing to do for now, would be to define a "core wallet file" which contains only the required data to successfully use the wallet. (I'll take some time today to define what might look like)
And then everything else:

• fields for data/compute tradeoffs for optimization of keypair retrieval (storing actual keypairs instead of deriving them on the fly)
• storing metadata like names, date created, etc.
• contract/template/account relationships
• transaction history
• last known balances
• contacts
• etc.

Can be stored in a separate data structure or even in separate files. This way, the core data required for every wallet (light clients, full nodes, archival nodes, anything else...) is standardized, and then the format of the "other stuff" can be suggested to, and interpreted by, the wallet/client developer.

[brusherru]

I had a conversation today with @pigmej about it. And I miss one important thing — all account types except SingleSig are scheduled for post genesis. In this case, current wallet file structure is enough.
So let's postpone this thing and stop spending time on it.
Sorry for bothering and taking your time :)