Solana Program ABI

[defactojob]

Solana Program ABI

This document proposes a Solana ABI to make it possible to create different
implementations of the same interface.

Having an ABI makes it possible to create mintable tokens, non-mintable
tokens, interest bearing tokens, rebase tokens, etc. without overloading one
single token implementation.

Furthermore, it would be possible for DeFi products to support different types of
tokens as long as they conform to the same standard interface. Right now, the
logic of SPL Token is hardwired into projects like spl-swap, spl-lend, and
serum.

The important ideas to enable ABI are:

• An interface defines messages passed to a target.
  • A target is a tuple of (program_id, accounts).
  • The program id specifies the implementation
  • The accounts specify the state of the target
• In a message, the referenced accounts are encoded as indexes
• Processor dispatches on messages using selectors, instead of hard coded enum
  • The selector is sha256 of the message signature.
  • This makes it possible to create a program that "mix and match" messages
    from different crates.
• Handle program accounts and normal accounts transparently in the ABI encoding.
  • This is important if a program need to invoke another program, yet we want
    to keep the ABI the same.

First a high-level overview of the ABI "from the outside", then a binary
encoding for instructions is given at the end.

A Mintable Token

Let's define an interface with a single mint message:

const TokenABI = {
  name: "MintableToken",

  mint: [
    {
      name: "token",
      type: "account",
      write: true,
    },

    {
      name: "auth",
      type: "account",
      signed: true,
    },

    {
      name: "dest",
      type: "account",
      write: true,
    },

    {
      name: "amount",
      type: "u64",
    }
  ]
}

An example of using an ABI client to mint some tokens:

// the programID can be any implementation
const token = new Interface(TokenABI, programID)

token.send("mint", {
  // not signed, pass in PubKey
  token: tokenAddress, // PubKey

  // signed, pass in an Account
  auth: tokenOwnerAccount, // Account

  // this should be deduplicated
  dest: tokenOwnerAccount.pubkey, // PubKey

  amount: 100,
})

The ABI client should generate the transaction by doing two things:

1. Convert the accounts referenced by the ABI to indexes, doing deduplication.
2. Collect the referenced accounts into an account_info array.

instruction_data: new MintMessage({
  token: 0,
  auth: 1,
  dest: 1,
  amount: 100,
}).encode()

account_infos: [
  tokenAddress, // write
  tokenOwnerAccount, // signed
]

A Faucet Program

Suppose that we want to build a token faucet that has minting authority, and
supports arbitrary token programs as long as the mint ABI interface is
satisfied.

In Solidity it might look like this:

function give(IERC20 token, uint64 amount, address receiver) {
  require(token.owner == address(this), "faucet is not owner of token");
  token.mint(receiver, amount);
}

The Solana ABI needs to have a way to encode two things:

• The target that should receive the mint message.
• Support using a program account as the minting authority.

The ABI client might look like:

const tokenOwnerAccount = new ProgramAccount(programID, "program token owner")

faucet.send("give", {
  // this specifies the target that will receive a `mint` message using invoke_signed
  token: [tokenProgramID, tokenAddress],

  // signed program account
  auth: tokenOwnerAccount, // Account

  dest: receiverAddress, // PubKey

  amount: 100,
})

In this case, the generated transaction needs to also provide the program seed:

1. Convert the accounts referenced by the ABI to indexes, doing deduplication.

• If the referenced account is a program account, it should also encode the
  index of the seed used.

2. Collect the referenced accounts into an account_info array.
3. Collect the referenced program seeds into an array, doing deduplication.

The transaction would be like:

instruction_data: new GiveInstruction({
  token: (0, 1),
  auth: (2, 0),
  dest: 3,
  amount: 100,
}).encode()

seeds: [
  // the seed is the user given seed plus the nonce
  ["program owner", 1]
]

account_infos: [
  tokenProgramID,
  tokenAddress, // write
  tokenOwnerAccount, // signed
  receiverAddress, // signed
]

• auth is now a tuple that refers to both program account info and
  program seed, so that invoke_signed may be called to mint the token.
• token is now a tuple, the first is the program id, and the second is the
  token address for that program.

The faucet program then use this information to send a message to the target token with invoke_signed.

instruction_data: new MintMessage({
  token: 1,
  auth: (2, 0),
  dest: 3,
  amount: 100,
}).encode()

seeds: [
  // the seed is the user given seed plus the nonce
  ["program owner", 1]
]

target_program: accounts[0]

Faucet With Counter

By encoding account indexes in the message, the programs do not need to agree on
the order of the accounts in a transaction.

Suppose that we want to extend the faucet with a counter to keep track of how
much supply the faucet had given out. We add a counter address to store that
data in the give message:

instruction_data: new GiveMessage({
  counter: 0,

  token: (1, 2),
  auth: (3, 0),
  dest: 4,

  amount: 100,
}).encode()

seeds: [
  // the seed is the user given seed plus the nonce
  ["program owner", 1]
]

account_infos: [
  counterAddress // write
  tokenProgramID,
  tokenAddress, // write
  tokenOwnerAccount, // signed
  receiverAddress, // write
]

Although the indexes are shifted by one, the logic that deals with
invoke_signed does not have to change at all. Nor does the token program rely
on accounts being passed in a fixed order.

SPL Token Compatibility

With an ABI token defined, it should be possible to create a wrapper for
SPL tokens to handle them in the same way as ABI tokens.

TODO Reader Interface

There need to be a way to define standard reader APIs, for both programs and ABI
clients to consume.

A typical example is the balanceOf interface. The states of different program
implementation may be different, and without a reader to compute value from
state is needed.

Readers would be like serverless lambda functions... would be awesome if it's possible to
subscribe to these lambda functions if the underlying states changed.

Encoding of a Message Into an Instruction

A message needs to specify:

• The binary encoded message (e.g. with borsh).
• Program seeds.

The mint token ABI would be calculated similar to Solidity's selector, using
the sha256 hash of its signature:

selector = sha256("spl.MintableToken.mint(account,account,account,u64)")[:8]

It differs from Solidity selector in the following ways:

• The message name should be namespaced to avoid collision
• Take 8 bytes instead of 4 bytes

The transaction data's layout:

transaction_data := [selector: [u8;8]][message: Vec<u8>][seeds: Vec<Vec<u8>>]

A program should use the selector to lookup the corresponding message type, and
ecode the message using borsh.

The accounts are provided to the message processor, to associate the encoded
account indexes with account info.

Furthermore, seeds (if any) in transaction_data are provided to the message
processor, if it needs to call invoke_signed

message encoding size

Assuming all the structs are encoded using borsch.

The representation for dynamic Vector has 4 bytes of overhead for length:

repr(len() as u32)
for el in x
  repr(el as ident)

We could use varuint to encode the
top-level instruction:

transaction_data :=
  [selector: [u8;8]]
  [message_size: varuint][message: [u8; message_size]]
  [seeds_size: varuint][seeds:
    [seed_size: varuint][seed: [u8; seed_size]* ]
  ]

• For size <= 240, it takes one byte.
• For size <= 2031, it takes two bytes.

The account indexes are encoded as a struct of 2 bytes:

struct AccountIndex {
  account: u8,
}

// if this is a program account that needs to invoke_signed
struct SignedProgramAccountIndex {
  account: u8,
  seed: u8,
}

struct Target {
  program_id: u8,
  account: u8,
}

The mint message with account indexes:

struct Mint {
  token: AccountIndex,
  from: AccountIndex,
  from_owner: AccountIndex,
  to: AccountIndex,

  amount: u64,
}

• overhead: 14 bytes
  • selector: 8, msg_size: 1, seeds_size: 1, account_indexes: 4

struct Give {
  token: Target,

  auth: SignedProgramAccountIndex,
  dest: AccountIndex,

  amount: u64,
}

• overhead: 16 bytes
  • selector: 8, msg_size: 1, seeds_size: 1, seed_size: 1, account_indexes: 5

[joncinque]

This is a very well thought-out proposal and answers a lot of questions that have come up in our ABI discussions, thanks for the great work!

A lot of this sounds very workable. We've been wondering about parameter differences in ABIs for our particular usecase of interest-bearing tokens. In order to mint tokens, imagine that we have to accrue interest, which means that we have to either send an additional instruction beforehand to accrue interest, or send the clock sysvar along with the normal parameters defined in the mint ABI.

How would you modify your model to make that work? It seems like it might be independent of your solution, requiring an additional hook implemented by the program library. Or the ABI solution includes an on-chain registry that gives more info about requirements of specific programs.