Protocol 14 support

[abuiles]

This issue lists the changes to the Horizon API introduced by CAP-23 and CAP-33. These two CAPs comprise the public-facing changes in Stellar Protocol 14. The first Horizon version with the updated API is Horizon 1.9.0-RC, to be released 2020-09-21 (monorepo release page).

This protocol upgrade is purely additive. We expect a protocol 14 compliant SDK to be able to run successfully against a protocol 13 network.

We are aiming for the following timeline:

• 2020-09-21 Horizon 1.9.0-RC is released with support for protocol 14
• 2020-09-30 Testnet will vote to update to protocol 14
• 2020-10-28 Pubnet will vote to update to protocol 14

In what follows, "canonical form" means code:address or native (for the XLM asset).

New objects

• Claimable Balance with the following fields:
  • id - balance ID,
  • paging_token - paging token,
  • asset - asset available to be claimed (in canonical form),
  • amount - amount available to be claimed (string, like all amounts),
  • sponsor - sponsoring account ID (can be null),
  • last_modified_ledger - sequence number of the ledger when the balance was last modified,
  • claimants - list of objects:
    • destination - destination account ID,
    • predicate - predicate required to claim a balance (see below).

Modified objects

• Account object:
  • New sponsor field (account ID, can be null).
  • New num_sponsoring field - number of reserves sponsored by this account,
  • New num_sponsored field - number of reserves sponsored for this account.
• New sponsor field (account ID) in Account > Balance object (only for non-native assets, can be null).
• New sponsor field (account ID) in Account > Signers object (can be null).
• New sponsor field (account ID) in Account's Data object (can be null).
• New sponsor field (account ID) in Offer object (can be null).

New endpoints

• /claimable_balances - the list of Claimable Balance objects with the following parameters (only one param per request allowed):
  • asset - find all claimable balances with the given asset (in canonical form),
  • claimant - find all claimable balances with the given claimant account ID,
  • sponsor - find all claimable balances sponsored by a given sponsor account ID.
• /claimable_balances/{id} - a single Claimable Balance object.

Modified endpoints

• /accounts can now by filtered by sponsor (new GET param).
• /offers can now by filtered by sponsor (new GET param).

New operations

• create_claimable_balance with the following fields:
  • asset - asset available to be claimed (in canonical form),
  • amount - amount available to be claimed,
  • claimants - list of claimants with predicates (see below):
    • destination - destination account ID,
    • predicate - predicate required to claim a balance (see below).
• claim_claimable_balance with the following fields:
  • balance_id - unique ID of balance to be claimed,
  • claimant - account ID of a claimant.
• begin_sponsoring_future_reserves with the following fields:
  • sponsored_id - account ID for which future reserves will be sponsored.
• end_sponsoring_future_reserves with the following fields:
  • begin_sponsor - account sponsoring reserves.
• revoke_sponsorship with the following fields:
  • account_id - if account sponsorship was revoked,
  • claimable_balance_id - if claimable balance sponsorship was revoked,
  • data_account_id - if account data sponsorship was revoked,
  • data_name - if account data sponsorship was revoked,
  • offer_id - if offer sponsorship was revoked,
  • trustline_account_id - if trustline sponsorship was revoked,
  • trustline_asset - if trustline sponsorship was revoked,
  • signer_account_id - if signer sponsorship was revoked,
  • signer_key - if signer sponsorship was revoked.

New effects

• claimable_balance_created with the following fields:
  • balance_id - unique ID of claimable balance,
  • asset - asset available to be claimed (in canonical form),
  • amount - amount available to be claimed.
• claimable_balance_claimant_created with the following fields:
  • balance_id - unique ID of a claimable balance,
  • asset - asset available to be claimed (in canonical form),
  • amount - amount available to be claimed,
  • predicate - predicate required to claim a balance (see below).
• claimable_balance_claimed with the following fields:
  • balance_id - unique ID of a claimable balance,
  • asset - asset available to be claimed (in canonical form),
  • amount - amount available to be claimed,
• account_sponsorship_created with the following fields:
  • sponsor - sponsor of an account.
• account_sponsorship_updated with the following fields:
  • new_sponsor - new sponsor of an account,
  • former_sponsor - former sponsor of an account.
• account_sponsorship_removed with the following fields:
  • former_sponsor - former sponsor of an account.
• trustline_sponsorship_created with the following fields:
  • sponsor - sponsor of a trustline.
• trustline_sponsorship_updated with the following fields:
  • new_sponsor - new sponsor of a trustline,
  • former_sponsor - former sponsor of a trustline.
• trustline_sponsorship_removed with the following fields:
  • former_sponsor - former sponsor of a trustline.
• claimable_balance_sponsorship_created with the following fields:
  • sponsor - sponsor of a claimable balance.
• claimable_balance_sponsorship_updated with the following fields:
  • new_sponsor - new sponsor of a claimable balance,
  • former_sponsor - former sponsor of a claimable balance.
• claimable_balance_sponsorship_removed with the following fields:
  • former_sponsor - former sponsor of a claimable balance.
• signer_sponsorship_created with the following fields:
  • signer - signer being sponsored.
  • sponsor - signer sponsor.
• signer_sponsorship_updated with the following fields:
  • signer - signer being sponsored.
  • former_sponsor - the former sponsor of the signer.
  • new_sponsor - the new sponsor of the signer.
• signer_sponsorship_removed with the following fields:
  • former_sponsor - former sponsor of a signer.

predicate field

predicate field is a JSON representation of xdr.ClaimPredicate as defined in CAP-23 and is a requirement that needs to be satisfied to claim the balance. It is a recursive structure that can be represented in JSON using for example the following Golang struct:

type claimPredicateJSON struct {
	And           *[]claimPredicateJSON `json:"and,omitempty"`
	Or            *[]claimPredicateJSON `json:"or,omitempty"`
	Not           *claimPredicateJSON   `json:"not,omitempty"`
	Unconditional bool                  `json:"unconditional,omitempty"`
	AbsBefore     *time.Time            `json:"absBefore,omitempty"`
	RelBefore     *int64                `json:"relBefore,omitempty"`
}

Please refer to the Golang implementation for details.

The following issue shows how the Golang SDK will handle predicate creation: stellar/go#3000

RevokeSponsorshipOp

The RevokeSponsorshipOp requires users to build LedgerKey or a struct for signers sponsorship. Ideally SDKs should expose helpers that build a valid operation for users without require them to pass an XDR LedgerKey. See the following issue for more information stellar/go#3001 .

[abuiles]

Hi! We just made available a Horizon standalone network running protocol 14 which you can use to test your changes.

• The Horizon URL is https://horizon-protocol14.stellar.org
• The Network Passphrase is Standalone protocol 14 Devnet; September 2020

To get test lumens, generate a new keypair here

And then replace the account query parameter in the following URL https://friend14.herokuapp.com/?account=G... with the public account ID generated in the laboratory.

The protocol 14 standalone test network will be available until 9/30/2020 and then it will go offline.

[abuiles]

We noticed some effects were missing and should be incorporated (they are not included in the 1.19.0 Horizon release candidate). You can find extra context at stellar/go#3060

Extra new effects in protocol 14

• data_sponsorship_created with the following fields:
  • sponsor - sponsor of account data.
  • data_name - name of the data created.
• data_sponsorship_updated with the following fields:
  • new_sponsor - new sponsor of account data.
  • former_sponsor - former sponsor of a claimable balance.
  • data_name - name of the data updated.
• data_sponsorship_removed with the following fields:
  • former_sponsor - former sponsor of account data.
  • data_name - name of the data removed.

Pre-existing effects (before protocol 14) which were already being emitted by Horizon but may not be covered by the SDKs (and should be)

• data_created with the following fields:
  • name - name of the account data created.
  • value - value of the account data created.
• data_updated with the following fields:
  • name - name of the account data updated.
  • value - value of the account data updated.
• data_removed with the following fields:
  • name - name of the account data removed.

Updates to future friend-bot

We added a new end-point to the future friend-bot which returns a sponsored account and a sponsored data entry in that account: https://friend14.herokuapp.com/new-account

[abuiles]

Hi there!

Just a quick heads up that we'll be changing the keys on the JSON response for claim predicates in the next release.

We introduced a couple of keys with camelCase, but we use snake_case in Horizon.

For consistency we decided to add this change in the next release and although it is a "breaking change", we'll be doing a patch release (1.10.0) - mostly because:

1. Testnet will be reset soon, so old data won't be relevant.
2. There is no production data with this format since it is only generated with Protocol 14 operations.

The change renames the field absBefore to abs_before and relBefore to relBefore.

Something like this:

{
  "predicate": {
    "and": [
      {
        "not": {
          "absBefore": "100"
        }
      },
      {
        "or": [
          {
            "relBefore": "1000"
          },
          {
            "not": {
              "relBefore": "108000"
            }
          }
        ]
      }
    ]
  }
}

Will become

{
  "predicate": {
    "and": [
      {
        "not": {
          "abs_before": "100"
        }
      },
      {
        "or": [
          {
            "rel_before": "1000"
          },
          {
            "not": {
              "rel_before": "108000"
            }
          }
        ]
      }
    ]
  }
}

The following PR implements the changes listed above in the JS-SDK stellar/js-stellar-sdk#575