diff --git a/docs/exchange-operators/faq.mdx b/docs/exchange-operators/faq.mdx deleted file mode 100644 index da1318b17..000000000 --- a/docs/exchange-operators/faq.mdx +++ /dev/null @@ -1,196 +0,0 @@ ---- -title: FAQ Listing Mina -description: Frequently asked questions about listing Mina -hide_title: true ---- - -import Subhead from "@site/src/components/common/Subhead"; - -# FAQ Listing Mina - - -Frequently asked questions about listing Mina. - - -## Basics - -### Where can I find third-party audit reports for Mina? - -The latest third-party audit reports are publicly available here: - -- [https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review](https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review/) -- [https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs](https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs/) -- [https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review](https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review) - -

- -:::note - -Any news and updates related to exchange listing shared by the Mina Foundation are on [www.minaprotocol.com](https://minaprotocol.com) or the official [Mina Protocol](https://x.com/MinaProtocol) X (Twitter). Mina Foundation cannot individually answer any listing questions. - -::: - -## Rosetta - -### Why do you recommend using Rosetta for integrating Mina to our exchange? - -Rosetta is an open-source specification that helps exchanges and developers integrate blockchains. Since Rosetta is actively maintained and specifically designed to enable simpler, faster, and more reliable blockchain integrations, we highly recommend using Rosetta to integrate Mina blockchain with your exchange. - -### What if I have a question about Rosetta? - -Ask in [Mina Protocol Discord](https://discord.gg/minaprotocol) or post to the Mina GitHub [Discussions](https://github.com/MinaProtocol/mina/discussions). - -## Accounts - -### Is there an account creation fee? - -Yes, Mina Protocol charges a fee of 1 MINA when you create a new account. This fee helps protect the network from denial of service-type attacks. Over time, this fee can change. - -## Transactions - -### What is the maximum size of the mempool? How do we work around this? - -The max mempool size is 3,000. After it hits that size, transactions with the lowest fees are discarded. - -Set your fee to an amount higher than 0.001 MINA, the current average fee for transactions in the pool. - -You can view the fees for pending transactions and adjust your fees accordingly: [https://minascan.io/mainnet/txs/pending-txs](https://minascan.io/mainnet/txs/pending-txs). - -### Why do some users appear to have lost their funds when sending to exchanges? - -:::tip - -While Mina and its SDKs do support the memo field when sending a transaction, the recommended best practice is do NOT require a memo for deposits. - -::: - -To associate the deposit with the user's account, some exchanges require their users to include a unique memo field when sending MINA deposits to the exchange's address. - -If the user does not include this unique memo when sending their deposit, the receiving exchange may not be able to associate the deposit properly with the user's exchange account. - -These funds are NOT lost. The exchanges have received the funds at the exchange's address, but the exchange may not be able to automatically associate the deposit with the user's exchange account without such a memo. - -To prevent this issue, we recommend that exchanges do NOT require a memo for deposits. At the same time, exchanges and wallet creators are recommended to expose an optional memo field during a Mina send transaction. - -### What is the maximum number of rollback blocks? - -The table in [Lifecycle of a Payment](/mina-protocol/lifecycle-of-a-payment) describes how many blocks you wait for a transaction to be confirmed. - -### How should I calculate transaction fees? - -To calculate your transaction fees, use [https://fees.mina.tools](https://fees.mina.tools/). - -## Running a node - -### My Mina node gets stuck sometimes. How can I detect this and fix it? - -This is a known issue for some earlier releases. Restart your mina node whenever this issue is detected. - -You can use the following script to run a cron job every 3 minutes (the slot length) or more frequently: - -``` -MINA_STATUS=$($MINA client status --json) -HIGHESTBLOCK="$(echo $MINA_STATUS | jq .highest_block_length_received)" -HIGHESTUNVALIDATEDBLOCK="$(echo $MINA_STATUS | jq .highest_unvalidated_block_length_received)" - -# Calculate difference between validated and unvalidated blocks. -# If block height is more than 4 block behind, something is likely wrong. - -DELTAVALIDATED="$(($HIGHESTUNVALIDATEDBLOCK-$HIGHESTBLOCK))" -if [[ "$DELTAVALIDATED" -gt 4 ]]; then - $MINA client stop -fi -``` - -:::tip - -Be sure your Mina daemon is monitored by something such as systemd, so it can auto-restart. - -::: - -### My archive node is missing block information after a restart. How can I recover the data? - -Archive node operators often choose to run redundant archive nodes to store block data to one or more locations of their choice (for example, PostgreSQL, GCP, local files, or a logging service) and to backfill any missed block data if needed. - -For convenience, [mina_network_block_data](https://console.cloud.google.com/storage/browser/mina_network_block_data) from the archive node is available to help others in the community backfill any missing information. - -This bucket contains blocks from various Mina networks — for example, Mainnet and the most recent Devnet `devnet2`. Filter by filename for the network you want. Note that this bucket contains blocks for various other networks too, such as QAnet, which is not recommended for your testing. QAnet is used by o1Labs during targeted iterative development. - -Filenames contain the network name, block height, and state hash of the block. Blocks older than height 25,705 include only the network name and state hash in the filename. - -Example filenames: - -(Recent) - -``` -mainnet-30627-3NLfKanQ53X2MRKx5ZRvb9nVCEB9eJpcnssGCTpT3J1cojhB5M19.json -``` - -(Older) - -``` -mainnet-3NKUBmkc7UZ7ik5JyCM4WNzkN1HG5heMB5zNDUkf3Kgta1MFY6LY.json -``` - -You can download a specific block using curl: - -``` -curl https://mina_network_block_data.storage.googleapis.com/ -``` - -You can import this file using the mina archive blocks tool. The command for it is: - -``` -mina-archive-blocks --precomputed --archive-uri FILE. -``` - -### How do I query for the canonical block at a certain height from the archive node - -Use a recursive query. See [Query the database](/node-operators/archive-node/getting-started#query-the-database) examples in the Archive Node docs. - -### Why am I getting this error message: "Not able to connect to the network"? - -This error message usually occurs due to a chain ID mismatch from running a Devnet build on Mainnet, or vice versa. - -To check whether you are running a devnet or mainnet build, run `Mina client status`. Next, compare the output's chain ID of your node to the expected chain ID of the network you are trying to connect to. You can find required information for comparison within the [GitHub announcements](https://github.com/MinaProtocol/mina/discussions/categories/announcements) or [Discord](https://discord.com/channels/484437221055922177/601171209287368715) server. - -### Are there any official broadcast nodes that can be used? - -No, there are no official broadcast nodes at this time. However, you can broadcast transactions using [https://minascan.io/mainnet/broadcast/payment](https://minascan.io/mainnet/broadcast/payment). Use this method as a backup, the recommended method is to broadcast transactions yourself. - -## Staking - -### Should I be staking my funds? - -Since Mina is a Proof of Stake (PoS) consensus network without lockup for staked tokens, it is recommended to stak these funds to support the quality of the Mina network. Additionally, by not staking, you are missing out on staking rewards that you can otherwise be receiving from the Mina blockchain. - -You can look into staking this wallet, either by running your own block production node or just by delegating your funds to a staking pool on the network. Delegating to a staking pool is simpler to set up. - -:::note - -Newly staked accounts incur a delay of 18 to 29 days before you start receiving rewards. - -::: - -### Why is there a delay for staking to take effect? - -For purposes of ensuring consensus, there is a delay between when delegations are sent on the blockchain and when they take effect with respect to staking on the network. The staking ledger always operates between 18 to 29 days behind the live ledger. - -### In that case, how long is the delay and when is the next staking snapshot? - -The timing of the next staking snapshot varies. - -Since the timing is based on a combination of consensus timing (epochs) and snarketplace throughput, it is difficult to determine exactly how long this delay can be. - -A conservative estimate is that delegations sent 3 days before the epoch transition can take effect in the upcoming epoch. This means that, for any given delegation, there is an average of 18 to 29 days delay before this delegation updates block production. - -You can use this Delegation Calculator tool built by Carbonara to see the next staking ledger cutoff: -[https://epoch.mina.tools](https://epoch.mina.tools/). - -## Testing - -### What is the best way to test tooling and integration with Mina? - -Test tooling and integrations on Devnet before going live on Mainnet. The Devnet network is dedicated for developers building on top of the Mina protocol and is designed for testing and experimentation. Be sure to simulate expected Mainnet conditions, such as transaction volume and frequency, to help identify and solve potential issues ahead of time. - -See [Connect to Mainnet or Devnet](/node-operators/validator-node/connecting-to-the-network). diff --git a/docs/exchange-operators/index.mdx b/docs/exchange-operators/index.mdx deleted file mode 100644 index a38819801..000000000 --- a/docs/exchange-operators/index.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: Exchange Operators -hide_title: true -description: Learn how to integrate Mina with an exchange. -keywords: - - historical data - - blockchain integration - - explorer - - wallet - - archive node - - graphQL - - rosetta ---- - -# Exchange Operators - -Exchange operators are network participants who run Mina nodes on a Mina network but usually don't produce blocks. - -Instead of running a Mina node and an archive node, use the Rosetta API that is designed specifically for exchange operators. - -It implements an [open-source blockchain-agnostic specification](https://docs.cloud.coinbase.com/rosetta/docs/welcome) designed by Coinbase. - -## Rosetta API - -This section describes how to install, run and use the Rosetta API instance connected to the Mina network. - -Use Rosetta API if you are: -- Querying historical data from the Mina blockchain -- Integrating the Mina blockchain with your exchange -- Building blockchain applications, such as explorers and wallets - -Using Rosetta for building your application is preferable, as it's API is more appropriate for applications and far more stable than GraphQL. - - -- [Running with Docker](/exchange-operators/rosetta/run-with-docker) - How to install Rosetta from Docker image -- [Building from source](/exchange-operators/rosetta/build-from-sources) - How to build and run Rosetta from source code -- [Sending requests](/exchange-operators/rosetta/send-requests) - How to send request to a Rosetta API -- [Code Samples](/exchange-operators/rosetta/samples) - Sample code for Rosetta integration integration - - [Requests and Responses](/exchange-operators/rosetta/samples/requests) - Implementing basic requests - - [Offline Signer Tool](/exchange-operators/rosetta/samples/using-signer) - Generating keys and signing transactions offline - - [Scanning Blocks](/exchange-operators/rosetta/samples/scan-blocks) - Example implementation of block scanning - - [Tracking Deposits](/exchange-operators/rosetta/samples/track-deposits) - Example implementation of deposit tracking - - [Sending Transactions](/exchange-operators/rosetta/samples/send-transactions) - Example implementation of withdrawal flow - - -## FAQ - -For answers to the most frequently asked questions about listing MINA, see [FAQ Listing MINA](/exchange-operators/faq). - -## Questions? - -Post any questions in Mina’s Github Discussions: https://github.com/MinaProtocol/mina/discussions diff --git a/docs/exchange-operators/rosetta/samples/index.mdx b/docs/exchange-operators/rosetta/samples/index.mdx deleted file mode 100644 index abdb11cb5..000000000 --- a/docs/exchange-operators/rosetta/samples/index.mdx +++ /dev/null @@ -1,68 +0,0 @@ ---- -title: Code Samples -description: Inspect integration code samples ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -The source code for all code samples is available on the official [Mina Documentation Github page](https://github.com/o1-labs/docs2/tree/main/examples/exchange-operators/) and should be used only for demonstration purposes. - -## Set up sample project - -Start with the required imports and define constants: - - - - -```ts -import axios from "axios" -import { Client } from "mina-signer" - -const TESTNET_NETWORK_IDENTIFIER = { - network_identifier: { - blockchain: "mina", - network: "testnet", - }, -} -const MINA_TOKEN_ID = "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf" -const MINA_DECIMALS = 9 -const MINA_SYMBOL = "MINA" -const MINA_CURVE_TYPE = "pallas" - -const mina = new Client({ network: "testnet" }) - -const request = axios.create({ - baseURL: "https://rosetta-devnet.minaprotocol.network/", -}) -const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)) -``` - - - - -```python -import json -from subprocess import run, PIPE -from time import sleep - -from requests import post - -BASE_URL = "http://localhost:3087" -SIGNER_PATH = 'path/to/signer.exe' -MAINNET_NETWORK_IDENTIFIER = { - "network_identifier": { - "blockchain": "mina", - "network": "mainnet" - } -} -MINA_TOKEN_ID = "1" -MINA_DECIMALS = 9 -MINA_SYMBOL = "MINA" -MINA_CURVE_TYPE = "pallas" -MINA_SIGNATURE_TYPE = "schnorr_poseidon" -``` - - - - diff --git a/docs/exchange-operators/rosetta/samples/requests.mdx b/docs/exchange-operators/rosetta/samples/requests.mdx deleted file mode 100644 index 82c0d5003..000000000 --- a/docs/exchange-operators/rosetta/samples/requests.mdx +++ /dev/null @@ -1,592 +0,0 @@ ---- -title: Requests and Responses -description: Learn about Mina-specific Rosetta request and response objects ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -# About Mina-specific requests and responses - -The Rosetta API specification defines high-level description of how requests and response objects should look like. -Exact json object layouts are different from blockchain to blockchain. - -In this article, you learn about Mina-specific Rosetta JSON objects and how to implement helper functions to construct and parse the JSON objects. - -### Network Identifier - -There is the `network_identifier` object, that should be passed as a parameter to all endpoints (except `/network/list` which returns possible identifiers). - -First, implement the generic `request` function to perform Rosetta API calls injecting that object under the hood: - - - - -```ts -async function makeRequest(endpoint: string, data?: any) { - if (data === undefined) { - data = {} - } - if (endpoint !== "/network/list") { - data = { ...TESTNET_NETWORK_IDENTIFIER, ...data } - } - - const r = await request.post(endpoint, data) - return r.data -} -``` - - - - - -```python -def _request(path, data=None): - """ - Generalized logic for sending a request to Rosetta API - """ - data = data or {} - nw = {} if path == "/network/list" else MAINNET_NETWORK_IDENTIFIER - path = BASE_URL + path - data = json.dumps({**data, **nw}) - r = post(path, data) - try: - r.raise_for_status() - except Exception as e: - # json.dumps(r.json()) - raise e - return r.json() -``` - - - - -### Block, Transaction, Account Identifiers and the Pubkey object - -According to Rosetta request and response models, each block, transaction and account are identified by an identifier object. -A pubkey is also represented by special object. - -To implement helpers to construct the objects: - - - - -```ts -function makeBlockIdentifier(idOrHash: number | string) { - const identifierKey = (typeof idOrHash === "number") ? "index" : "hash" - return { - block_identifier: { [identifierKey]: idOrHash }, - } -} - -function makeTxIdentifier(hash: string) { - return { transaction_identifier: { hash: hash } } -} - -function makeAccountIdentifier(address: string) { - return { - account_identifier: { address, token_id: MINA_TOKEN_ID }, - } -} - -function makePublicKey(publicKey: string) { - return { - hex_bytes: publicKey, - curve_type: MINA_CURVE_TYPE, - } -} -``` - - - - - -```python -def _make_block_identifier(index_or_hash): - key = "index" if type(index_or_hash) == int \ - else "hash" if type(index_or_hash) == str \ - else None - if not key: - raise ValueError("index_or_hash should have either int or str type") - return {"block_identifier": {key: index_or_hash}} - -def _make_tx_identifier(tx_hash): - return {"transaction_identifier": {'hash': tx_hash}} - -def _make_account_identifier(address): - return { - "account_identifier": {"address": address, "token_id": MINA_TOKEN_ID} - } - -def _make_pubkey(pubkey): - return { - "hex_bytes": pubkey, - "curve_type": MINA_CURVE_TYPE - } -``` - - - - -### Operation object - -In Rosetta terminology, each transaction consist of one or more operations. -In Mina implementation, each operation object has following fields: -- `operation_identifier` and `related_operations`: a mandatory index of an operation, and optional array of related operations. -- `type`: the type of an operation. -- `account`: the account identifier which the operation relates to -- `amount`: an object with `value` - a signed number representing of the accounts' balance - -
-A sample JSON object that represents an operation - -```json -{ - "operation_identifier": { - "index": 2 - }, - "related_operations": [ - { - "index": 1 - } - ], - "type": "payment_receiver_inc", - "account": { - "address": "B62qqJ1AqK3YQmEEALdJeMw49438Sh6zuQ5cNWUYfCgRsPkduFE2uLU", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "90486110", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } -} -``` - -
- -:::note -All possible operation types are available on the `network/options` endpoint. The most common types are `fee_payment`, `payment_source_dec` and `payment_receiver_inc` -::: - - -### Layout of the transfer transaction - -In Mina, each operation represents an [Account Update](/zkapps/writing-a-zkapp/introduction-to-zkapps/interact-with-mina#account-updates). Therefore, a MINA token transfer transaction is represented by three Account Updates: -- for decreasing fee payer account balance (the fee payer's account is the same as the sender's) -- for decreasing sender account balance -- for increasing receiver account balance - -
-A sample object that represents a transfer transaction - -```json -{ - "transaction_identifier": { - "hash": "CkpYVELyYvzbyAwYcnMQryEeQ7Gd6Ws7mZNXpmF5kEAyvwoTiUfbX" - }, - "operations": [ - { - "operation_identifier": { - "index": 0 - }, - "type": "fee_payment", - "account": { - "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "-37000000", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } - }, - { - "operation_identifier": { - "index": 1 - }, - "type": "payment_source_dec", - "account": { - "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "-58486000", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } - }, - { - "operation_identifier": { - "index": 2 - }, - "related_operations": [ - { - "index": 1 - } - ], - "type": "payment_receiver_inc", - "account": { - "address": "B62qkiF5CTjeiuV1HSx4SpEytjiCptApsvmjiHHqkb1xpAgVuZTtR14", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "58486000", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } - } - ] -} -``` - -
- -According to the Rosetta specification, the array of `operation` objects must be passed as a parameter in the `/construction/preprocess` and `/construction/payloads` endpoints. - -To implement a helper function to construct the operations array for a MINA token transfer: - - - - -```ts -function makeTransferPayload(from: string, to: string, feeNano: number, valueNano: number) { - function makeOperation( - idx: number, - relatedIdxs: number[], - opType: string, - addr: string, - value: number, - isPositive: boolean, - ) { - const relatedOps = (relatedIdxs.length == 0) ? {} : { - related_operations: relatedIdxs.map((i) => { - return { index: i } - }), - } - - return { - operation_identifier: { index: idx }, - ...relatedOps, - type: opType, - account: { - address: addr, - metadata: { - token_id: MINA_TOKEN_ID, - }, - }, - amount: { - value: (isPositive ? "" : "-") + value.toString(), - currency: { - symbol: MINA_SYMBOL, - decimals: MINA_DECIMALS, - }, - }, - } - } - - return { - operations: [ - makeOperation( - 0, - [], - "fee_payment", - from, - feeNano, - false, - ), - makeOperation( - 1, - [], - "payment_source_dec", - from, - valueNano, - false, - ), - makeOperation( - 2, - [1], - "payment_receiver_inc", - to, - valueNano, - true, - ), - ], - } -} -``` - - - - - -```python -def _make_transfer_payload(address_from, address_to, fee_nano, value_nano): - """ - Helper for generating "operations" object in a Rosetta format - This helper generates mina native token transfer payloads - The resulting object is an array of 3 so called account updates: - - for decreasing the sender's balance by the fee amount - - for decreasing the sender's balance by the transfer amount - - for increasing the receiver's balance by the transfer amount - """ - def _make_operation(idx, related_idxs, op_type, addr, value, is_positive): - related_ops = {} if not related_idxs else { - "related_operations": [{"index": i} for i in related_idxs] - } - return { - "operation_identifier": {"index": idx}, - **related_ops, - "type": op_type, - "account": { - "address": addr, - "metadata": { - "token_id": MINA_TOKEN_ID - } - }, - "amount": { - "value": ("" if is_positive else "-") + str(value), - "currency": { - "symbol": MINA_SYMBOL, - "decimals": MINA_DECIMALS - } - } - } - return { - "operations": [ - _make_operation( - 0, [], "fee_payment", address_from, fee_nano, False), - _make_operation( - 1, [], "payment_source_dec", address_from, value_nano, False), - _make_operation( - 2, [1], "payment_receiver_inc", address_to, value_nano, True) - ] - } -``` - - - - -### Implementing client methods - -Now that you have all helpers in place, you can implement all of the Rosetta client methods according to the specification: - - - - -```ts -async function networkList() { - return await makeRequest("/network/list") -} - -async function networkStatus() { - return await makeRequest("/network/status") -} - -async function networkOptions() { - return await makeRequest("/network/options") -} - -async function block(idOrHash: number | string) { - return await makeRequest("/block", makeBlockIdentifier(idOrHash)) -} - -async function mempool() { - return await makeRequest("/mempool") -} - -async function mempoolTx(hash: string) { - return await makeRequest("/mempool/transaction", makeTxIdentifier(hash)) -} - -async function accountBalance(address: string) { - return await makeRequest("/account/balance", makeAccountIdentifier(address)) -} - -async function accountTransactions(address: string) { - return await makeRequest("/search/transactions", { address: address }) -} - -async function getTransaction(hash: string) { - return await makeRequest("/search/transactions", makeTxIdentifier(hash)) -} - -async function deriveAccountIdentifier(publicKey: string) { - return await makeRequest("/construction/derive", { - public_key: makePublicKey(publicKey), - }) -} - -async function txPreprocess(from: string, to: string, feeNano: number, valueNano: number) { - const payload = makeTransferPayload(from, to, feeNano, valueNano) - return await makeRequest("/construction/preprocess", payload) -} - -async function txMetadata( - srcPublicKey: string, - srcAddress: string, - destAddress: string, - feeNano: number, - valueNano: number, -) { - const options = await txPreprocess(srcAddress, destAddress, feeNano, valueNano) - return await makeRequest("/construction/metadata", { - ...options, - public_keys: [makePublicKey(srcPublicKey)], - }) -} - -async function txPayloads( - srcPublicKey: string, - srcAddress: string, - destAddress: string, - feeNano: number, - valueNano: number, -) { - // If fee_nano is undefined, it will get suggested fee from /construction/metadata response - const meta = await txMetadata( - srcPublicKey, - srcAddress, - destAddress, - feeNano, - valueNano, - ) - - if (feeNano === 0) { - feeNano = meta.suggested_fee[0].value - } - console.log(feeNano) - const operations = makeTransferPayload(srcAddress, destAddress, feeNano, valueNano) - return makeRequest("/construction/payloads", { ...operations, ...meta }) -} - -async function txCombine(payloadsResponse: any, privateKey: string) { - // console.dir(payloadsResponse, {depth: null}) - const combinePayload = mina.rosettaCombinePayload(payloadsResponse, privateKey) - const r = await makeRequest("/construction/combine", combinePayload) - return r -} - -async function txParse(isSigned: boolean, blob: string) { - return makeRequest("/construction/parse", { - signed: isSigned, - transaction: blob, - }) -} - -async function txHash(blob: string) { - return await makeRequest("construction/hash", { signed_transaction: blob }) -} - -async function txSubmit(blob: string) { - return await makeRequest("construction/submit", { signed_transaction: blob }) -} -``` - - - - - -```python -def network_list(): - return _request("/network/list") - -def network_status(): - return _request("/network/status") - -def network_options(): - return _request("/network/options") - -def block(index_or_hash): - return _request("/block", _make_block_identifier(index_or_hash)) - -def mempool(): - return _request("/mempool") - -def mempool_tx(tx_hash): - return _request("/mempool/transaction", _make_tx_identifier(tx_hash)) - -def account_balance(address): - return _request("/account/balance", _make_account_identifier(address)) - -def derive_account_identifier(pubkey): - return _request("/construction/derive", { - "public_key": _make_pubkey(pubkey) - }) - -def tx_preprocess(src, dest, fee_nano, value_nano): - return _request( - "/construction/preprocess", - _make_transfer_payload(src, dest, fee_nano, value_nano)) - -def tx_metadata(src_pubkey, src_address, dest_address, fee_nano, value_nano): - options = tx_preprocess(src_address, dest_address, fee_nano, value_nano) - return _request("/construction/metadata", { - **options, "public_keys": [_make_pubkey(src_pubkey)] - }) - -def tx_payloads(src_pubkey, src_address, dest_address, fee_nano, value_nano): - """ - If fee_nano is None, it will get suggested fee from /construction/metadata response - """ - fee_nano = fee_nano or 0 - meta = tx_metadata( - src_pubkey, src_address, dest_address, fee_nano, value_nano) - - if not fee_nano: - fee_nano = meta["suggested_fee"][0]["value"] - - operations = _make_transfer_payload( - src_address, dest_address, fee_nano, value_nano) - return _request("/construction/payloads", {**operations, **meta}) - -def tx_combine(src_pubkey, tx_payloads_response, signature): - combine_payload = { - "unsigned_transaction": tx_payloads_response["unsigned_transaction"], - "signatures": [ - { - "signing_payload": tx_payloads_response["payloads"][0], - "public_key": _make_pubkey(src_pubkey), - "signature_type": MINA_SIGNATURE_TYPE, - "hex_bytes": signature - } - ] - } - return _request("/construction/combine", combine_payload) - -def tx_parse(transaction_blob, is_signed): - return _request("/construction/parse", { - "signed": is_signed, - "transaction": transaction_blob - }) - -def tx_hash(signed_transaction_blob): - return _request("/construction/hash", { - "signed_transaction": signed_transaction_blob - }) - -def tx_submit(signed_transaction_blob): - return _request("/construction/submit", { - "signed_transaction": signed_transaction_blob - }) -``` - - - - \ No newline at end of file diff --git a/docs/exchange-operators/rosetta/samples/scan-blocks.mdx b/docs/exchange-operators/rosetta/samples/scan-blocks.mdx deleted file mode 100644 index 210902935..000000000 --- a/docs/exchange-operators/rosetta/samples/scan-blocks.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: Scanning Blocks -description: Learn how to scan blocks and inspect transactions ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -To implement a simple function to wait for a given block: - - - - -```ts -async function waitForBlock(blockHeight: number) { - let latestBlock = (await networkStatus()).current_block_identifier.index - while (true) { - if (blockHeight <= latestBlock) { - return await block(blockHeight) - } - await sleep(10000) - latestBlock = (await networkStatus()).current_block_identifier.index - } -} -``` - - - - - -```python -def wait_for_block(block_index): - """ - Checks if block with given index exist - Once the /block response is succesful - returns the response - Otherwise, retries fetching it with 10 seconds delay - """ - - latest_block = network_status()["current_block_identifier"]["index"] - while True: - if block_index <= latest_block: - return block(block_index) - sleep(10) - latest_block = network_status()["current_block_identifier"]["index"] -``` - - - - -It can be used to scan blocks like this: - - - - -```ts -let latestBlockHeight = (await network_status()).current_block_identifier.index -while (true) { - const lastBlock = waitForBlock(latestBlockHeight) - // some processing according to business logic - - latestBlockHeight += 1 -} -``` - - - - - -```python -latest_block = network_status()["current_block_identifier"]["index"] -while True: - last_block = wait_for_block(latest_block) - - # some processing according to business logic - - latest_block += 1 -``` - - - \ No newline at end of file diff --git a/docs/exchange-operators/rosetta/samples/send-transactions.mdx b/docs/exchange-operators/rosetta/samples/send-transactions.mdx deleted file mode 100644 index 2ddda5448..000000000 --- a/docs/exchange-operators/rosetta/samples/send-transactions.mdx +++ /dev/null @@ -1,134 +0,0 @@ ---- -title: Sending transactions -description: Learn how to send transactions using a key pair ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -:::info -This flow is also described in the [Construction API Overview](https://docs.cloud.coinbase.com/rosetta/docs/construction-api-overview) article in the official Rosetta documentation. -::: - -The steps needed to send payment in MINA token are following: -1. Given a key pair, derive the account indentifier using `/construction/derive` endpoint -1. Call `/construction/preprocess` and `/construction/metadata` to construct parameters for `/construction/payloads` request -1. Create an unsigned transaction blob using `construction/payloads` endpoint -1. Call `construction/parse` (optional) to check if the unsigned transaction does what you expect -1. Use detached signer to sign the transaction -1. Call `construction/combine` to generate signed blob to be sent via `/construction/submit` endpoint -1. Call `construction/parse` again (optional) to confirm correctness of the signed transaction -1. Get a future transaction hash using `/construction/hash` enpoint -1. Submit the signed transaction blob via `/construction/submit` endpoint - -For ease of readability, this sample implementation skips the sanity checks (steps 4 and 7) and combines steps 2 and 3 in a single `tx_payloads` function call. - -Before running this sample: -- Make sure you followed [Offline signer tool](using-signer) article and generated a key pair -- Send some test funds on the account you've generated. - -The implementation is as follows: - - - - -```ts -async function send(privateKey: string, to: string, valueNano: number, feeNano: number) { - const publicKey = mina.derivePublicKey(privateKey) - const publicKeyRaw = mina.publicKeyToRaw(publicKey) - // get transfer payload to sign - const payloadsResponse = await txPayloads( - publicKeyRaw, - publicKey, - to, - feeNano, - valueNano, - ) - - // sign and combine transfer payload - const combineResponse = await txCombine(payloadsResponse, privateKey) - const blob = combineResponse.signed_transaction - - // // get future transaction hash - const txHashResponse = await txHash(blob) - const hash = txHashResponse.transaction_identifier.hash - - // submit transaction. this call will fail if tx is not in mempool - await txSubmit(blob) - // wait for transaction confirmation: - // for that, track blocks until we meet our transaction in the last one - let latestBlock = (await networkStatus()).current_block_identifier.index - while (true) { - // check if our transaction exists in the latest block - const txs = (await waitForBlock(latestBlock)).block.transactions - const hashes = txs.map((tx: any) => tx.transaction_identifier.hash) - if (hashes.includes(hash)) { - break - } - - latestBlock += 1 - } - return hash -} -``` - - - - - -```python -def withdrawal_flow(vault_keypair, dest_address, value_nano, fee_nano=None): - """ - Full withdrawal flow example (without sanity check) - If fee_nano is None - fee will be suggested by Rosetta API response - """ - - vault_pubkey = vault_keypair["public"] - vault_private_key = vault_keypair["private"] - - # derive vault address - derive_response = derive_account_identifier(vault_pubkey) - vault_address = derive_response["account_identifier"]["address"] - - # get transfer payload to sign - payloads_response = tx_payloads( - vault_pubkey, vault_address, dest_address, fee_nano, value_nano) - - # sign transfer payload - signature = sign_transaction( - SIGNER_PATH, - vault_private_key, - payloads_response["unsigned_transaction"]) - - # get signed transaction blob - combine_response = tx_combine(vault_pubkey, payloads_response, signature) - signed_transaction_blob = combine_response['signed_transaction'] - - # sanity check may be performed here - # for that - use tx_parse function output - # to doublecheck it corresponds what you want to do - # parsed_tx = tx_parse(signed_transaction_blob, True) - - # get future transaction hash - tx_hash_response = tx_hash(signed_transaction_blob) - future_tx_hash = tx_hash_response['transaction_identifier']['hash'] - - # submit transaction. this call will fail if tx is not in mempool - tx_submit(signed_transaction_blob) - - # wait for transaction confirmation: - # for that, track blocks until we meet our transaction in the last one - latest_block = network_status()["current_block_identifier"]["index"] - while True: - # check if our transaction exists in the latest block - txs = wait_for_block(latest_block)['block']['transactions'] - hashes = [tx['transaction_identifier']['hash'] for tx in txs] - if future_tx_hash in hashes: - break - latest_block += 1 - - return future_tx_hash -``` - - - \ No newline at end of file diff --git a/docs/exchange-operators/rosetta/samples/track-deposits.mdx b/docs/exchange-operators/rosetta/samples/track-deposits.mdx deleted file mode 100644 index ca5e70849..000000000 --- a/docs/exchange-operators/rosetta/samples/track-deposits.mdx +++ /dev/null @@ -1,81 +0,0 @@ ---- -title: Tracking deposits -description: Learn how to track deposits on the given address ---- - -import Tabs from '@theme/Tabs'; -import TabItem from '@theme/TabItem'; - -After everything is ready, you can implement deposit tracking. - -For that, iterate through all transactions in each block to search for `payment_receiver_inc` operation with an address matching the address in which you are interested: - - - - -```ts -async function trackDeposit(address: string) { - // some logic we want to execute on deposit - function onDeposit(deposit: any) { - console.log(deposit) - } - let latestBlock = (await networkStatus()).current_block_identifier.index - while (true) { - // check if transactions to the deposit address are present in the latest block - const txs = (await waitForBlock(latestBlock)).block.transactions - const deposits = [] - for (let tx of txs) { - for (let op of tx.operations) { - if (op.account.address === address && op.type === "payment_receiver_inc") { - deposits.push({ - tx_hash: tx.transaction_identifier.hash, - amount: op.amount.value, - }) - } - } - } - - // process deposits - for (let d of deposits) { - onDeposit(d) - } - - latestBlock += 1 - } -} -``` - - - - - -```python -def deposit_flow(deposit_address): - # some logic we want to execute on deposit - def on_deposit(deposit): - print(json.dumps(deposit, indent=2)) - - latest_block = network_status()["current_block_identifier"]["index"] - while True: - # check if transactions to the deposit address are present in the latest block - txs = wait_for_block(latest_block)['block']['transactions'] - deposits = [] - for tx in txs: - for op in tx["operations"]: - if op["account"]["address"] == deposit_address \ - and op["type"] == "payment_receiver_inc": - deposits.append({ - "tx_hash": tx["transaction_identifier"]["hash"], - "amount": op["amount"]["value"] - }) - - # process deposits - for d in deposits: - on_deposit(d) - - latest_block += 1 -``` - - - - diff --git a/docs/exchange-operators/rosetta/samples/using-signer.mdx b/docs/exchange-operators/rosetta/samples/using-signer.mdx deleted file mode 100644 index 2e8bfcd6b..000000000 --- a/docs/exchange-operators/rosetta/samples/using-signer.mdx +++ /dev/null @@ -1,80 +0,0 @@ ---- -title: Offline signer tool -description: Using signer tool for key pair generation and signing transactions ---- - -# Offline signer tool - -According to the Rosetta specification, you use an offline tool to sign transactions and use Rosetta to submit the signed transactions. -The Mina Rosetta implementation comes with an offline signer CLI tool and a [`mina-signer`](/mina-signer) npm package. To generate a key pair, sign a transaction and verify a signature with it. - -In a native build, the path to the signer tool is `_build/default/src/app/rosetta/ocaml-signer/signer.exe`. - -In a Docker container, the path is `/rosetta/app/mina-ocaml-signer`. - -To simplify examples, it is referred to as the `signer` alias. - -## Generating a key pair with `mina-signer` - -See [Generating Keys](/mina-signer#generating-keys) in the Mina Signer documentation for full details. A quick example: - -```ts -import { Client } from 'mina-signer' -const mina = new Client({network: 'testnet'}) -const keypair = mina.genKeys() -``` - -## Generating a key pair with signer CLI - -To generate a private key, run: - - signer generate-private-key - -To derive public key, run: - - signer derive-public-key --private-key `` - -In the output you will also get a Mina account address associated with this key pair. - -## Signing a transaction with `mina-signer` - -A sample code to sign a transaction: - -```ts -mina.signRosettaTransaction(payload, privateKey) -``` - -:::note - -We advice to use `mina.rosettaCombinePayload` helper to sign and prepare a payload for `/construction/combine` request - -See [Sending Transactions](send-transactions) page to learn more. - -::: - -## Signing a transaction with CLI - -To sign a transaction, run: - - signer sign --private-key `` --unsigned-transaction `` - -:::note - -Replace `` with the unsigned transaction string, which you can construct using `construction/payloads` endpoint. -See [Sending Transactions](send-transactions) page to learn more. - -::: - - -To implement a helper function to sign a transaction (in Python): - -```python -def sign_transaction(path_to_signer, private_key, unsigned_transaction): - cmd = "{signer} sign --private-key {pk} " \ - "--unsigned-transaction {tx_blob}".format( - signer=path_to_signer, - pk=private_key, - tx_blob=unsigned_transaction) - r = run(cmd.split(), stdout=PIPE).stdout - return r.decode().strip() -``` \ No newline at end of file diff --git a/docs/exchange-operators/rosetta/send-requests.mdx b/docs/exchange-operators/rosetta/send-requests.mdx deleted file mode 100644 index 5377a379e..000000000 --- a/docs/exchange-operators/rosetta/send-requests.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Sending requests -description: Learn about Mina-specific request/response models ---- - -# Sending requests - -After the Rosetta app is running, you can send requests to it. -To run a `/network/list` request: - - curl -L -X POST 'http://localhost:3087/network/list' \ - -H 'Content-Type: application/json' \ - -H 'Accept: application/json' -d '{}' - -In the case of successful setups, you see a response similar to this one: - - -```json -{"network_identifiers":[{"blockchain":"mina","network":"mainnet"}]} -``` - -You must pass the `network_identifier` object as a parameter to all other endpoints. In Mina Rosetta implementation, it exists only for the network you run Rosetta for, so this array always contains one object. - -Learn how to use Rosetta API for different use cases. See in the [Code Samples](samples) section. \ No newline at end of file diff --git a/docs/mina-protocol/index.mdx b/docs/mina-protocol/index.mdx index 30dd7ee2b..6b66f4e06 100644 --- a/docs/mina-protocol/index.mdx +++ b/docs/mina-protocol/index.mdx @@ -55,6 +55,6 @@ This section describes how the Mina Protocol works. [Node Developers](../node-developers) describes how developers can add to and improve Mina nodes. -## Exchange Operators +## Exchange Integration -[Exchange Operators](../exchange-operators/faq) describes how to connect to Mina networks and provide access to the MINA token. +[Exchange Operators](/node-operators/exchange-operators) describes how exchanges can integrate with the Mina blockchain using Rosetta API, archive nodes, and validator nodes. diff --git a/docs/mina-signer.mdx b/docs/mina-signer.mdx index c36ffcbc4..0e59f27bb 100644 --- a/docs/mina-signer.mdx +++ b/docs/mina-signer.mdx @@ -261,7 +261,7 @@ const signedGraphQLCommand = client.signedRosettaTransactionToSignedCommand(signedRosettaTx); ``` -For detailed Rosetta usage including the offline signer CLI tool and `signRosettaTransaction`, see the [Rosetta Offline Signer](/exchange-operators/rosetta/samples/using-signer) documentation. +For detailed Rosetta usage including the offline signer CLI tool and `signRosettaTransaction`, see the [Mina Signer for Node Operators](/node-operators/mina-signer) documentation. ## o1js Integration diff --git a/docs/node-operators/data-and-history/rosetta.mdx b/docs/node-operators/data-and-history/rosetta.mdx index 6c73430dc..1030d13e1 100644 --- a/docs/node-operators/data-and-history/rosetta.mdx +++ b/docs/node-operators/data-and-history/rosetta.mdx @@ -98,7 +98,7 @@ The all-in-one image supports the following environment variables for customizat For production deployments, use Docker Compose to run each component as a separate container. This gives you more control over resource allocation, logging, and restarts. -For production deployments, a complete Docker Compose configuration with all four services (PostgreSQL, archive node, Mina daemon, and Rosetta API) is available in the [Rosetta Docker Compose guide](/exchange-operators/rosetta/docker-compose). +For production deployments, a complete Docker Compose configuration with all four services (PostgreSQL, archive node, Mina daemon, and Rosetta API) is available in the [Rosetta Docker Compose guide](/node-operators/rosetta/docker-compose). ### Option 3: Debian Packages (Manual setup) diff --git a/docs/node-operators/exchange-operators.mdx b/docs/node-operators/exchange-operators.mdx new file mode 100644 index 000000000..14e59092e --- /dev/null +++ b/docs/node-operators/exchange-operators.mdx @@ -0,0 +1,49 @@ +--- +title: Exchange Operators +hide_title: true +description: Guide for exchanges integrating with the Mina blockchain using Rosetta API, archive nodes, and block producer nodes. +keywords: + - exchange + - listing mina + - rosetta + - archive node + - block producer + - integration + - mina protocol +--- + +# Exchange Operators + +import Subhead from "@site/src/components/common/Subhead"; + + +Exchange operators are node operators who run additional infrastructure for blockchain integration. + + +To list MINA on your exchange, you need to run a Mina node with archive and Rosetta API support. This page provides an overview of the components involved and links to the relevant setup guides. + +## Components + +### Validator Node + +A Mina daemon connected to the network. This is the base requirement for any node operator. See [Validator Node](/node-operators/validator-node) for setup instructions. + +### Archive Node + +An archive node stores the full history of the blockchain in a PostgreSQL database. Exchanges need this to query historical transactions and track deposits. + +- [Archive Node](/node-operators/archive-node) — set up and run an archive node + +### Rosetta API + +[Rosetta](https://www.rosetta-api.org/) is a standardized API for blockchain integration. It is the recommended way to integrate MINA deposits, withdrawals, and balance queries. + +- [Rosetta API Overview](/node-operators/data-and-history/rosetta) — architecture and available endpoints +- [Run with Docker](/node-operators/rosetta/run-with-docker) — quickest way to get started +- [Docker Compose](/node-operators/rosetta/docker-compose) — production setup with all services +- [Build from Sources](/node-operators/rosetta/build-from-sources) — build and run from source +- [Code Samples](/node-operators/rosetta/samples) — example integrations for deposits, withdrawals, and block scanning + +## FAQ + +See the [Exchange Integration FAQ](/node-operators/faq#exchange-integration) for answers to common questions about listing MINA, account creation fees, transaction memos, staking, and more. diff --git a/docs/node-operators/faq.mdx b/docs/node-operators/faq.mdx index e23906bb9..a3523d6e7 100644 --- a/docs/node-operators/faq.mdx +++ b/docs/node-operators/faq.mdx @@ -1,7 +1,7 @@ --- title: FAQ hide_title: true -description: Frequently asked questions about node operators in the Mina network. +description: Troubleshooting tips and answers to frequently asked questions about node operators and exchange integration in the Mina network. keywords: - FAQ - snark @@ -9,6 +9,9 @@ keywords: - block producer - mina protocol - mina signer + - exchange + - listing mina + - rosetta --- # FAQ @@ -82,3 +85,168 @@ No, when a new block is generated, Mina computes the proof recursively over the ### How do you control or limit the number of threads that SNARK workers use? When you start the mina daemon, use the `-snark-worker-parallelism` flag. This flag is equivalent to setting `OMP_NUM_THREADS`, but doesn't affect block production. + +## Exchange Integration + +#### Where can I find third-party audit reports for Mina? + +The latest third-party audit reports are publicly available here: + +- [https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review](https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review/) +- [https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs](https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs/) +- [https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review](https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review) + +:::note + +Any news and updates related to exchange listing shared by the Mina Foundation are on [www.minaprotocol.com](https://minaprotocol.com) or the official [Mina Protocol](https://x.com/MinaProtocol) X (Twitter). Mina Foundation cannot individually answer any listing questions. + +::: + +#### Why do you recommend using Rosetta for integrating Mina to our exchange? + +Rosetta is an open-source specification that helps exchanges and developers integrate blockchains. Since Rosetta is actively maintained and specifically designed to enable simpler, faster, and more reliable blockchain integrations, we highly recommend using Rosetta to integrate Mina blockchain with your exchange. + +#### Is there an account creation fee? + +Yes, Mina Protocol charges a fee of 1 MINA when you create a new account. This fee helps protect the network from denial of service-type attacks. Over time, this fee can change. + +#### What is the maximum size of the mempool? How do we work around this? + +The max mempool size is 3,000. After it hits that size, transactions with the lowest fees are discarded. + +Set your fee to an amount higher than 0.001 MINA, the current average fee for transactions in the pool. + +You can view the fees for pending transactions and adjust your fees accordingly: [https://minascan.io/mainnet/txs/pending-txs](https://minascan.io/mainnet/txs/pending-txs). + +#### Why do some users appear to have lost their funds when sending to exchanges? + +:::tip + +While Mina and its SDKs do support the memo field when sending a transaction, the recommended best practice is do NOT require a memo for deposits. + +::: + +To associate the deposit with the user's account, some exchanges require their users to include a unique memo field when sending MINA deposits to the exchange's address. + +If the user does not include this unique memo when sending their deposit, the receiving exchange may not be able to associate the deposit properly with the user's exchange account. + +These funds are NOT lost. The exchanges have received the funds at the exchange's address, but the exchange may not be able to automatically associate the deposit with the user's exchange account without such a memo. + +To prevent this issue, we recommend that exchanges do NOT require a memo for deposits. At the same time, exchanges and wallet creators are recommended to expose an optional memo field during a Mina send transaction. + +#### What is the maximum number of rollback blocks? + +The table in [Lifecycle of a Payment](/mina-protocol/lifecycle-of-a-payment) describes how many blocks you wait for a transaction to be confirmed. + +#### How should I calculate transaction fees? + +To calculate your transaction fees, use [https://fees.mina.tools](https://fees.mina.tools/). + +#### My Mina node gets stuck sometimes. How can I detect this and fix it? + +This is a known issue for some earlier releases. Restart your mina node whenever this issue is detected. + +You can use the following script to run a cron job every 3 minutes (the slot length) or more frequently: + +``` +MINA_STATUS=$($MINA client status --json) +HIGHESTBLOCK="$(echo $MINA_STATUS | jq .highest_block_length_received)" +HIGHESTUNVALIDATEDBLOCK="$(echo $MINA_STATUS | jq .highest_unvalidated_block_length_received)" + +# Calculate difference between validated and unvalidated blocks. +# If block height is more than 4 block behind, something is likely wrong. + +DELTAVALIDATED="$(($HIGHESTUNVALIDATEDBLOCK-$HIGHESTBLOCK))" +if [[ "$DELTAVALIDATED" -gt 4 ]]; then + $MINA client stop +fi +``` + +:::tip + +Be sure your Mina daemon is monitored by something such as systemd, so it can auto-restart. + +::: + +#### My archive node is missing block information after a restart. How can I recover the data? + +Archive node operators often choose to run redundant archive nodes to store block data to one or more locations of their choice (for example, PostgreSQL, GCP, local files, or a logging service) and to backfill any missed block data if needed. + +For convenience, [mina_network_block_data](https://console.cloud.google.com/storage/browser/mina_network_block_data) from the archive node is available to help others in the community backfill any missing information. + +This bucket contains blocks from various Mina networks — for example, Mainnet and the most recent Devnet `devnet2`. Filter by filename for the network you want. Note that this bucket contains blocks for various other networks too, such as QAnet, which is not recommended for your testing. QAnet is used by o1Labs during targeted iterative development. + +Filenames contain the network name, block height, and state hash of the block. Blocks older than height 25,705 include only the network name and state hash in the filename. + +Example filenames: + +(Recent) + +``` +mainnet-30627-3NLfKanQ53X2MRKx5ZRvb9nVCEB9eJpcnssGCTpT3J1cojhB5M19.json +``` + +(Older) + +``` +mainnet-3NKUBmkc7UZ7ik5JyCM4WNzkN1HG5heMB5zNDUkf3Kgta1MFY6LY.json +``` + +You can download a specific block using curl: + +``` +curl https://mina_network_block_data.storage.googleapis.com/ +``` + +You can import this file using the mina archive blocks tool. The command for it is: + +``` +mina-archive-blocks --precomputed --archive-uri FILE. +``` + +#### How do I query for the canonical block at a certain height from the archive node? + +Use a recursive query. See [Query the database](/node-operators/archive-node/getting-started#query-the-database) examples in the Archive Node docs. + +#### Why am I getting this error message: "Not able to connect to the network"? + +This error message usually occurs due to a chain ID mismatch from running a Devnet build on Mainnet, or vice versa. + +To check whether you are running a devnet or mainnet build, run `Mina client status`. Next, compare the output's chain ID of your node to the expected chain ID of the network you are trying to connect to. You can find required information for comparison within the [GitHub announcements](https://github.com/MinaProtocol/mina/discussions/categories/announcements) or [Discord](https://discord.com/channels/484437221055922177/601171209287368715) server. + +#### Are there any official broadcast nodes that can be used? + +No, there are no official broadcast nodes at this time. However, you can broadcast transactions using [https://minascan.io/mainnet/broadcast/payment](https://minascan.io/mainnet/broadcast/payment). Use this method as a backup, the recommended method is to broadcast transactions yourself. + +#### Should I be staking my funds? + +Since Mina is a Proof of Stake (PoS) consensus network without lockup for staked tokens, it is recommended to stake these funds to support the quality of the Mina network. Additionally, by not staking, you are missing out on staking rewards that you can otherwise be receiving from the Mina blockchain. + +You can look into staking this wallet, either by running your own block production node or just by delegating your funds to a staking pool on the network. Delegating to a staking pool is simpler to set up. + +:::note + +Newly staked accounts incur a delay of 18 to 29 days before you start receiving rewards. + +::: + +#### Why is there a delay for staking to take effect? + +For purposes of ensuring consensus, there is a delay between when delegations are sent on the blockchain and when they take effect with respect to staking on the network. The staking ledger always operates between 18 to 29 days behind the live ledger. + +#### How long is the delay and when is the next staking snapshot? + +The timing of the next staking snapshot varies. + +Since the timing is based on a combination of consensus timing (epochs) and snarketplace throughput, it is difficult to determine exactly how long this delay can be. + +A conservative estimate is that delegations sent 3 days before the epoch transition can take effect in the upcoming epoch. This means that, for any given delegation, there is an average of 18 to 29 days delay before this delegation updates block production. + +You can use this Delegation Calculator tool built by Carbonara to see the next staking ledger cutoff: +[https://epoch.mina.tools](https://epoch.mina.tools/). + +#### What is the best way to test tooling and integration with Mina? + +Test tooling and integrations on Devnet before going live on Mainnet. The Devnet network is dedicated for developers building on top of the Mina protocol and is designed for testing and experimentation. Be sure to simulate expected Mainnet conditions, such as transaction volume and frequency, to help identify and solve potential issues ahead of time. + +See [Connect to Devnet](/node-operators/validator-node/connecting-to-the-network). diff --git a/docs/node-operators/index.mdx b/docs/node-operators/index.mdx index 97df19d4e..58512ccfc 100644 --- a/docs/node-operators/index.mdx +++ b/docs/node-operators/index.mdx @@ -59,8 +59,12 @@ This section describes how to run each node role and where to find the operation ### Reference - [Mina CLI Reference](/node-operators/reference/mina-cli-reference) - Guide to CLI interactions with Mina networks -- [Mina Signer](/node-operators/reference/mina-signer) - NodeJS SDK for signing +- [Mina Signer](/node-operators/mina-signer) - Key generation and transaction signing ### Support - [Troubleshooting](/node-operators/troubleshooting) - Solutions to common problems - [FAQ](/node-operators/faq) - Frequently asked questions + +## Exchange Integration + +[Exchange Operators](/node-operators/exchange-operators) describes how exchanges can integrate with the Mina blockchain using Rosetta API, archive nodes, and validator nodes. diff --git a/docs/node-operators/mina-signer.mdx b/docs/node-operators/mina-signer.mdx new file mode 100644 index 000000000..27cedbea4 --- /dev/null +++ b/docs/node-operators/mina-signer.mdx @@ -0,0 +1,71 @@ +--- +title: Mina Signer +hide_title: true +description: Key generation and transaction signing for node operators using mina-signer and the Rosetta offline signer CLI. +keywords: + - mina signer + - sign + - signature + - rosetta + - offline signer + - key pair +--- + +# Mina Signer + +[Mina Signer](/mina-signer) is a NodeJS/Browser-compatible library for signing transactions and generating keys. The Rosetta stack also ships an offline signer CLI tool. Both can be used to sign transactions offline for later submission via the [Construction API](/node-operators/rosetta/samples/send-transactions). + +## Migration from o1labs/client-sdk + +The signing library `o1labs/client-sdk` is deprecated and will stop working after the Mina mainnet upgrade. All users should migrate to [mina-signer](https://www.npmjs.com/package/mina-signer). + +When migrating: + +1. Adjust the `nonce` to the correct nonce on the sender account +2. Update the `url` variable with an existing Mina Node GraphQL endpoint + +See [Broadcasting a Signed Payment](/mina-signer#broadcasting-a-signed-payment) for a complete example. + +## Generating a key pair with mina-signer + +See [Generating Keys](/mina-signer#generating-keys) for full details. A quick example: + +```ts +import { Client } from 'mina-signer' +const mina = new Client({network: 'testnet'}) +const keypair = mina.genKeys() +``` + +## Generating a key pair with signer CLI + +In a native build, the signer is at `_build/default/src/app/rosetta/ocaml-signer/signer.exe`. In Docker, it's at `/rosetta/app/mina-ocaml-signer`. Examples below use `signer` as an alias. + +Generate a private key: + +```bash +signer generate-private-key +``` + +Derive the public key and account address: + +```bash +signer derive-public-key --private-key +``` + +## Signing a transaction with mina-signer + +```ts +mina.signRosettaTransaction(payload, privateKey) +``` + +:::note +We recommend using `mina.rosettaCombinePayload` to sign and prepare a payload for the `/construction/combine` request. See [Sending Transactions](/node-operators/rosetta/samples/send-transactions) for the full flow. +::: + +## Signing a transaction with signer CLI + +```bash +signer sign --private-key --unsigned-transaction +``` + +Replace `` with the unsigned transaction string from the `/construction/payloads` endpoint. See [Sending Transactions](/node-operators/rosetta/samples/send-transactions) for the full flow. diff --git a/docs/node-operators/reference/index.mdx b/docs/node-operators/reference/index.mdx index bd3017428..6922a5653 100644 --- a/docs/node-operators/reference/index.mdx +++ b/docs/node-operators/reference/index.mdx @@ -2,20 +2,15 @@ title: Reference sidebar_label: Reference hide_title: true -description: Reference documentation for Mina node operators, including CLI reference, troubleshooting, and FAQ. +description: Reference documentation for Mina node operators. keywords: - CLI reference - - troubleshooting - - FAQ - - mina signer - node operators --- # Reference -Reference documentation and support resources for Mina node operators. - - **[Mina CLI Reference](/node-operators/reference/mina-cli-reference)** -- Complete reference for the `mina` command line interface. -- **[Mina Signer](/node-operators/reference/mina-signer)** -- NodeJS SDK for signing payments and delegations. +- **[Mina Signer](/node-operators/mina-signer)** -- Key generation and transaction signing for node operators. - **[Troubleshooting](/node-operators/troubleshooting)** -- Solutions to common problems when running a Mina node. - **[FAQ](/node-operators/faq)** -- Frequently asked questions about node operations, SNARKs, and the Mina network. diff --git a/docs/node-operators/reference/mina-signer.mdx b/docs/node-operators/reference/mina-signer.mdx deleted file mode 100644 index 001c9039d..000000000 --- a/docs/node-operators/reference/mina-signer.mdx +++ /dev/null @@ -1,25 +0,0 @@ ---- -title: Mina Signer for Node Operators -hide_title: true -description: Using Mina Signer to sign and broadcast payments and delegations for node operators. -keywords: - - mina signer - - sign - - signature - - node operators ---- - -# Mina Signer for Node Operators - -[Mina Signer](/mina-signer) is a NodeJS SDK that allows you to sign strings, payments, and delegations using Mina's key pairs for various specified networks. See the full [Mina Signer documentation](/mina-signer) for installation, API details, and usage with o1js and Rosetta. - -## Migration from o1labs/client-sdk to mina-signer - -The signing library `o1labs/client-sdk` was deprecated some time ago and will stop working after the Mina mainnet upgrade. All users should upgrade to use the [mina-signer](https://www.npmjs.com/package/mina-signer) library. - -Please keep in mind the following: - -1. Make sure to adjust the `nonce` to the correct nonce on the account you want to use as "sender" -1. Update the `url` variable with an existing Mina Node GraphQL endpoint - -See the [Broadcasting a Signed Payment](/mina-signer#broadcasting-a-signed-payment) section for a complete example of signing and broadcasting a payment via GraphQL. diff --git a/docs/exchange-operators/rosetta/build-from-sources.mdx b/docs/node-operators/rosetta/build-from-sources.mdx similarity index 100% rename from docs/exchange-operators/rosetta/build-from-sources.mdx rename to docs/node-operators/rosetta/build-from-sources.mdx diff --git a/docs/exchange-operators/rosetta/docker-compose.mdx b/docs/node-operators/rosetta/docker-compose.mdx similarity index 100% rename from docs/exchange-operators/rosetta/docker-compose.mdx rename to docs/node-operators/rosetta/docker-compose.mdx diff --git a/docs/exchange-operators/rosetta/run-with-docker.mdx b/docs/node-operators/rosetta/run-with-docker.mdx similarity index 100% rename from docs/exchange-operators/rosetta/run-with-docker.mdx rename to docs/node-operators/rosetta/run-with-docker.mdx diff --git a/docs/node-operators/rosetta/samples/index.mdx b/docs/node-operators/rosetta/samples/index.mdx new file mode 100644 index 000000000..e5a3c3528 --- /dev/null +++ b/docs/node-operators/rosetta/samples/index.mdx @@ -0,0 +1,21 @@ +--- +title: Code Samples +description: Curl-based examples for common Rosetta API operations +--- + +# Code Samples + +These samples use `curl` and [`jq`](https://jqlang.github.io/jq/) to interact with the Rosetta API. They assume a running Rosetta instance on `localhost:3087`. + +Set these shell variables before running the examples: + +```bash +ROSETTA_URL="http://localhost:3087" +NETWORK='{"blockchain":"mina","network":"mainnet"}' +``` + +All endpoints except `/network/list` require a `network_identifier` parameter. The samples include it in each request body. + +:::tip +Replace `mainnet` with `devnet` if you are testing against a devnet instance. +::: diff --git a/docs/node-operators/rosetta/samples/requests.mdx b/docs/node-operators/rosetta/samples/requests.mdx new file mode 100644 index 000000000..9fa3e2e14 --- /dev/null +++ b/docs/node-operators/rosetta/samples/requests.mdx @@ -0,0 +1,188 @@ +--- +title: Requests and Responses +description: Mina-specific Rosetta request and response objects with curl examples +--- + +# Requests and Responses + +The Rosetta API specification defines high-level descriptions of request and response objects. Exact JSON layouts differ between blockchains. This page covers Mina-specific objects and shows how to query each endpoint with curl. + +All examples assume the shell variables from the [Code Samples](/node-operators/rosetta/samples) setup. + +## Network endpoints + +List available networks: + +```bash +curl -s "$ROSETTA_URL/network/list" \ + -H 'Content-Type: application/json' \ + -d '{"metadata":{}}' | jq . +``` + +Sample response: + +```json +{"network_identifiers":[{"blockchain":"mina","network":"mainnet"}]} +``` + +You must pass the `network_identifier` object as a parameter to all other endpoints. In Mina's Rosetta implementation, it exists only for the network you run Rosetta for, so this array always contains one object. + +Get network status (current block height, sync state, peers): + +```bash +curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq . +``` + +Get supported options and operation types: + +```bash +curl -s "$ROSETTA_URL/network/options" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq . +``` + +## Block and transaction queries + +Fetch a block by index: + +```bash +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":1000}}" | jq . +``` + +Fetch a block by hash: + +```bash +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"hash\":\"BLOCK_HASH\"}}" | jq . +``` + +List pending transactions in the mempool: + +```bash +curl -s "$ROSETTA_URL/mempool" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq . +``` + +## Account queries + +Query an account balance: + +```bash +curl -s "$ROSETTA_URL/account/balance" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"account_identifier\":{\"address\":\"B62qr...\",\"token_id\":\"wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf\"} + }" | jq . +``` + +Search for transactions by address: + +```bash +curl -s "$ROSETTA_URL/search/transactions" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"address\":\"B62qr...\"}" | jq . +``` + +Search for a specific transaction by hash: + +```bash +curl -s "$ROSETTA_URL/search/transactions" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"transaction_identifier\":{\"hash\":\"CkpY...\"}}" | jq . +``` + +Derive an account address from a public key: + +```bash +curl -s "$ROSETTA_URL/construction/derive" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"public_key\":{\"hex_bytes\":\"PUBLIC_KEY_HEX\",\"curve_type\":\"pallas\"} + }" | jq . +``` + +## Operation object + +In Rosetta terminology, each transaction consists of one or more operations. In Mina's implementation, each operation has: + +- `operation_identifier` and `related_operations`: a mandatory index, and optional array of related operations +- `type`: the operation type +- `account`: the account identifier the operation relates to +- `amount`: an object with `value` — a signed number representing the balance change + +
+Sample operation JSON + +```json +{ + "operation_identifier": { "index": 2 }, + "related_operations": [{ "index": 1 }], + "type": "payment_receiver_inc", + "account": { + "address": "B62qqJ1AqK3YQmEEALdJeMw49438Sh6zuQ5cNWUYfCgRsPkduFE2uLU", + "metadata": { "token_id": "1" } + }, + "amount": { + "value": "90486110", + "currency": { "symbol": "MINA", "decimals": 9 } + } +} +``` + +
+ +:::note +All possible operation types are available from the `/network/options` endpoint. The most common types are `fee_payment`, `payment_source_dec`, and `payment_receiver_inc`. +::: + +## Transfer transaction layout + +A MINA token transfer is represented by three operations (account updates): + +1. Decrease fee payer balance (fee payer = sender) +2. Decrease sender balance by the transfer amount +3. Increase receiver balance by the transfer amount + +
+Sample transfer transaction JSON + +```json +{ + "transaction_identifier": { + "hash": "CkpYVELyYvzbyAwYcnMQryEeQ7Gd6Ws7mZNXpmF5kEAyvwoTiUfbX" + }, + "operations": [ + { + "operation_identifier": { "index": 0 }, + "type": "fee_payment", + "account": { "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", "metadata": { "token_id": "1" } }, + "amount": { "value": "-37000000", "currency": { "symbol": "MINA", "decimals": 9 } } + }, + { + "operation_identifier": { "index": 1 }, + "type": "payment_source_dec", + "account": { "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", "metadata": { "token_id": "1" } }, + "amount": { "value": "-58486000", "currency": { "symbol": "MINA", "decimals": 9 } } + }, + { + "operation_identifier": { "index": 2 }, + "related_operations": [{ "index": 1 }], + "type": "payment_receiver_inc", + "account": { "address": "B62qkiF5CTjeiuV1HSx4SpEytjiCptApsvmjiHHqkb1xpAgVuZTtR14", "metadata": { "token_id": "1" } }, + "amount": { "value": "58486000", "currency": { "symbol": "MINA", "decimals": 9 } } + } + ] +} +``` + +
+ +This operations array is what you pass to `/construction/preprocess` and `/construction/payloads` when building a transfer. See [Sending Transactions](send-transactions) for the full flow. diff --git a/docs/node-operators/rosetta/samples/scan-blocks.mdx b/docs/node-operators/rosetta/samples/scan-blocks.mdx new file mode 100644 index 000000000..ac0cff3e3 --- /dev/null +++ b/docs/node-operators/rosetta/samples/scan-blocks.mdx @@ -0,0 +1,48 @@ +--- +title: Scanning Blocks +description: Poll for new blocks and inspect transactions using curl +--- + +# Scanning Blocks + +To poll for new blocks, query `/network/status` for the current block height, then fetch each block sequentially. + +Get the current block height: + +```bash +curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq '.current_block_identifier.index' +``` + +Fetch a specific block by index: + +```bash +BLOCK_INDEX=1000 + +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$BLOCK_INDEX}}" | jq . +``` + +A simple polling loop that waits for new blocks: + +```bash +HEIGHT=$(curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq '.current_block_identifier.index') + +while true; do + BLOCK=$(curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$HEIGHT}}") + + if echo "$BLOCK" | jq -e '.block' > /dev/null 2>&1; then + echo "Block $HEIGHT:" + echo "$BLOCK" | jq '.block.transactions[] | .transaction_identifier.hash' + HEIGHT=$((HEIGHT + 1)) + else + sleep 10 + fi +done +``` diff --git a/docs/node-operators/rosetta/samples/send-transactions.mdx b/docs/node-operators/rosetta/samples/send-transactions.mdx new file mode 100644 index 000000000..fb9abe598 --- /dev/null +++ b/docs/node-operators/rosetta/samples/send-transactions.mdx @@ -0,0 +1,159 @@ +--- +title: Sending Transactions +description: Build, sign, and submit a MINA transfer using the Rosetta Construction API +--- + +# Sending Transactions + +:::info +This flow follows the [Construction API Overview](https://docs.cloud.coinbase.com/rosetta/docs/construction-api-overview) from the official Rosetta documentation. +::: + +The steps to send a MINA payment: + +1. Derive the account address from a public key +2. Build the unsigned transaction via preprocess → metadata → payloads +3. Sign offline with the [signer tool](/node-operators/mina-signer) +4. Combine the signature into a signed blob +5. Submit the signed transaction + +## Prerequisites + +- A key pair generated with the [offline signer tool](/node-operators/mina-signer) +- The account must have a balance (send test funds on devnet first) +- Set the shell variables from the [Code Samples](/node-operators/rosetta/samples) setup + +Set your keys and transfer parameters: + +```bash +PUBLIC_KEY="YOUR_PUBLIC_KEY_HEX" +SENDER="B62q..." +RECEIVER="B62q..." +TOKEN_ID="wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf" +FEE=10000000 # 0.01 MINA in nanomina +VALUE=1000000000 # 1 MINA in nanomina +``` + +## Step 1: Derive account address + +```bash +curl -s "$ROSETTA_URL/construction/derive" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"public_key\":{\"hex_bytes\":\"$PUBLIC_KEY\",\"curve_type\":\"pallas\"} + }" | jq . +``` + +## Step 2: Build the operations payload + +Construct the three operations that represent a MINA transfer (see [Requests and Responses](requests#transfer-transaction-layout) for details on the structure): + +```bash +OPERATIONS='[ + { + "operation_identifier":{"index":0}, + "type":"fee_payment", + "account":{"address":"'"$SENDER"'","metadata":{"token_id":"'"$TOKEN_ID"'"}}, + "amount":{"value":"-'"$FEE"'","currency":{"symbol":"MINA","decimals":9}} + }, + { + "operation_identifier":{"index":1}, + "type":"payment_source_dec", + "account":{"address":"'"$SENDER"'","metadata":{"token_id":"'"$TOKEN_ID"'"}}, + "amount":{"value":"-'"$VALUE"'","currency":{"symbol":"MINA","decimals":9}} + }, + { + "operation_identifier":{"index":2}, + "related_operations":[{"index":1}], + "type":"payment_receiver_inc", + "account":{"address":"'"$RECEIVER"'","metadata":{"token_id":"'"$TOKEN_ID"'"}}, + "amount":{"value":"'"$VALUE"'","currency":{"symbol":"MINA","decimals":9}} + } +]' +``` + +## Step 3: Preprocess + +```bash +PREPROCESS=$(curl -s "$ROSETTA_URL/construction/preprocess" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"operations\":$OPERATIONS}") + +echo "$PREPROCESS" | jq . +``` + +## Step 4: Metadata + +```bash +METADATA=$(curl -s "$ROSETTA_URL/construction/metadata" \ + -H 'Content-Type: application/json' \ + -d "$(echo "$PREPROCESS" | jq -c ". + { + \"network_identifier\":$NETWORK, + \"public_keys\":[{\"hex_bytes\":\"$PUBLIC_KEY\",\"curve_type\":\"pallas\"}] + }")") + +echo "$METADATA" | jq . +``` + +## Step 5: Payloads (unsigned transaction) + +```bash +PAYLOADS=$(curl -s "$ROSETTA_URL/construction/payloads" \ + -H 'Content-Type: application/json' \ + -d "$(echo "$METADATA" | jq -c ". + { + \"network_identifier\":$NETWORK, + \"operations\":$OPERATIONS + }")") + +echo "$PAYLOADS" | jq . +UNSIGNED_TX=$(echo "$PAYLOADS" | jq -r '.unsigned_transaction') +``` + +## Step 6: Sign offline + +Use the [signer CLI tool](/node-operators/mina-signer#signing-a-transaction-with-signer-cli): + +```bash +SIGNATURE=$(signer sign --private-key "$PRIVATE_KEY" --unsigned-transaction "$UNSIGNED_TX") +``` + +## Step 7: Combine + +```bash +SIGNING_PAYLOAD=$(echo "$PAYLOADS" | jq -c '.payloads[0]') + +COMBINE=$(curl -s "$ROSETTA_URL/construction/combine" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"unsigned_transaction\":\"$UNSIGNED_TX\", + \"signatures\":[{ + \"signing_payload\":$SIGNING_PAYLOAD, + \"public_key\":{\"hex_bytes\":\"$PUBLIC_KEY\",\"curve_type\":\"pallas\"}, + \"signature_type\":\"schnorr_poseidon\", + \"hex_bytes\":\"$SIGNATURE\" + }] + }") + +SIGNED_TX=$(echo "$COMBINE" | jq -r '.signed_transaction') +echo "$COMBINE" | jq . +``` + +## Step 8: Get transaction hash + +```bash +curl -s "$ROSETTA_URL/construction/hash" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"signed_transaction\":\"$SIGNED_TX\"}" | jq . +``` + +## Step 9: Submit + +```bash +curl -s "$ROSETTA_URL/construction/submit" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"signed_transaction\":\"$SIGNED_TX\"}" | jq . +``` + +After submission, you can monitor for confirmation using the [block scanning](scan-blocks) approach — poll blocks until your transaction hash appears. diff --git a/docs/node-operators/rosetta/samples/track-deposits.mdx b/docs/node-operators/rosetta/samples/track-deposits.mdx new file mode 100644 index 000000000..e15772197 --- /dev/null +++ b/docs/node-operators/rosetta/samples/track-deposits.mdx @@ -0,0 +1,65 @@ +--- +title: Tracking Deposits +description: Monitor an address for incoming MINA deposits using curl +--- + +# Tracking Deposits + +To track deposits, scan each block for `payment_receiver_inc` operations matching your deposit address. + +Fetch a block and filter for deposits to a specific address: + +```bash +DEPOSIT_ADDRESS="B62qr..." +BLOCK_INDEX=1000 + +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$BLOCK_INDEX}}" \ + | jq --arg addr "$DEPOSIT_ADDRESS" ' + .block.transactions[] + | { + tx_hash: .transaction_identifier.hash, + deposits: [ + .operations[] + | select(.account.address == $addr and .type == "payment_receiver_inc") + | { amount: .amount.value } + ] + } + | select(.deposits | length > 0) + ' +``` + +A continuous deposit monitoring loop: + +```bash +DEPOSIT_ADDRESS="B62qr..." + +HEIGHT=$(curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq '.current_block_identifier.index') + +while true; do + BLOCK=$(curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$HEIGHT}}") + + if echo "$BLOCK" | jq -e '.block' > /dev/null 2>&1; then + echo "$BLOCK" | jq --arg addr "$DEPOSIT_ADDRESS" ' + .block.transactions[] + | { + tx_hash: .transaction_identifier.hash, + deposits: [ + .operations[] + | select(.account.address == $addr and .type == "payment_receiver_inc") + | { amount: .amount.value } + ] + } + | select(.deposits | length > 0) + ' + HEIGHT=$((HEIGHT + 1)) + else + sleep 10 + fi +done +``` diff --git a/docs/node-operators/validator-node/generating-a-keypair.mdx b/docs/node-operators/validator-node/generating-a-keypair.mdx index 3adea6a6e..8162859bf 100644 --- a/docs/node-operators/validator-node/generating-a-keypair.mdx +++ b/docs/node-operators/validator-node/generating-a-keypair.mdx @@ -100,7 +100,7 @@ You can use your [Ledger Nano S](https://www.ledger.com/) hardware wallet to sec ### Mina Signer -You can also use [Mina Signer](/node-operators/reference/mina-signer) to generate key pairs and sign transactions. +You can also use [Mina Signer](/node-operators/mina-signer) to generate key pairs and sign transactions. ## Validate your private key diff --git a/docusaurus.config.js b/docusaurus.config.js index d410eeaf0..418442b9b 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -214,7 +214,67 @@ module.exports = { }, { from: '/exchange-operators/exchange-faq', - to: '/exchange-operators/faq', + to: '/node-operators/faq', + }, + { + from: '/exchange-operators/faq', + to: '/node-operators/faq', + }, + { + from: '/exchange-operators', + to: '/node-operators/exchange-operators', + }, + { + from: '/exchange-operators/rosetta/run-with-docker', + to: '/node-operators/rosetta/run-with-docker', + }, + { + from: '/exchange-operators/rosetta/docker-compose', + to: '/node-operators/rosetta/docker-compose', + }, + { + from: '/exchange-operators/rosetta/build-from-sources', + to: '/node-operators/rosetta/build-from-sources', + }, + { + from: '/exchange-operators/rosetta/send-requests', + to: '/node-operators/rosetta/samples/requests', + }, + { + from: '/node-operators/rosetta/send-requests', + to: '/node-operators/rosetta/samples/requests', + }, + { + from: '/exchange-operators/rosetta/samples', + to: '/node-operators/rosetta/samples', + }, + { + from: '/exchange-operators/rosetta/samples/requests', + to: '/node-operators/rosetta/samples/requests', + }, + { + from: '/exchange-operators/rosetta/samples/using-signer', + to: '/node-operators/mina-signer', + }, + { + from: '/node-operators/rosetta/samples/using-signer', + to: '/node-operators/mina-signer', + }, + { + from: '/node-operators/reference/mina-signer', + to: '/node-operators/mina-signer', + }, + { + from: '/exchange-operators/rosetta/samples/scan-blocks', + to: '/node-operators/rosetta/samples/scan-blocks', + }, + { + from: '/exchange-operators/rosetta/samples/track-deposits', + to: '/node-operators/rosetta/samples/track-deposits', + }, + { + from: '/exchange-operators/rosetta/samples/send-transactions', + to: '/node-operators/rosetta/samples/send-transactions', }, { from: '/zkapps/snarkyjs-reference', @@ -304,10 +364,6 @@ module.exports = { from: '/node-operators/mina-cli-reference', to: '/node-operators/reference/mina-cli-reference', }, - { - from: '/node-operators/mina-signer', - to: '/node-operators/reference/mina-signer', - }, { from: '/node-operators/reference/troubleshooting', to: '/node-operators/troubleshooting', diff --git a/sidebars.js b/sidebars.js index 84fa9ed2f..df6506e5e 100644 --- a/sidebars.js +++ b/sidebars.js @@ -1557,33 +1557,51 @@ module.exports = { }, { type: 'category', - label: 'Delegation Program', - link: { - type: 'doc', - id: 'node-operators/delegation-program/index', - }, + label: 'Rosetta API', items: [ - 'node-operators/delegation-program/foundation-delegation-program', - 'node-operators/delegation-program/uptime-tracking-system', + 'node-operators/rosetta/run-with-docker', + 'node-operators/rosetta/docker-compose', + 'node-operators/rosetta/build-from-sources', + { + type: 'category', + label: 'Code Samples', + link: { + type: 'doc', + id: 'node-operators/rosetta/samples/index', + }, + items: [ + 'node-operators/rosetta/samples/requests', + 'node-operators/rosetta/samples/scan-blocks', + 'node-operators/rosetta/samples/track-deposits', + 'node-operators/rosetta/samples/send-transactions', + ], + }, ], }, { type: 'category', - label: 'Reference', + label: 'Delegation Program', link: { type: 'doc', - id: 'node-operators/reference/index', + id: 'node-operators/delegation-program/index', }, items: [ - 'node-operators/reference/mina-cli-reference', - 'node-operators/reference/mina-signer', + 'node-operators/delegation-program/foundation-delegation-program', + 'node-operators/delegation-program/uptime-tracking-system', ], }, + 'node-operators/mina-signer', + 'node-operators/reference/mina-cli-reference', 'node-operators/downgrading-to-older-versions', 'node-operators/troubleshooting', 'node-operators/faq', ], }, + { + type: 'doc', + label: 'Exchange Operators', + id: 'node-operators/exchange-operators', + }, { type: 'category', label: 'Node Developers', @@ -1602,42 +1620,6 @@ module.exports = { 'node-developers/contributing', ], }, - { - type: 'category', - label: 'Exchange Operators', - link: { - type: 'doc', - id: 'exchange-operators/index', - }, - items: [ - { - type: 'category', - label: 'Rosetta API', - items: [ - 'exchange-operators/rosetta/run-with-docker', - 'exchange-operators/rosetta/docker-compose', - 'exchange-operators/rosetta/build-from-sources', - 'exchange-operators/rosetta/send-requests', - { - type: 'category', - label: 'Code Samples', - link: { - type: 'doc', - id: 'exchange-operators/rosetta/samples/index', - }, - items: [ - 'exchange-operators/rosetta/samples/requests', - 'exchange-operators/rosetta/samples/using-signer', - 'exchange-operators/rosetta/samples/scan-blocks', - 'exchange-operators/rosetta/samples/track-deposits', - 'exchange-operators/rosetta/samples/send-transactions', - ], - }, - ], - }, - 'exchange-operators/faq', - ], - }, { type: 'category', label: 'Developer Tools', diff --git a/static/llms-full.txt b/static/llms-full.txt index eb441ca52..bfab5df49 100644 --- a/static/llms-full.txt +++ b/static/llms-full.txt @@ -1,9010 +1,7725 @@ --- -url: /exchange-operators/faq +url: /glossary --- -# FAQ Listing Mina +# Glossary - -Frequently asked questions about listing Mina. - +[A](#a) [B](#b) [C](#c) [D](#d) [E](#e) [F](#f) G [H](#h) [I](#i) J [K](#k) [L](#l) [M](#m) [N](#n) [O](#o) +[P](#p) Q [R](#r) [S](#s) [T](#t) [U](#u) [V](#v) W X Y [Z](#z) -## Basics +Are you looking for a zero knowledge, o1js, zkApp, or Mina term that isn't here yet? To let us know, please [create an issue](https://github.com/o1-labs/docs2/issues) or click **EDIT THIS PAGE** to submit a PR. -### Where can I find third-party audit reports for Mina? +## A -The latest third-party audit reports are publicly available here: +### account -- [https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review](https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review/) -- [https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs](https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs/) -- [https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review](https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review) +Mina uses accounts to track each public key's state. This is distinct from Bitcoin's UTXO model of maintaining ledger state. -

+### archive node -:::note +A Mina node that stores the historical chain data to a persistent data source so it can later be retrieved. A zkApp can retrieve events and actions from one or more Mina [archive nodes](/node-operators/archive-node/getting-started). -Any news and updates related to exchange listing shared by the Mina Foundation are on [www.minaprotocol.com](https://minaprotocol.com) or the official [Mina Protocol](https://x.com/MinaProtocol) X (Twitter). Mina Foundation cannot individually answer any listing questions. +## B -::: +### Base58 -## Rosetta +A group of encoding/decoding schemes used to switch data between binary format (hexadecimal) and alphanumeric text format (ASCII). The Base58 alphabet includes numbers (1 to 9) and English letters, except O (uppercase o), I (uppercase i), and l (lowercase L). These letters are omitted to avoid confusion. -### Why do you recommend using Rosetta for integrating Mina to our exchange? +### Base64 -Rosetta is an open-source specification that helps exchanges and developers integrate blockchains. Since Rosetta is actively maintained and specifically designed to enable simpler, faster, and more reliable blockchain integrations, we highly recommend using Rosetta to integrate Mina blockchain with your exchange. +A binary-to-text encoding scheme that represents binary data in a human-readable ASCII string format string. For example, the zero knowledge proof is a Base64 string inside the `authorization` field of a transaction. -### What if I have a question about Rosetta? +### best tip -Ask in [Mina Protocol Discord](https://discord.gg/minaprotocol) or post to the Mina GitHub [Discussions](https://github.com/MinaProtocol/mina/discussions). +The blockchain's latest block with the highest [chain strength](#chain-strength) known to the mina daemon. -## Accounts +### bitwise operations -### Is there an account creation fee? +Generally available in most programming languages, including TypeScript, [bitwise operations](/zkapps/o1js/bitwise-operations) manipulate individual bits within a binary representation of a number. o1js provides versions of bitwise operations that operate on Field elements and result in the necessary circuit constraints to generate a zero knowledge proof of the computation. -Yes, Mina Protocol charges a fee of 1 MINA when you create a new account. This fee helps protect the network from denial of service-type attacks. Over time, this fee can change. +### block -## Transactions +A set of transactions and consensus information that extend the state of the network. A block in Mina includes a proof that the current state of the network is fully valid. See also [extensional blocks](/glossary#extensional-blocks) and [precomputed blocks](/glossary#precomputed-blocks). See [What's in a Block?](/mina-protocol/whats-in-a-block) -### What is the maximum size of the mempool? How do we work around this? +### blockchain -The max mempool size is 3,000. After it hits that size, transactions with the lowest fees are discarded. +The data structure that is used in a cryptocurrency to maintain a shared state of all accounts in the network. -Set your fee to an amount higher than 0.001 MINA, the current average fee for transactions in the pool. +### block confirmations -You can view the fees for pending transactions and adjust your fees accordingly: [https://minascan.io/mainnet/txs/pending-txs](https://minascan.io/mainnet/txs/pending-txs). +The number of blocks added after the reference block. As the number of confirmations increases, the likelihood of a [reorganization](/glossary#reorganization) decreases, thereby increasing the likelihood of all transactions in the reference block being confirmed. -### Why do some users appear to have lost their funds when sending to exchanges? +### block explorer -:::tip +A web-based tool to extract, visualize, and review blockchain network metrics, including transaction histories, wallet balances, and details about individual blocks and transactions. Block explorers for Mina include: -While Mina and its SDKs do support the memo field when sending a transaction, the recommended best practice is do NOT require a memo for deposits. +- https://minascan.io +- https://minataur.net -::: +### block fill rate -To associate the deposit with the user's account, some exchanges require their users to include a unique memo field when sending MINA deposits to the exchange's address. +The proportion of [slots](#slot) that should contain a block. Some slots are intentionally empty to ensure the network can [catch up](/glossary#catchup) in case of delay of messages. -If the user does not include this unique memo when sending their deposit, the receiving exchange may not be able to associate the deposit properly with the user's exchange account. +### block header -These funds are NOT lost. The exchanges have received the funds at the exchange's address, but the exchange may not be able to automatically associate the deposit with the user's exchange account without such a memo. +The portion of a block that contains information about the block itself (block metadata), typically includes a timestamp, a hash representation of the block data, the hash of the previous block's header, and a cryptographic nonce (if needed). -To prevent this issue, we recommend that exchanges do NOT require a memo for deposits. At the same time, exchanges and wallet creators are recommended to expose an optional memo field during a Mina send transaction. +### block producer -### What is the maximum number of rollback blocks? +A node that participates in a process to determine what blocks it is allowed to produce and then produces blocks containing transactions that can be broadcast to the network. People who run [block producer](/mina-protocol/block-producers) nodes are also called block producers. -The table in [Lifecycle of a Payment](/mina-protocol/lifecycle-of-a-payment) describes how many blocks you wait for a transaction to be confirmed. +### bootstrap -### How should I calculate transaction fees? +Part of the [syncing](#syncing) process of a node, bootstrapping gets the current [root](#root-of-transition-frontier) of the [transition frontier](#transition-frontier) from peers. Additional [transitions](#transition) obtained during the [catchup](#catchup) process are applied from this initial root state. -To calculate your transaction fees, use [https://fees.mina.tools](https://fees.mina.tools/). +### breadcrumb -## Running a node +A node in the [transition frontier](#transition-frontier) that contains the external transition, staged ledger, and pending coinbases and is generated by applying the transition to the prior state. -### My Mina node gets stuck sometimes. How can I detect this and fix it? +## C -This is a known issue for some earlier releases. Restart your mina node whenever this issue is detected. +### catch up {#catchup} -You can use the following script to run a cron job every 3 minutes (the slot length) or more frequently: +The final stage of the [syncing](#syncing) process where the node attempts to catch up to the current [best tip](#best-tip) by determining and then downloading all [transitions](#transition) between the transition frontier [root](#root-of-transition-frontier) and the current best tip. First, a node requests the missing transition hashes and a transaction chain proof. This proof proves the path provided is valid, for example, that the provided transition hashes lead from the root to the best tip. After the node has all transition hashes, it requests the full external transition for each transition hash from peers. -``` -MINA_STATUS=$($MINA client status --json) -HIGHESTBLOCK="$(echo $MINA_STATUS | jq .highest_block_length_received)" -HIGHESTUNVALIDATEDBLOCK="$(echo $MINA_STATUS | jq .highest_unvalidated_block_length_received)" +With each external transition, the node builds up its transition frontier by applying each to the prior state to construct a [breadcrumb](#breadcrumb). When the catch up stage is complete, the node's local best tip is the same as the network's best tip, and breadcrumbs have been constructed for all transitions from the transition frontier root (best tip - k) to the current tip, and each has been validated. At this point, the node is [synced](#syncing). -# Calculate difference between validated and unvalidated blocks. -# If block height is more than 4 block behind, something is likely wrong. +A catch up can be triggered at any time if the node sees a disjoint transition in the same path that indicates there are missing transitions. -DELTAVALIDATED="$(($HIGHESTUNVALIDATEDBLOCK-$HIGHESTBLOCK))" -if [[ "$DELTAVALIDATED" -gt 4 ]]; then - $MINA client stop -fi -``` +### chain strength -:::tip +Full history is not available in Mina, so a newly connected node to the network cannot sync from genesis by applying all prior transitions. To allow a node to determine the strongest chain, a minimum chain density is stored for a sliding window of time. As a result, honest nodes can choose the blockchain with the higher minimum density or chain strength. -Be sure your Mina daemon is monitored by something such as systemd, so it can auto-restart. +### cold wallet -::: +A wallet is "cold" if the private key is not, and never has been, available on the internet. Cold storage is preferred for wallets associated with meaningful stake as it is harder to hack into cold wallet systems if they never have been on the internet. This could be as easy as generating a key pair on a laptop with the internet turned off or using a hardware wallet, like a [Ledger](https://shop.ledger.com/) device. -### My archive node is missing block information after a restart. How can I recover the data? +### compressing -Archive node operators often choose to run redundant archive nodes to store block data to one or more locations of their choice (for example, PostgreSQL, GCP, local files, or a logging service) and to backfill any missed block data if needed. +Generating a SNARK for a computation output can be thought of as compressing that output, as the proofs are fixed size. For example, Mina maintains a succinct blockchain by compressing all the historical data in a blockchain into a zk-SNARK. However, this is computationally different from lossy compression. The term _compress_ is used to more figuratively describe the process of reducing the size of required data. -For convenience, [mina_network_block_data](https://console.cloud.google.com/storage/browser/mina_network_block_data) from the archive node is available to help others in the community backfill any missing information. +### consensus -This bucket contains blocks from various Mina networks — for example, Mainnet and the most recent Devnet `devnet2`. Filter by filename for the network you want. Note that this bucket contains blocks for various other networks too, such as QAnet, which is not recommended for your testing. QAnet is used by o1Labs during targeted iterative development. +A process through which all the peers of a blockchain network reach a common agreement about the present state of the distributed ledger. A consensus algorithm or set of rules that Mina nodes all agree upon when deciding to update the state of the network. Rules can include what data a new block can contain and how nodes are selected and rewarded for adding a block. Mina implements the [Ouroboros Samisika](/glossary#ouroboros-samisika) consensus mechanism. -Filenames contain the network name, block height, and state hash of the block. Blocks older than height 25,705 include only the network name and state hash in the filename. +### consensus node -Example filenames: +A participant in the Mina network that performs the consensus function, for example, a [block producer](#block-producer). -(Recent) +### cryptocurrency -``` -mainnet-30627-3NLfKanQ53X2MRKx5ZRvb9nVCEB9eJpcnssGCTpT3J1cojhB5M19.json -``` +A digital asset or currency that uses cryptographic primitives to secure financial transactions and to verify ownership by using public/private key pairs. -(Older) +## D -``` -mainnet-3NKUBmkc7UZ7ik5JyCM4WNzkN1HG5heMB5zNDUkf3Kgta1MFY6LY.json -``` +### daemon -You can download a specific block using curl: +The Mina daemon is a background process that implements the Mina protocol and runs on a node locally so a local client or wallet can talk to the Mina network. For example, when a CLI is used to issue a command to send a transaction, this request is made to the Mina daemon, which then broadcasts it to the peer-to-peer network. The daemon also listens for events like new blocks and relays this to the client by using a publish-subcribe model. -``` -curl https://mina_network_block_data.storage.googleapis.com/ -``` +### DAO -You can import this file using the mina archive blocks tool. The command for it is: +A decentralized autonomous organization (DAO) operates based on rules that are encoded on a blockchain and executed through smart contracts. DAOs are an organizational structure built with blockchain technology. -``` -mina-archive-blocks --precomputed --archive-uri FILE. -``` +### Dapp -### How do I query for the canonical block at a certain height from the archive node +A decentralized application (Dapp) runs on a blockchain or decentralized network and offers benefits such as transparency, security, and censorship resistance. In the Mina ecosystem, Dapps are known as [zkApps](#zkapps). -Use a recursive query. See [Query the database](/node-operators/archive-node/getting-started#query-the-database) examples in the Archive Node docs. +### delegating -### Why am I getting this error message: "Not able to connect to the network"? +Because staking MINA requires nodes to be online, some nodes delegate their MINA to another node that runs a staking service. This process is called delegating a stake. The service provider or staking pool operator can charge a fee that is deducted any time the delegator gets selected to be a block producer. -This error message usually occurs due to a chain ID mismatch from running a Devnet build on Mainnet, or vice versa. +### deploy alias -To check whether you are running a devnet or mainnet build, run `Mina client status`. Next, compare the output's chain ID of your node to the expected chain ID of the network you are trying to connect to. You can find required information for comparison within the [GitHub announcements](https://github.com/MinaProtocol/mina/discussions/categories/announcements) or [Discord](https://discord.com/channels/484437221055922177/601171209287368715) server. +Created with the zkApp CLI, a [deploy alias](/zkapps/tutorials/deploying-to-a-network#deploy-alias) in your project `config.json` file contains the details to manage deployments. -### Are there any official broadcast nodes that can be used? +### Devnet -No, there are no official broadcast nodes at this time. However, you can broadcast transactions using [https://minascan.io/mainnet/broadcast/payment](https://minascan.io/mainnet/broadcast/payment). Use this method as a backup, the recommended method is to broadcast transactions yourself. +Dedicated for developers building on top of the Mina protocol, Devnet is designed for testing and experimentation so you can test tooling and integrations before going live on [Mainnet](#mainnet). See [Connect to Mainnet or Devnet](/node-operators/validator-node/connecting-to-the-network). -## Staking +### distributed ledger technology (DLT) -### Should I be staking my funds? +A digital system for recording the transaction of assets in which the transactions and their details are recorded in multiple places at the same time. In contrast to traditional databases, distributed ledgers have no central data store or administration functionality. -Since Mina is a Proof of Stake (PoS) consensus network without lockup for staked tokens, it is recommended to stak these funds to support the quality of the Mina network. Additionally, by not staking, you are missing out on staking rewards that you can otherwise be receiving from the Mina blockchain. +## E -You can look into staking this wallet, either by running your own block production node or just by delegating your funds to a staking pool on the network. Delegating to a staking pool is simpler to set up. +## ECDSA -:::note +Elliptic Curve Digital Signature Algorithm ([ECDSA](/zkapps/o1js/ecdsa)) a cryptographic algorithm used to sign and verify messages. It is used in many blockchains, including Ethereum, to sign transactions. -Newly staked accounts incur a delay of 18 to 29 days before you start receiving rewards. +## elliptic curves -::: +Equations with a specific template, including: _y^2 = x^3 + ax^ + b_: [secp256k1](/glossary#secp256k1). -### Why is there a delay for staking to take effect? +### elliptic-curve cryptography (ECC) -For purposes of ensuring consensus, there is a delay between when delegations are sent on the blockchain and when they take effect with respect to staking on the network. The staking ledger always operates between 18 to 29 days behind the live ledger. +An approach to public key cryptography based on the algebraic structure of elliptic curves over finite fields. ECC is the basis of how Ethereum and other cryptocurrencies use private keys and digital signatures. -### In that case, how long is the delay and when is the next staking snapshot? +### epoch -The timing of the next staking snapshot varies. +A unit of time equal to 7140 slots at Mainnet. An epoch is divided into [slots](#slot) of 3 minutes each. -Since the timing is based on a combination of consensus timing (epochs) and snarketplace throughput, it is difficult to determine exactly how long this delay can be. +### extensional blocks -A conservative estimate is that delegations sent 3 days before the epoch transition can take effect in the upcoming epoch. This means that, for any given delegation, there is an average of 18 to 29 days delay before this delegation updates block production. +Blocks extracted from the `mina-archive` database contain only the information required to restore data to the archive node and are more lightweight than [precomputed blocks](/glossary#precomputed-blocks). -You can use this Delegation Calculator tool built by Carbonara to see the next staking ledger cutoff: -[https://epoch.mina.tools](https://epoch.mina.tools/). +### external port -## Testing +The port that the Mina daemon uses to connect to other nodes on the network. When starting the daemon, set using `-external-port`. -### What is the best way to test tooling and integration with Mina? +### external transition -Test tooling and integrations on Devnet before going live on Mainnet. The Devnet network is dedicated for developers building on top of the Mina protocol and is designed for testing and experimentation. Be sure to simulate expected Mainnet conditions, such as transaction volume and frequency, to help identify and solve potential issues ahead of time. +Also referred to as a [block](#block), an external transition is generated externally, for example, by another block producer, and gossiped to a node. -See [Connect to Mainnet or Devnet](/node-operators/validator-node/connecting-to-the-network). +## F ---- -url: /exchange-operators ---- +## fee payer account -# Exchange Operators +A developer account that is funded and can always pay fees immediately. When you configure a zkApp, you can choose to use a stored account or create a new fee payer account. -Exchange operators are network participants who run Mina nodes on a Mina network but usually don't produce blocks. +### field element -Instead of running a Mina node and an archive node, use the Rosetta API that is designed specifically for exchange operators. +The basic unit of data in zero knowledge proof programming. Each field element can store a number up to almost 256 bits in size. You can think of a field element as a uint256 in Solidity. For the cryptography inclined, the exact max value that a field can store is 28,948,022,309,329,048,855,892,746,252,171,976,963,363,056,481,941,560,715,954,676,764,349,967,630,336. -It implements an [open-source blockchain-agnostic specification](https://docs.cloud.coinbase.com/rosetta/docs/welcome) designed by Coinbase. +### finality -## Rosetta API +A consensus constant `k` is the point at which chain [reorganizations](#reorganization) are no longer possible. After a block has `k` [block confirmations](#block-confirmations) as defined by the consensus constants, it is considered final. -This section describes how to install, run and use the Rosetta API instance connected to the Mina network. +## foreign field -Use Rosetta API if you are: -- Querying historical data from the Mina blockchain -- Integrating the Mina blockchain with your exchange -- Building blockchain applications, such as explorers and wallets +A finite field different from the native field of the proof system. [Foreign Field Arithmetic](/zkapps/o1js/ecdsa) lets you perform algorithms that connect your zkApp with the outside world of cryptography. -Using Rosetta for building your application is preferable, as it's API is more appropriate for applications and far more stable than GraphQL. +### full node +A Mina node that is able to verify the state of the network trustlessly. In Mina, every node is a full node since all nodes can receive and verify zk-SNARKs. -- [Running with Docker](/exchange-operators/rosetta/run-with-docker) - How to install Rosetta from Docker image -- [Building from source](/exchange-operators/rosetta/build-from-sources) - How to build and run Rosetta from source code -- [Sending requests](/exchange-operators/rosetta/send-requests) - How to send request to a Rosetta API -- [Code Samples](/exchange-operators/rosetta/samples) - Sample code for Rosetta integration integration - - [Requests and Responses](/exchange-operators/rosetta/samples/requests) - Implementing basic requests - - [Offline Signer Tool](/exchange-operators/rosetta/samples/using-signer) - Generating keys and signing transactions offline - - [Scanning Blocks](/exchange-operators/rosetta/samples/scan-blocks) - Example implementation of block scanning - - [Tracking Deposits](/exchange-operators/rosetta/samples/track-deposits) - Example implementation of deposit tracking - - [Sending Transactions](/exchange-operators/rosetta/samples/send-transactions) - Example implementation of withdrawal flow +# G +## gadgets -## FAQ +Small, reusable, low-level building blocks that simplify the process of creating new cryptographic primitives. Most [gadgets](/zkapps/o1js/gadgets) build upon custom gates and act as low-level accelerators in the proof system. -For answers to the most frequently asked questions about listing MINA, see [FAQ Listing MINA](/exchange-operators/faq). +## H -## Questions? +### hash -Post any questions in Mina’s Github Discussions: https://github.com/MinaProtocol/mina/discussions +A mathematical cryptographic function that converts an input of arbitrary length into an encrypted output of a fixed length. Hashing provides security through encryption and is an efficient store of data because the hash is of a fixed size. ---- -url: /exchange-operators/rosetta/build-from-sources ---- +### hot wallet -# Building and running Rosetta from source code +A wallet is "hot" if the private key is available on a machine that is connected to the internet. To mitigate risk in the case of hackers breaking into their systems, careful block producers avoid having hot wallets with substantial stake on them. -In [Running with Docker](run-with-docker), you learned that the Docker container runs the mina daemon, mina-archive, and Rosetta API. +## I -If you already have mina-archive up and running, you can also build and run the Rosetta API natively and wire it to the existing mina daemon and mina-archive. +### internal transition -:::tip +A [transition](#transition) that is produced locally, for example, by a block producer. The generated transition is applied locally and added to the [transition frontier](#transition-frontier) before being broadcast to peers. -To run mina daemon and mina-archive natively, follow the [Archive Node page](/node-operators/archive-node/getting-started) instructions. +## K -::: +## Keccak -The easiest way to build Rosetta natively is to use the Nix development environment. +[Keccak (SHA-3)](/zkapps/o1js/keccak) is a flexible cryptographic hash function that provides more security than traditional SHA hash algorithms. -1. - Clone the official [mina repository](https://github.com/MinaProtocol/mina.git) and switch to the `compatible` branch. +### key pair -1. - If you don't already have Nix on your machine, install it following the steps in the [nix/README.md](https://github.com/MinaProtocol/mina/blob/develop/nix/README.md). +A combination of a [private key](#private-key) and [public key](#public-key). Key pairs can be generated by using a running daemon or using a dedicated keygen tool, see [Generating a Key Pair](/node-operators/validator-node/generating-a-keypair). In Mina, public keys start with `B62` and private keys start with `EK` for easy differentiability. -1. - Run the `./nix/pin.sh` script to enable submodules to be available to the build: +### Kimchi - ```shell - ./nix/pin.sh - ``` +The proof system for Mina, Kimchi is the main machinery that generates the recursive proofs that keep the Mina blockchain small (about 22 KB). Kimchi is a zero knowledge proof system that's a variant of [PLONK](/glossary#plonk) and features a polynomial commitment scheme that supports verifiable computation using traditional Turing machine-based instruction sets. -1. - Launch the development shell: +## L - ```shell - nix develop mina - ``` +### layer 1 (L1) -1. - Build Rosetta app: +The fundamental, base-level chain in a network. An L1 blockchain provides the essential services to a network, like recording transactions on the public ledger and ensuring adequate security. Mina is a layer 1 blockchain. - ```shell - dune build --profile=mainnet src/app/rosetta - ``` +### layer 2 (L2) -1. - After a successful build, the Rosetta app is available here: `_build/default/src/app/rosetta/rosetta.exe` +An off-chain network, system, or technology built on top of a layer 1 blockchain that helps extend the capabilities of the underlying base layer network. - You can run it with following command: - ```shell - MINA_ROSETTA_MAX_DB_POOL_SIZE=64 \ - _build/default/src/app/rosetta/rosetta.exe --port 3087 \ - --graphql-uri - --archive-uri postgres://:@:/ - ``` +### ledger ---- -url: /exchange-operators/rosetta/docker-compose ---- +A cryptocurrency public record-keeping system. Mina has three types of ledgers: [staged ledger](/glossary#staged-ledger), [staking ledger](/glossary#staking-ledger), and [SNARKed ledger](/glossary#snarked-ledger). -# Docker Compose Rosetta +### libp2p -For production deployments, use Docker Compose to run each Rosetta component as a separate container. This gives you control over resource allocation, logging, and restarts. +Mina uses this peer-to-peer networking library to provide things like message broadcast and file sharing. -The full Docker Compose configuration — including `docker-compose.yml`, example environment files for mainnet and devnet, a `Makefile`, and a `README` — is maintained in the Mina repository: +### Lightnet -**[`mina/src/app/rosetta/docker-compose/`](https://github.com/MinaProtocol/mina/tree/compatible/src/app/rosetta/docker-compose)** +A lightweight Mina network in a single Docker container. [Lightnet](zkapps/writing-a-zkapp/introduction-to-zkapps/testing-zkapps-lightnet) is a resource-efficient solution with fast startup and syncing times that lets you test your zkApp locally on an accurate representation of Mina blockchain before you test with a live network. -## Quick start +### lightweight Mina Explorer -:::tip -Before running the commands below, review the [Configuration](#configuration) section to see all available options — including image tags, network selection, ports, and database settings. -::: +Provided in Lightnet, a lightweight Mina Explorer lets you monitor transactions on your local network. See [Testing zkApps with Lightnet](zkapps/writing-a-zkapp/introduction-to-zkapps/testing-zkapps-lightnet#lightweight-mina-explorer). -```bash -git clone https://github.com/MinaProtocol/mina.git -cd mina/src/app/rosetta/docker-compose +## M -# For mainnet -cp example.mainnet.env .env +### major upgrade -# For devnet -cp example.devnet.env .env +Changes to the network that make the old chain incompatible with the new chain. A major upgrade to the Mina network requires all nodes or users to upgrade to the latest version of the protocol software. -# Edit .env — set MINA_LIBP2P_PASS and review POSTGRES_PASSWORD -vi .env +### Mainnet -# Start all services -docker compose up -d +The live version of the Mina blockchain network that is fully operational. On the Mina Mainnet public blockchain, real-world transactions are performed. See [Connect to the Mina Network](node-operators/validator-node/connecting-to-the-network). A Mainnet is different from a [Testnet](#testnet) and [Devnet](#devnet) which are used for development and testing. -# Or use make shortcuts -make mainnet # copies mainnet env and starts services -make devnet # copies devnet env and starts services -``` +### MINA -## Services overview +The unit of the cryptocurrency that is exchanged by participating nodes on the Mina network. MINA is the exclusive currency of the [snarketplace](#snarketplace). -The Docker Compose setup includes six services: +### Mina -| Service | Description | Default Port | -|---------|-------------|--------------| -| **postgres** | PostgreSQL 17 with health checks | 5432 (container), configurable host port | -| **bootstrap_db** | One-shot: downloads and imports the latest daily archive dump | — | -| **mina_archive** | Archive process, stores block data in PostgreSQL | 3086 | -| **mina_node** | Mina daemon with GraphQL API | 3085 (GraphQL), 8302 (P2P) | -| **mina_rosetta** | Rosetta API for exchange integration | 3087 | -| **missing_blocks_guardian** | Monitors and recovers missing blocks between nightly dumps and chain tip | — | +The underlying protocol and the network infrastructure that the system depends on. -## Configuration +### Mina CLI -All configuration is done through a single `.env` file. Key variables: +The primary way for users to interact with the Mina network. The [Mina CLI](/node-operators/reference/mina-cli-reference) command line tool provides standard client functionality to create accounts, send transactions, and participate in consensus and advanced client and daemon commands for power users. -### Docker images +The Mina CLI is installed when you [install Mina](/node-operators/validator-node/installing-on-ubuntu-and-debian). -| Variable | Description | -|----------|-------------| -| `MINA_DAEMON_IMAGE` | Mina daemon Docker image | -| `MINA_ARCHIVE_IMAGE` | Mina archive Docker image | -| `MINA_ROSETTA_IMAGE` | Mina Rosetta Docker image | +### Mina nodes -For mainnet, images are pulled from [Docker Hub](https://hub.docker.com/u/minaprotocol) (`minaprotocol/mina-*`). -For devnet, images are pulled from the o1Labs GCR registry. +Mina nodes fulfill different roles within the network, including [block producers](#block-producer) and [SNARK coordinators](#snark-coordinator). -### Network +## N -| Variable | Description | -|----------|-------------| -| `MINA_NETWORK` | `mainnet` or `devnet` | -| `MINA_PEERLIST_URL` | Bootstrap peers URL | -| `MINA_LIBP2P_PASS` | Passphrase for the libp2p key (required) | +### node -### Ports +A machine running the Mina daemon. -| Variable | Default | Description | -|----------|---------|-------------| -| `POSTGRES_PORT` | `5433` | Host port mapped to PostgreSQL | -| `MINA_REST_PORT` | `3085` | GraphQL API port | -| `MINA_P2P_PORT` | `8302` | P2P networking port | -| `MINA_ARCHIVE_PORT` | `3086` | Archive server port | -| `MINA_ROSETTA_PORT` | `3087` | Rosetta API port | +### node operators -### Archive bootstrap +People who run Mina nodes. Node operators participate in consensus to create new blocks and help compress data by generating zk-SNARKs. -| Variable | Description | -|----------|-------------| -| `ARCHIVE_DUMP_BASE_URL` | Base URL for daily archive dumps | -| `ARCHIVE_DUMP_PREFIX` | Dump filename prefix (`mainnet-archive-dump` or `devnet-archive-dump`) | -| `GUARDIAN_PRECOMPUTED_BLOCKS_URL` | S3 bucket URL for precomputed blocks used by the missing blocks guardian | +### non-consensus node -## Data persistence +A [full node](#full-node) in the Mina protocol that does not participate in consensus but can still fully verify the zero knowledge proof to trustlessly validate the state of the chain. The size of Mina as 22 KB is in reference to non-consensus nodes. -Bind mounts preserve data across `docker compose down` / `up`: +### non-upgradeable -| Host path | Container path | Contents | -|-----------|---------------|----------| -| `./archive/postgresql/data` | `/var/lib/postgresql/data` | PostgreSQL data | -| `./archive/data` | `/data` | Archive node data | -| `./mina_node/.mina-config` | `/root/.mina-config` | Daemon config, keys, peers | -| `./mina_rosetta/.mina-config` | `/root/.mina-config` | Rosetta config | +If the verification key cannot be changed, a zkApp smart contract is considered non-upgradeable. You can make a smart contract upgradeable or not upgradeable using [permissions](/zkapps/writing-a-zkapp/feature-overview/permissions#upgradeability-of-smart-contracts). -## Make targets +### nonce -| Command | Description | -|---------|-------------| -| `make devnet` | Copy devnet env and start services | -| `make mainnet` | Copy mainnet env and start services | -| `make stop` | Stop all services | -| `make clean` | Stop services, remove volumes and all persisted data | -| `make logs` | Follow logs for all services | -| `make status` | Show container status | -| `make health` | Check health of Postgres, GraphQL, and Rosetta endpoints | +An incrementing number attached to a transaction used to prevent a replay of a transaction on the network. Transactions are always included in blocks in the sequential order of the nonce. -## Verifying the deployment +## O -Once services are running and the node is synced: +### o1js -```bash -# Check container status -make status +A TypeScript library for zk-SNARKs and zkApps. Use o1js to write zk smart contracts based on zero knowledge proofs for the Mina Protocol. -# Run health checks -make health +### off-chain -# Check sync status -docker compose exec mina_node mina client status +A transfer of value or data, including transactions, that occurs outside a given blockchain network. These transfers do not need blockchain network confirmation, which speeds up the transaction process and reduces lag time. zkApps use an off-chain execution and mostly off-chain state model that allows for private computation and state that can be either private or public. -# Query Rosetta API -curl -s http://localhost:3087/network/list \ - -H 'Content-Type: application/json' -d '{}' | jq . +### off-chain state -# Connect to archive database -psql postgres://postgres:postgres@localhost:5433/archive -``` +State stored anywhere other than the Mina blockchain. -## Clean start +### on-chain -To wipe all data and start fresh: +A transfer of value or data, including transactions, that exist on and have been verified to a blockchain network. All relevant information is timestamped and stored on the public ledger. On-chain transactions are recorded on the blockchain and need network confirmation before they are completed. -```bash -make clean -docker compose up -d -``` +### on-chain state ---- -url: /exchange-operators/rosetta/run-with-docker ---- +State that lives on the Mina blockchain. Each zkApp account provides eight fields of 32 bytes each of arbitrary storage for the on-chain state. -# Running with Docker +### oracle -:::info +Specialized software or services that act as intermediaries between blockchain smart contracts and external data sources. [Oracles](/zkapps/tutorials/oracle) connect zkApp smart contracts with the outside world to get data on-chain. -There is a known issue that you can't use the docker image on the Apple silicon chip. -To run Mina Rosetta on Apple silicon, you can use the steps in [Building and running Rosetta from source code](build-from-sources). +### Ouroboros Samisika -::: +Mina builds on this provably secure proof of stake (PoS) protocol that combines the best features of each iteration of Ouroboros to deliver a PoS consensus mechanism that can resolve long-range forks without requiring history or risking centralization by relying on trusted third parties to provide fork information. -1. [Install Docker](https://www.docker.com/get-started) and check that your Docker configuration has at least 16 GB RAM (the recommended amount is 32 GB). +## P -1. Check the latest release for Mainnet on the official [Mina GitHub releases](https://github.com/MinaProtocol/mina/releases) page. +## pasta curves -1. Use the Mina Rosetta Docker image: +A collective term for the Pallas and Vesta elliptic curves that are used by the Mina Protocol to generate proofs. See [Pasta Curves](https://o1-labs.github.io/proof-systems/specs/pasta.html?highlight=curves#pasta-curves) in the Mina book. +### peer-to-peer networks -:::note +Networking systems that rely on peer nodes to distribute information amongst each other, are often distributed, and do not rely on any centralized resource broker. -If you want to build your own docker image, you can find more details in [Mina's Rosetta repository](https://github.com/MinaProtocol/mina/blob/develop/src/app/rosetta/README.md). However, for most users, it's not necessary to build your own image in order to interact with the API. +### Pickles -::: +Mina's inductive zk-SNARK composition system. See [Pickles](https://o1-labs.github.io/proof-systems/specs/pickles.html). -The container in `/rosetta` includes three entrypoints, which each run a different set of services connected to a particular network. +### Pickles SNARK -- **docker-start.sh** (default) - connects the mina node to a network (defaults to Mainnet) and initializes the archive database from publicly-available nightly O(1) Labs backups. This script runs a mina node, mina-archive, a postgresql DB, and mina-rosetta. The script also periodically checks for blocks that may be missing between the nightly backup and the tip of the chain and will fill in those gaps by walking back the linked list of blocks in the canonical chain and importing them one at a time. The script is configurable through environment variables. Take a look at the source for more information about what and how you can configure. -- **docker-standalone-start.sh** - starts only the mina-rosetta API endpoint and any flags passed into the script go to mina-rosetta. You may use this for the "offline" part of the Construction API or if you want a setup with each service in its own container. -- **docker-demo-start.sh** launches a mina node with a very simple 1-address genesis ledger as a sandbox for developing and playing around in. This script starts the full suite of tools (a mina node, mina-archive, a postgresql DB, and mina-rosetta), but for a demo network with all operations occurring inside this container and no external network activity. +A proof system and associated toolkit that is the first deployed [SNARK](#snark) capable of recursive composition with no trusted setup. Pickles serves as the basis for developers to build private, scalable smart contracts on Mina. [Meet Pickles SNARK: Enabling Smart Contracts on Mina Protocol](https://medium.com/minaprotocol/meet-pickles-snark-enabling-smart-contract-on-coda-protocol-7ede3b54c250). -Run the container with following command (replace the image tag with one from dockerhub that's compatible with the network you are trying to connect to, also replace `--entrypoint` and any environment variable if needed): +### PLONK -``` - docker run -it --rm --name rosetta \ - --entrypoint=./docker-start.sh \ - -p 10101:10101 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ - minaprotocol/mina-rosetta:3.3.0-8c0c2e6-bullseye-mainnet -``` +Permutations over Lagrange-bases for Oecumenical Noninteractive arguments of Knowledge (PLONK) is a general-purpose zero knowledge proof scheme. -You can also create a file with the environment variables and pass it to the docker run command with `--env-file` flag. For example, create a file named `mainnet.env` with the following content: +### polynomial commitment -``` - MINA_NETWORK=mainnet - PEERS_LIST_URL=https://bootnodes.minaprotocol.com/networks/mainnet.txt - MINA_ARCHIVE_DUMP_URL=https://storage.googleapis.com/mina-archive-dumps - MINA_GENESIS_LEDGER_URL=https://storage.googleapis.com/o1labs-gitops-infrastructure/mainnet/mainnet.json - BLOCKS_BUCKET=https://storage.googleapis.com/mina_network_block_data -``` +A commitment scheme that allows a committer to commit to a polynomial with a short string that can be used by a verifier to confirm claimed evaluations of the committed polynomial. -Then run the container with the following command: +### Poseidon -``` - docker run -it --rm --name rosetta \ - --entrypoint=./docker-start.sh \ - -p 8302:8302 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ - --env-file mainnet.env \ - minaprotocol/mina-rosetta: -``` +A family of [hash](#hash) functions that can efficiently run in a zk circuit. Poseidon operates over the native [Pallas base field](https://electriccoin.co/blog/the-pasta-curves-for-halo-2-and-beyond/) and uses parameters generated specifically for Mina, which makes Poseidon the most efficient hash function available in o1js. See [Poseidon: ZK-friendly Hashing](https://www.poseidon-hash.info/). Poseidon is a sponge construction based on the Hades permutation, with a state composed of field elements and a permutation based on field element operation (addition and exponentiation). -Example environment files for public networks can be found in [this](https://github.com/MinaProtocol/mina/blob/develop/src/app/rosetta/scripts) directory of Mina's Rosetta repository. +### precomputed blocks -:::note +Precomputed blocks are [blocks](/glossary#block) logged by mina to disk, log file, or to cloud storage. -- It can take 20 min to 1 hour for your node to sync -- Port 8302 is the default P2P port and must be exposed to the open internet -- The GraphQL API runs on port 3085 (accessible via localhost:3085/graphql) -- PostgreSQL runs on port 3086 -- Rosetta runs on port 3087 +### preconditions -::: +Conditions that must be true for the account update to be applied. Corresponds to assertions in an o1js method. ---- -url: /exchange-operators/rosetta/samples ---- +### private key -The source code for all code samples is available on the official [Mina Documentation Github page](https://github.com/o1-labs/docs2/tree/main/examples/exchange-operators/) and should be used only for demonstration purposes. +A component of public key cryptography, private keys are held privately, while public keys can be issued publicly. Only the holder of the public key's corresponding private key can attest to ownership of the public key. This allows for signing transactions to prove that you are the honest holder of any funds associated with any given public key. In Mina, private keys start with `EK` for easy differentiability from [public keys](#public-key). -## Set up sample project +### protocol state -Start with the required imports and define constants: +The state of the network that comprises the previous protocol state hash to link blocks together and a body that contains the genesis state hash, blockchain state, consensus state, and consensus constants. - - +### protocol state hash -```ts +The hash of hashes of the previous state and [protocol state](#protocol-state) body. Acts as a unique identifier for a block. +### proof of liabilities (PoL) +A cryptographic primitive to prove the size of funds a bank, or centralized exchange (CEX), owes to its customers in a decentralized manner and can be used for solvency audits with better privacy guarantees. -const TESTNET_NETWORK_IDENTIFIER = { - network_identifier: { - blockchain: "mina", - network: "testnet", - }, -} -const MINA_TOKEN_ID = "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf" -const MINA_DECIMALS = 9 -const MINA_SYMBOL = "MINA" -const MINA_CURVE_TYPE = "pallas" +### proof of stake (PoS) -const mina = new Client({ network: "testnet" }) +The Mina consensus algorithm that allows nodes to agree on the state of the network. PoS allows nodes to [stake](/node-operators/validator-node/staking-and-snarking) MINA on the network to increase their chance of being selected as the next block producer. The winning validators are compensated with a percentage yield of the crypto they have staked as an incentive for engaging in this process. See [Proof-of-Work vs Proof-of-Stake](https://minaprotocol.com/blog/proof-of-work-vs-proof-of-stake). -const request = axios.create({ - baseURL: "https://rosetta-devnet.minaprotocol.network/", -}) -const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)) -``` +### proof of work (PoW) - - +The original consensus process used by Bitcoin, the first cryptocurrency. PoW achieves the decentralized consensus needed to add new blocks to a blockchain by using machines to compete against one another by guessing the answer to math problems that have no feasible faster solution. Computational power is a requirement for the PoW protocol success and assumes that those contributing more resources (energy, supercomputers, and infrastructure) to a problem are less likely to want to destroy the protocol. As an incentive to participate in this consensus process, miners are rewarded with tokens. -```python +### prover function -from subprocess import run, PIPE -from time import sleep +The function that generates a zero knowledge proof from the smart contract logic. -from requests import post +### public key -BASE_URL = "http://localhost:3087" -SIGNER_PATH = 'path/to/signer.exe' -MAINNET_NETWORK_IDENTIFIER = { - "network_identifier": { - "blockchain": "mina", - "network": "mainnet" - } -} -MINA_TOKEN_ID = "1" -MINA_DECIMALS = 9 -MINA_SYMBOL = "MINA" -MINA_CURVE_TYPE = "pallas" -MINA_SIGNATURE_TYPE = "schnorr_poseidon" -``` +A component of public key cryptography, public keys can be widely shared with the world and can be thought of as _addresses_ or identifiers for the person who holds the corresponding private key. In Mina, public keys start with `B62` for easy differentiability from [private keys](#private-key). +### publish-subscribe (pub-sub) - - +A messaging pattern where message senders broadcast messages and notifies any listeners that have previously subscribed to that sender's messages. Mina utilizes pub-sub as a way to notify clients when a new block has been added to the chain. This event can be heard by all listeners, so each listener does not need to independently poll for new data. ---- -url: /exchange-operators/rosetta/samples/requests ---- +## R -# About Mina-specific requests and responses +### recursion -The Rosetta API specification defines high-level description of how requests and response objects should look like. -Exact json object layouts are different from blockchain to blockchain. +The layer 1 architecture of Mina Protocol is based on recursive composition, which means that each block in the blockchain is a tiny snapshot of the entire state of the network. This approach enables Mina to maintain a constant blockchain size of only 22KB regardless of the number of transactions processed. With recursion, you can realize composability between zero knowledge proofs to unlock many powerful technical abilities, such as creating high-throughput applications, creating proofs of large computations, and constructing multi-party proofs. -In this article, you learn about Mina-specific Rosetta JSON objects and how to implement helper functions to construct and parse the JSON objects. +### reorganization -### Network Identifier +When a competing fork of the blockchain increases in length relative to the main branch, the blockchain undergoes a reorganization to reflect the stronger fork as the main branch. After a reorganization, the transactions on the dropped branch are no longer guaranteed inclusion into the blockchain and must be added to new blocks on the longest branch. -There is the `network_identifier` object, that should be passed as a parameter to all endpoints (except `/network/list` which returns possible identifiers). +### root of transition frontier -First, implement the generic `request` function to perform Rosetta API calls injecting that object under the hood: +The root of the [transition frontier](#transition-frontier) is the block `k` blocks from the [best tip](#best-tip). The root is obtained from peers during [bootstrap](#bootstrap). After a new best tip is seen, the root is moved, so only `k` blocks are persisted. The root is the point where the block has been [finalized](#finality) due to consensus. - - +### remote procedure call (RPC) -```ts -async function makeRequest(endpoint: string, data?: any) { - if (data === undefined) { - data = {} - } - if (endpoint !== "/network/list") { - data = { ...TESTNET_NETWORK_IDENTIFIER, ...data } - } +An [RPC](https://en.wikipedia.org/wiki/Remote_procedure_call) is used to communicate between nodes on the network and to interact with the running [daemon](#daemon). - const r = await request.post(endpoint, data) - return r.data -} -``` +## S - +### scan state - - -```python -def _request(path, data=None): - """ - Generalized logic for sending a request to Rosetta API - """ - data = data or {} - nw = {} if path == "/network/list" else MAINNET_NETWORK_IDENTIFIER - path = BASE_URL + path - data = json.dumps({**data, **nw}) - r = post(path, data) - try: - r.raise_for_status() - except Exception as e: - # json.dumps(r.json()) - raise e - return r.json() -``` +A data structure that allows decoupling the production of transaction SNARKs from block producers to [SNARK workers](#snark-worker). See [Scan State](/mina-protocol/scan-state). - - +### secp256k1 -### Block, Transaction, Account Identifiers and the Pubkey object - -According to Rosetta request and response models, each block, transaction and account are identified by an identifier object. -A pubkey is also represented by special object. +The elliptic curve using this equation _y²=x³+7, a=0 b=7_, constructed in a special non-random way to allow for especially efficient computation. secp256k1 can have a key size of up to 256 bits. All points on this curve are valid public keys. -To implement helpers to construct the objects: +### seed nodes - - +A Mina node that keeps a record of nodes in the network and enables nodes that are joining the network to connect to peer nodes. -```ts -function makeBlockIdentifier(idOrHash: number | string) { - const identifierKey = (typeof idOrHash === "number") ? "index" : "hash" - return { - block_identifier: { [identifierKey]: idOrHash }, - } -} +### SHA-2 -function makeTxIdentifier(hash: string) { - return { transaction_identifier: { hash: hash } } -} +Secure Hash Algorithm 2 (SHA-2) is a family of two similar hash functions with different block sizes known as SHA-256 and SHA-512 that differ in word size. SHA-256 uses 32-bit words and SHA-512 uses 64-bit words. -function makeAccountIdentifier(address: string) { - return { - account_identifier: { address, token_id: MINA_TOKEN_ID }, - } -} +### SHA-256 -function makePublicKey(publicKey: string) { - return { - hex_bytes: publicKey, - curve_type: MINA_CURVE_TYPE, - } -} -``` +SHA-256, a part of the SHA-2 family, is a cryptographic hash function that generates a 256-bit (32-byte) hash output and is widely used for traditional Web2 applications and protocols as well as blockchain technology. - +### SHA-3 - +Secure Hash Algorithm 3 (SHA-3) is the latest member of the Secure Hash Algorithm family of standards, released by NIST on August 5, 2015. [Keccak](#keccak) was standardized as SHA-3. -```python -def _make_block_identifier(index_or_hash): - key = "index" if type(index_or_hash) == int \ - else "hash" if type(index_or_hash) == str \ - else None - if not key: - raise ValueError("index_or_hash should have either int or str type") - return {"block_identifier": {key: index_or_hash}} +### signature -def _make_tx_identifier(tx_hash): - return {"transaction_identifier": {'hash': tx_hash}} +Short for digital signature, a way to establish authenticity or ownership of digitally signed messages. -def _make_account_identifier(address): - return { - "account_identifier": {"address": address, "token_id": MINA_TOKEN_ID} - } +### simulated local blockchain -def _make_pubkey(pubkey): - return { - "hex_bytes": pubkey, - "curve_type": MINA_CURVE_TYPE - } -``` +The local testing blockchain you use in the first phase of testing. Using a simulated local blockchain speeds up development and tests the behavior of your smart contract locally. See [Testing zkApps Locally](zkapps/writing-a-zkapp/introduction-to-zkapps/testing-zkapps-locally) and get step-by-steps guidance in [Tutorial 1: Hello World](/zkapps/tutorials/hello-world#simulated-local-blockchain). - - +### slot -### Operation object +A unit of time in the Mina network. As of Mainnet launch, a slot in Mina is 3minutes long. An [epoch](#epoch) is divided into slots. Block producers can find eligible slots to produce blocks in to earn rewards. -In Rosetta terminology, each transaction consist of one or more operations. -In Mina implementation, each operation object has following fields: -- `operation_identifier` and `related_operations`: a mandatory index of an operation, and optional array of related operations. -- `type`: the type of an operation. -- `account`: the account identifier which the operation relates to -- `amount`: an object with `value` - a signed number representing of the accounts' balance +### smart contract -
-A sample JSON object that represents an operation +A tamper-proof program that runs on a blockchain network when certain predefined conditions are satisfied. On Mina, smart contracts (zkApps) are written with [o1js](#o1js). -```json -{ - "operation_identifier": { - "index": 2 - }, - "related_operations": [ - { - "index": 1 - } - ], - "type": "payment_receiver_inc", - "account": { - "address": "B62qqJ1AqK3YQmEEALdJeMw49438Sh6zuQ5cNWUYfCgRsPkduFE2uLU", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "90486110", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } -} -``` +### SNARK -
+An acronym for succinct non-interactive argument of knowledge. See [zk-SNARK](/glossary#zk-snark). -:::note -All possible operation types are available on the `network/options` endpoint. The most common types are `fee_payment`, `payment_source_dec` and `payment_receiver_inc` -::: +### SNARK coordinator +A role on a mina node in the Mina network. SNARK coordinators generate proofs of transactions by distributing work to a series of [SNARK workers](/node-operators/snark-workers). SNARK coordinators then submit that work to the network, and the proofs are sold to block producers. -### Layout of the transfer transaction +### SNARK pool -In Mina, each operation represents an [Account Update](/zkapps/writing-a-zkapp/introduction-to-zkapps/interact-with-mina#account-updates). Therefore, a MINA token transfer transaction is represented by three Account Updates: -- for decreasing fee payer account balance (the fee payer's account is the same as the sender's) -- for decreasing sender account balance -- for increasing receiver account balance +Referred to as the _snarketplace_, the pool that contains work completed by [SNARK workers](#snark-worker) for required work in the [scan state](#scan-state). The SNARK pool contains only the cheapest work offered by SNARK workers for each work bundle. Multiple SNARK workers compete for the same SNARK work, with only the lowest fee for each being included in the SNARK pool to be bought by block producers. -
-A sample object that represents a transfer transaction +### SNARK worker -```json -{ - "transaction_identifier": { - "hash": "CkpYVELyYvzbyAwYcnMQryEeQ7Gd6Ws7mZNXpmF5kEAyvwoTiUfbX" - }, - "operations": [ - { - "operation_identifier": { - "index": 0 - }, - "type": "fee_payment", - "account": { - "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "-37000000", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } - }, - { - "operation_identifier": { - "index": 1 - }, - "type": "payment_source_dec", - "account": { - "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "-58486000", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } - }, - { - "operation_identifier": { - "index": 2 - }, - "related_operations": [ - { - "index": 1 - } - ], - "type": "payment_receiver_inc", - "account": { - "address": "B62qkiF5CTjeiuV1HSx4SpEytjiCptApsvmjiHHqkb1xpAgVuZTtR14", - "metadata": { - "token_id": "1" - } - }, - "amount": { - "value": "58486000", - "currency": { - "symbol": "MINA", - "decimals": 9 - } - } - } - ] -} -``` +External processes that connect to a Mina node on the network and create zk-SNARK proofs of transactions to compress the transactions so they can be folded into the tiny blockchain proof. The SNARK worker is incentivized with MINA as compensation to help compress transactions. See [What are SNARK Workers and the Snarketplace?](https://minaprotocol.com/blog/what-are-snark-workers-and-the-snarketplace). -
+### SNARKed ledger -According to the Rosetta specification, the array of `operation` objects must be passed as a parameter in the `/construction/preprocess` and `/construction/payloads` endpoints. +The ledger that contains only the transactions that have an associated proof. The SNARKed ledger is updated after a proof has been emitted from the [scan state](#scan-state). -To implement a helper function to construct the operations array for a MINA token transfer: +### snarketplace - - +Similar to a marketplace where people, or nodes, exchange services for a fee. It revolves around a fixed-size buffer like a queue or shelf of work to do. Block producers add work to this shelf in the form of transactions that need to be SNARKed, and then SNARK workers take the work off the shelf and create SNARKs out of them to process the transactions. Block producers purchase SNARK work for the lowest price from the snarketplace. Conversely, the SNARK workers want to maximize their profit while also being able to sell their SNARK work. These two roles act as the two sides of the marketplace and, over time, establish an equilibrium at a market price for SNARK work. -```ts -function makeTransferPayload(from: string, to: string, feeNano: number, valueNano: number) { - function makeOperation( - idx: number, - relatedIdxs: number[], - opType: string, - addr: string, - value: number, - isPositive: boolean, - ) { - const relatedOps = (relatedIdxs.length == 0) ? {} : { - related_operations: relatedIdxs.map((i) => { - return { index: i } - }), - } +### soft fork - return { - operation_identifier: { index: idx }, - ...relatedOps, - type: opType, - account: { - address: addr, - metadata: { - token_id: MINA_TOKEN_ID, - }, - }, - amount: { - value: (isPositive ? "" : "-") + value.toString(), - currency: { - symbol: MINA_SYMBOL, - decimals: MINA_DECIMALS, - }, - }, - } - } +An upgrade to the network that is backward-compatible. - return { - operations: [ - makeOperation( - 0, - [], - "fee_payment", - from, - feeNano, - false, - ), - makeOperation( - 1, - [], - "payment_source_dec", - from, - valueNano, - false, - ), - makeOperation( - 2, - [1], - "payment_receiver_inc", - to, - valueNano, - true, - ), - ], - } -} -``` +### staged ledger - +The current account state that contains a pending accounts ledger and also a pending queue of un-SNARKed transactions known as the [scan state](#scan-state). - - -```python -def _make_transfer_payload(address_from, address_to, fee_nano, value_nano): - """ - Helper for generating "operations" object in a Rosetta format - This helper generates mina native token transfer payloads - The resulting object is an array of 3 so called account updates: - - for decreasing the sender's balance by the fee amount - - for decreasing the sender's balance by the transfer amount - - for increasing the receiver's balance by the transfer amount - """ - def _make_operation(idx, related_idxs, op_type, addr, value, is_positive): - related_ops = {} if not related_idxs else { - "related_operations": [{"index": i} for i in related_idxs] - } - return { - "operation_identifier": {"index": idx}, - **related_ops, - "type": op_type, - "account": { - "address": addr, - "metadata": { - "token_id": MINA_TOKEN_ID - } - }, - "amount": { - "value": ("" if is_positive else "-") + str(value), - "currency": { - "symbol": MINA_SYMBOL, - "decimals": MINA_DECIMALS - } - } - } - return { - "operations": [ - _make_operation( - 0, [], "fee_payment", address_from, fee_nano, False), - _make_operation( - 1, [], "payment_source_dec", address_from, value_nano, False), - _make_operation( - 2, [1], "payment_receiver_inc", address_to, value_nano, True) - ] - } -``` +### staking - - +Staking MINA allows nodes on the network to increase their chances of being selected as a block producer in accordance with the consensus mechanism. -### Implementing client methods +### staking ledger -Now that you have all helpers in place, you can implement all of the Rosetta client methods according to the specification: +The ledger used to determine block producers for a slot, as the probability of finding eligible slots to produce blocks in is proportional to the amount of stake. - - +### staking pool -```ts -async function networkList() { - return await makeRequest("/network/list") -} +A pool of delegated funds that is run by a staking pool owner. To avoid the requirement of being online, other nodes can choose to delegate funds to a staking pool. -async function networkStatus() { - return await makeRequest("/network/status") -} +### state -async function networkOptions() { - return await makeRequest("/network/options") -} +The current status or snapshot of all data stored within the blockchain. -async function block(idOrHash: number | string) { - return await makeRequest("/block", makeBlockIdentifier(idOrHash)) -} +### syncing -async function mempool() { - return await makeRequest("/mempool") -} +To successfully produce a block that extends the Mina blockchain, a node requires the current state. To achieve this, a node initializes to connect to peers, [bootstrap](#bootstrap), and then performs a [catchup](#catchup). -async function mempoolTx(hash: string) { - return await makeRequest("/mempool/transaction", makeTxIdentifier(hash)) -} +The syncing process builds the node's [transition frontier](#transition-frontier) by creating [breadcrumbs](#breadcrumb) for all transitions between the transition frontier's [root](#root-of-transition-frontier) to the current [best tip](#best-tip). When complete, the node is synced. -async function accountBalance(address: string) { - return await makeRequest("/account/balance", makeAccountIdentifier(address)) -} +## T -async function accountTransactions(address: string) { - return await makeRequest("/search/transactions", { address: address }) -} +### Testnet -async function getTransaction(hash: string) { - return await makeRequest("/search/transactions", makeTxIdentifier(hash)) -} +An instance of a blockchain used for testing and experimenting where MINA tokens have no real value. For example, for [zkApp Developer Tutorials](/zkapps/tutorials) and other development, you can request tMINA funds from the Testnet Faucet to fund your [fee payer account](#fee-payer-account). Mina's public Testnet is feature-complete and is called the `Devnet`. -async function deriveAccountIdentifier(publicKey: string) { - return await makeRequest("/construction/derive", { - public_key: makePublicKey(publicKey), - }) -} +### time-locked accounts -async function txPreprocess(from: string, to: string, feeNano: number, valueNano: number) { - const payload = makeTransferPayload(from, to, feeNano, valueNano) - return await makeRequest("/construction/preprocess", payload) -} +An account with a non-vested amount of tokens that cannot be moved until a specific condition has been met, like a number of blocks that has been produced. See [Time-Locked Accounts](/zkapps/writing-a-zkapp/feature-overview/time-locked-accounts). -async function txMetadata( - srcPublicKey: string, - srcAddress: string, - destAddress: string, - feeNano: number, - valueNano: number, -) { - const options = await txPreprocess(srcAddress, destAddress, feeNano, valueNano) - return await makeRequest("/construction/metadata", { - ...options, - public_keys: [makePublicKey(srcPublicKey)], - }) -} +### tMINA -async function txPayloads( - srcPublicKey: string, - srcAddress: string, - destAddress: string, - feeNano: number, - valueNano: number, -) { - // If fee_nano is undefined, it will get suggested fee from /construction/metadata response - const meta = await txMetadata( - srcPublicKey, - srcAddress, - destAddress, - feeNano, - valueNano, - ) +MINA tokens that have no real-world value. You can request tMINA funds from the Testnet Faucet to fund your [fee payer account](#fee-payer-account) during development. - if (feeNano === 0) { - feeNano = meta.suggested_fee[0].value - } - console.log(feeNano) - const operations = makeTransferPayload(srcAddress, destAddress, feeNano, valueNano) - return makeRequest("/construction/payloads", { ...operations, ...meta }) -} +### tokens -async function txCombine(payloadsResponse: any, privateKey: string) { - // console.dir(payloadsResponse, {depth: null}) - const combinePayload = mina.rosettaCombinePayload(payloadsResponse, privateKey) - const r = await makeRequest("/construction/combine", combinePayload) - return r -} +A digital asset that typically represents an asset, utility, or value in a particular blockchain ecosystem. The native cryptocurrency of the Mina blockchain is MINA. Tokens can serve a variety of purposes, including accessing platform features (utility tokens), representing ownership of assets (security tokens), or facilitating smart contracts and decentralized applications (Dapps). -async function txParse(isSigned: boolean, blob: string) { - return makeRequest("/construction/parse", { - signed: isSigned, - transaction: blob, - }) -} +### transaction pool -async function txHash(blob: string) { - return await makeRequest("construction/hash", { signed_transaction: blob }) -} +An in-memory store of all the transactions that peer has heard on the network. Sometimes referred to as the mempool, each node has a local list of all pending transactions that have been gossiped to the node and validated. -async function txSubmit(blob: string) { - return await makeRequest("construction/submit", { signed_transaction: blob }) -} -``` +### transition - +A transition in Mina is synonymous with a [block](#block). - - -```python -def network_list(): - return _request("/network/list") - -def network_status(): - return _request("/network/status") - -def network_options(): - return _request("/network/options") - -def block(index_or_hash): - return _request("/block", _make_block_identifier(index_or_hash)) - -def mempool(): - return _request("/mempool") - -def mempool_tx(tx_hash): - return _request("/mempool/transaction", _make_tx_identifier(tx_hash)) - -def account_balance(address): - return _request("/account/balance", _make_account_identifier(address)) - -def derive_account_identifier(pubkey): - return _request("/construction/derive", { - "public_key": _make_pubkey(pubkey) - }) - -def tx_preprocess(src, dest, fee_nano, value_nano): - return _request( - "/construction/preprocess", - _make_transfer_payload(src, dest, fee_nano, value_nano)) - -def tx_metadata(src_pubkey, src_address, dest_address, fee_nano, value_nano): - options = tx_preprocess(src_address, dest_address, fee_nano, value_nano) - return _request("/construction/metadata", { - **options, "public_keys": [_make_pubkey(src_pubkey)] - }) - -def tx_payloads(src_pubkey, src_address, dest_address, fee_nano, value_nano): - """ - If fee_nano is None, it will get suggested fee from /construction/metadata response - """ - fee_nano = fee_nano or 0 - meta = tx_metadata( - src_pubkey, src_address, dest_address, fee_nano, value_nano) - - if not fee_nano: - fee_nano = meta["suggested_fee"][0]["value"] - - operations = _make_transfer_payload( - src_address, dest_address, fee_nano, value_nano) - return _request("/construction/payloads", {**operations, **meta}) - -def tx_combine(src_pubkey, tx_payloads_response, signature): - combine_payload = { - "unsigned_transaction": tx_payloads_response["unsigned_transaction"], - "signatures": [ - { - "signing_payload": tx_payloads_response["payloads"][0], - "public_key": _make_pubkey(src_pubkey), - "signature_type": MINA_SIGNATURE_TYPE, - "hex_bytes": signature - } - ] - } - return _request("/construction/combine", combine_payload) +### transition frontier -def tx_parse(transaction_blob, is_signed): - return _request("/construction/parse", { - "signed": is_signed, - "transaction": transaction_blob - }) +A local data store that contains the last `k` blocks of the network. With a rose tree-type data structure, each node of the tree is a [breadcrumb](#breadcrumb) and can have multiple children (forks). -def tx_hash(signed_transaction_blob): - return _request("/construction/hash", { - "signed_transaction": signed_transaction_blob - }) +### TypeScript -def tx_submit(signed_transaction_blob): - return _request("/construction/submit", { - "signed_transaction": signed_transaction_blob - }) -``` +A superset of JavaScript that adds compile-time type safety. [o1js](https://www.npmjs.com/package/o1js) is a TypeScript library for zk-SNARKs and zkApps. See the official [TypeScript docs](https://www.typescriptlang.org). +## U - - +### user transaction ---- -url: /exchange-operators/rosetta/samples/scan-blocks ---- +A transaction that is issued by a user, like a payment or a delegation change. -To implement a simple function to wait for a given block: +## V - - +### verification key -```ts -async function waitForBlock(blockHeight: number) { - let latestBlock = (await networkStatus()).current_block_identifier.index - while (true) { - if (blockHeight <= latestBlock) { - return await block(blockHeight) - } - await sleep(10000) - latestBlock = (await networkStatus()).current_block_identifier.index - } -} -``` +A piece of data that is generated by the smart contract build process. When a smart contract is deployed, a transaction that contains the verification key is sent to an address on the Mina blockchain. Sending a verification key to a zkApp account allows Mina to verify zero knowledge proofs that were generated by a smart contract's prover function. - +### verifier function - +The function that verifies a zero knowledge proof using the [verification key](#verification-key). -```python -def wait_for_block(block_index): - """ - Checks if block with given index exist - Once the /block response is succesful - returns the response - Otherwise, retries fetching it with 10 seconds delay - """ +### verifiable random function (VRF) - latest_block = network_status()["current_block_identifier"]["index"] - while True: - if block_index <= latest_block: - return block(block_index) - sleep(10) - latest_block = network_status()["current_block_identifier"]["index"] -``` +A function that generates an output that can be cryptographically verified as random. Mina uses VRF to select a block producer for a slot, taking as input a random seed that is derived from the previous epoch's VRF outputs, a public key, and the current staking ledger. VRF is deterministic, so the same output is returned regardless of how often it is run. - - +## Z -It can be used to scan blocks like this: +### zero knowledge proof - - +A proof by which one party (a prover) can prove to another party (a verifier) that they have knowledge of something, without giving away that specific knowledge. Mina uses zero knowledge proofs, specifically zk-SNARKs, to generate a proof attesting to the blockchain's validity and allows any node on the network to verify the validity. -```ts -let latestBlockHeight = (await network_status()).current_block_identifier.index -while (true) { - const lastBlock = waitForBlock(latestBlockHeight) - // some processing according to business logic - - latestBlockHeight += 1 -} -``` +### zkApps - +Zero knowledge apps ([zkApps](/zkapps/writing-a-zkapp)) are Mina Protocol's smart contracts powered by zero-knowledge proofs, specifically using zk-SNARKs. zkApps provide powerful and unique characteristics such as unlimited off-chain execution, privacy for private data inputs that are never seen by the blockchain, the ability to write smart contracts in TypeScript, and more. The easiest way to write zk programs is using [o1js](https://www.npmjs.com/package/o1js). - +### zkApp CLI -```python -latest_block = network_status()["current_block_identifier"]["index"] -while True: - last_block = wait_for_block(latest_block) - - # some processing according to business logic - - latest_block += 1 -``` +A command line tool that zkApp developers use to scaffold and deploy smart contracts. Install the [zkApp CLI](https://www.npmjs.com/package/zkapp-cli). - - +### zkApp account ---- -url: /exchange-operators/rosetta/samples/send-transactions ---- +A zkApp account is an account on the Mina blockchain where a zkApp smart contract is deployed. Each zkApp account provides 8 fields of 32 bytes each of arbitrary storage. When a Mina address contains a verification key, it acts as a zkApp account. -:::info -This flow is also described in the [Construction API Overview](https://docs.cloud.coinbase.com/rosetta/docs/construction-api-overview) article in the official Rosetta documentation. -::: +### zkApp manager account -The steps needed to send payment in MINA token are following: -1. Given a key pair, derive the account indentifier using `/construction/derive` endpoint -1. Call `/construction/preprocess` and `/construction/metadata` to construct parameters for `/construction/payloads` request -1. Create an unsigned transaction blob using `construction/payloads` endpoint -1. Call `construction/parse` (optional) to check if the unsigned transaction does what you expect -1. Use detached signer to sign the transaction -1. Call `construction/combine` to generate signed blob to be sent via `/construction/submit` endpoint -1. Call `construction/parse` again (optional) to confirm correctness of the signed transaction -1. Get a future transaction hash using `/construction/hash` enpoint -1. Submit the signed transaction blob via `/construction/submit` endpoint - -For ease of readability, this sample implementation skips the sanity checks (steps 4 and 7) and combines steps 2 and 3 in a single `tx_payloads` function call. - -Before running this sample: -- Make sure you followed [Offline signer tool](using-signer) article and generated a key pair -- Send some test funds on the account you've generated. - -The implementation is as follows: - - - - -```ts -async function send(privateKey: string, to: string, valueNano: number, feeNano: number) { - const publicKey = mina.derivePublicKey(privateKey) - const publicKeyRaw = mina.publicKeyToRaw(publicKey) - // get transfer payload to sign - const payloadsResponse = await txPayloads( - publicKeyRaw, - publicKey, - to, - feeNano, - valueNano, - ) +A specific type of smart contract that manages a particular thing. For example, the zkApp manager account for a token controls all properties of token accounts and determines rules for token minting, burning, and transfer. - // sign and combine transfer payload - const combineResponse = await txCombine(payloadsResponse, privateKey) - const blob = combineResponse.signed_transaction - - // // get future transaction hash - const txHashResponse = await txHash(blob) - const hash = txHashResponse.transaction_identifier.hash - - // submit transaction. this call will fail if tx is not in mempool - await txSubmit(blob) - // wait for transaction confirmation: - // for that, track blocks until we meet our transaction in the last one - let latestBlock = (await networkStatus()).current_block_identifier.index - while (true) { - // check if our transaction exists in the latest block - const txs = (await waitForBlock(latestBlock)).block.transactions - const hashes = txs.map((tx: any) => tx.transaction_identifier.hash) - if (hashes.includes(hash)) { - break - } +### zkBridge - latestBlock += 1 - } - return hash -} -``` +Technology that the Mina network uses to connect to other chains. - +### zk-SNARK - - -```python -def withdrawal_flow(vault_keypair, dest_address, value_nano, fee_nano=None): - """ - Full withdrawal flow example (without sanity check) - If fee_nano is None - fee will be suggested by Rosetta API response - """ - - vault_pubkey = vault_keypair["public"] - vault_private_key = vault_keypair["private"] - - # derive vault address - derive_response = derive_account_identifier(vault_pubkey) - vault_address = derive_response["account_identifier"]["address"] - - # get transfer payload to sign - payloads_response = tx_payloads( - vault_pubkey, vault_address, dest_address, fee_nano, value_nano) - - # sign transfer payload - signature = sign_transaction( - SIGNER_PATH, - vault_private_key, - payloads_response["unsigned_transaction"]) - - # get signed transaction blob - combine_response = tx_combine(vault_pubkey, payloads_response, signature) - signed_transaction_blob = combine_response['signed_transaction'] - - # sanity check may be performed here - # for that - use tx_parse function output - # to doublecheck it corresponds what you want to do - # parsed_tx = tx_parse(signed_transaction_blob, True) - - # get future transaction hash - tx_hash_response = tx_hash(signed_transaction_blob) - future_tx_hash = tx_hash_response['transaction_identifier']['hash'] - - # submit transaction. this call will fail if tx is not in mempool - tx_submit(signed_transaction_blob) - - # wait for transaction confirmation: - # for that, track blocks until we meet our transaction in the last one - latest_block = network_status()["current_block_identifier"]["index"] - while True: - # check if our transaction exists in the latest block - txs = wait_for_block(latest_block)['block']['transactions'] - hashes = [tx['transaction_identifier']['hash'] for tx in txs] - if future_tx_hash in hashes: - break - latest_block += 1 - - return future_tx_hash -``` +A zero knowledge proof. zk-SNARK is the acronym for zero knowledge succinct non-interactive argument of knowledge. Specific properties of interest in Mina's implementation of SNARKs are succinctness and non-interactivity, which allow for any node to quickly verify the state of the network. SNARK workers create [zk-SNARKs](https://minaprotocol.com/blog/what-are-zk-snarks) for each transaction. zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of a block, and in turn, these zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of the network. - - +[A](#a) [B](#b) [C](#c) [D](#d) [E](#e) [F](#f) G [H](#h) [I](#i) J [K](#k) [L](#l) [M](#m) [N](#n) [O](#o) [P](#p) Q [R](#r) [S](#s) [T](#t) [U](#u) [V](#v) W X Y [Z](#z) --- -url: /exchange-operators/rosetta/samples/track-deposits +url: /mina-protocol/block-producers --- -After everything is ready, you can implement deposit tracking. +# Block Producers -For that, iterate through all transactions in each block to search for `payment_receiver_inc` operation with an address matching the address in which you are interested: +The role of a block producer in Mina is to achieve [consensus](https://minaprotocol.com/blog/what-is-ouroboros-samasika) and provide security to the blockchain. The block producer is responsible for creating new blocks that include recent transactions broadcast on the network and a blockchain proof that proves the current state of the chain is valid. - - +In Mina, anyone can become a block producer. There is an unbounded number of participants with the chance of producing a block proportional to the funds staked. Funds are not locked and are not subject to slashing. -```ts -async function trackDeposit(address: string) { - // some logic we want to execute on deposit - function onDeposit(deposit: any) { - console.log(deposit) - } - let latestBlock = (await networkStatus()).current_block_identifier.index - while (true) { - // check if transactions to the deposit address are present in the latest block - const txs = (await waitForBlock(latestBlock)).block.transactions - const deposits = [] - for (let tx of txs) { - for (let op of tx.operations) { - if (op.account.address === address && op.type === "payment_receiver_inc") { - deposits.push({ - tx_hash: tx.transaction_identifier.hash, - amount: op.amount.value, - }) - } - } - } +In return for staking funds and generating the required blockchain proofs, blocks that are produced and included in the canonical chain are rewarded in the form of a coinbase and transaction fees, less any fees paid to purchase required [transaction SNARK work](./snark-workers). - // process deposits - for (let d of deposits) { - onDeposit(d) - } +To successfully produce a block, a block producer must have the current state of the blockchain. A block producer must have enough available compute to produce a blockchain SNARK within the slot time and be connected to peers to broadcast the generated block within an acceptable delay as defined by the network consensus parameters. - latestBlock += 1 - } -} -``` +### Select a block producer - +The opportunity to produce a block for a slot is determined by a [verifiable random function](/glossary#verifiable-random-function-vrf) (VRF). Think of this function as a lottery. Each block producer independently runs this VRF for each slot and if the output is greater than a threshold proportional to the producer's stake, they have the chance to produce a block at the designated slot. - - -```python -def deposit_flow(deposit_address): - # some logic we want to execute on deposit - def on_deposit(deposit): - print(json.dumps(deposit, indent=2)) - - latest_block = network_status()["current_block_identifier"]["index"] - while True: - # check if transactions to the deposit address are present in the latest block - txs = wait_for_block(latest_block)['block']['transactions'] - deposits = [] - for tx in txs: - for op in tx["operations"]: - if op["account"]["address"] == deposit_address \ - and op["type"] == "payment_receiver_inc": - deposits.append({ - "tx_hash": tx["transaction_identifier"]["hash"], - "amount": op["amount"]["value"] - }) - - # process deposits - for d in deposits: - on_deposit(d) - - latest_block += 1 -``` +This process is secret so that only the private key holder can determine the VRF output and only they know when they are to produce a block. This selection process aids security as it is impossible for an adversary to target a known block producer at a certain slot, e.g., by a denial of service or targeted attack. As a result, multiple producers can be selected for the same slot. When multiple producers produce a valid block for the same slot, a short-range fork is produced where the consensus rules select the longest chain. - - +The stake distribution is determined from the SNARKed ledger at the last block of `current epoch-2`, so there is a delay for any recently acquired or [delegated stake](#stake-delegation). For example, if the current epoch is 10, the staking distribution is determined from the SNARKed ledger of the last block of the 8th epoch. ---- -url: /exchange-operators/rosetta/samples/using-signer ---- +To view the output of the VRF in the logs, look for `Checking VRF evaluations`. -# Offline signer tool +### Generating a block -According to the Rosetta specification, you use an offline tool to sign transactions and use Rosetta to submit the signed transactions. -The Mina Rosetta implementation comes with an offline signer CLI tool and a [`mina-signer`](/mina-signer) npm package. To generate a key pair, sign a transaction and verify a signature with it. +When a block producer is selected to produce a block for a slot, they perform the following actions: -In a native build, the path to the signer tool is `_build/default/src/app/rosetta/ocaml-signer/signer.exe`. +- Choose the current best tip from their transition frontier (local store of blocks) on which to build the new block. -In a Docker container, the path is `/rosetta/app/mina-ocaml-signer`. +- Select transactions and any SNARK work required from the transaction and SNARK pools. -To simplify examples, it is referred to as the `signer` alias. + A block producer must purchase SNARK work at least in equal quantity to the transactions they add to a block. + In addition to any user transactions, a block producer must also add a coinbase transaction as a reward for producing the block and any fee transfers to pay the SNARK workers. -## Generating a key pair with `mina-signer` +- Generate the proposed next state of the blockchain. -See [Generating Keys](/mina-signer#generating-keys) in the Mina Signer documentation for full details. A quick example: + - Create a diff of the staged ledger that includes the account ledger and scan state (a queue of transactions yet to have proofs). + - Apply this diff to the existing staged ledger to produce the new state. -```ts +- Create a blockchain proof to prove that the new state is valid. -const mina = new Client({network: 'testnet'}) -const keypair = mina.genKeys() -``` + This SNARK additionally validates the prior protocol state proof. -## Generating a key pair with signer CLI +- Create a delta transition chain proof that proves the validity of the block if it is received within an acceptable network delay as defined by the network consensus parameters. -To generate a private key, run: +- Apply this newly generated state locally and add it into the existing transition frontier. - signer generate-private-key +- Broadcast the block (call an external transition) to its peers. -To derive public key, run: +### Stake delegation - signer derive-public-key --private-key `` +Delegated funds are not spendable and can be undelegated at any time by re-delegating the stake back to the original account. -In the output you will also get a Mina account address associated with this key pair. +--- +url: /mina-protocol +--- -## Signing a transaction with `mina-signer` +# Introduction -A sample code to sign a transaction: +The Mina Protocol is a layer one protocol designed to deliver on the original promise of blockchain, true decentralization, scale and security. -```ts -mina.signRosettaTransaction(payload, privateKey) -``` +Mina offers an elegant solution: replacing the blockchain with an easily verifiable, consistent-sized cryptographic proof. Mina dramatically reduces the amount of data each user needs to download. Instead of verifying the entire chain from the beginning of time, participants fully verify the network and transactions using recursive zero knowledge proofs (or zk-SNARKs). Nodes can then store the small proof, as opposed to the entire chain. Because it’s a consistent size, Mina stays accessible even as it scales to many users and accumulates years of transaction data. -:::note +## The Mina Protocol -We advice to use `mina.rosettaCombinePayload` helper to sign and prepare a payload for `/construction/combine` request +There are three public Mina Protocol networks: -See [Sending Transactions](send-transactions) page to learn more. +1. `mainnet` - the production network +2. `devnet` - the test network based on the same software versions as the Mainnet +3. `berkeley` - a development network where new features are trialed -::: +You check the identity of the network with this graphQL query: -## Signing a transaction with CLI +``` +query MyQuery { + networkID +} +``` -To sign a transaction, run: +This section describes how the Mina Protocol works. - signer sign --private-key `` --unsigned-transaction `` +- [Proof Of Stake](/mina-protocol/proof-of-stake) +- [What's in a Block](/mina-protocol/whats-in-a-block) +- [Block Producers](/mina-protocol/block-producers) +- [SNARK Workers](/mina-protocol/snark-workers) +- [Scan State](/mina-protocol/scan-state) +- [Time-Locked Accounts](/mina-protocol/time-locked-accounts) +- [Sending a Payment](/mina-protocol/sending-a-payment) +- [Lifecycle of a Payment](/mina-protocol/lifecycle-of-a-payment) -:::note +## Node Operators -Replace `` with the unsigned transaction string, which you can construct using `construction/payloads` endpoint. -See [Sending Transactions](send-transactions) page to learn more. +[Node Operators](../node-operators) describe how to run Mina nodes on a Mina network. Mina nodes fulfill different roles within the network. -::: +## Node Developers +[Node Developers](../node-developers) describes how developers can add to and improve Mina nodes. -To implement a helper function to sign a transaction (in Python): +## Exchange Integration -```python -def sign_transaction(path_to_signer, private_key, unsigned_transaction): - cmd = "{signer} sign --private-key {pk} " \ - "--unsigned-transaction {tx_blob}".format( - signer=path_to_signer, - pk=private_key, - tx_blob=unsigned_transaction) - r = run(cmd.split(), stdout=PIPE).stdout - return r.decode().strip() -``` +[Exchange Operators](/node-operators/exchange-operators) describes how exchanges can integrate with the Mina blockchain using Rosetta API, archive nodes, and validator nodes. --- -url: /exchange-operators/rosetta/send-requests +url: /mina-protocol/lifecycle-of-a-payment --- -# Sending requests - -After the Rosetta app is running, you can send requests to it. -To run a `/network/list` request: - - curl -L -X POST 'http://localhost:3087/network/list' \ - -H 'Content-Type: application/json' \ - -H 'Accept: application/json' -d '{}' +# Lifecycle of a Payment -In the case of successful setups, you see a response similar to this one: +In Mina, payments pass through several steps before they are considered verified and complete. +This document walks through what happens to a single payment in a simplified overview to help you understand how Mina payments work. It it not a comprehensive technical overview, but instead a simplified walkthrough for users. -```json -{"network_identifiers":[{"blockchain":"mina","network":"mainnet"}]} -``` +## Gossip Protocol -You must pass the `network_identifier` object as a parameter to all other endpoints. In Mina Rosetta implementation, it exists only for the network you run Rosetta for, so this array always contains one object. +Mina uses a gossip protocol to ensure that messages can be reliably transmitted to all other members of the network in a timely manner. -Learn how to use Rosetta API for different use cases. See in the [Code Samples](samples) section. +## Payments ---- -url: /glossary ---- +A payment is a type of transaction, requesting to transfer value from one account to another account, and the associated fee the sender is willing to pay for the payment to go through. -# Glossary +This scenario walks through a scenario where a sender, Bob, wants to send some MINA to a receiver, Alice. -[A](#a) [B](#b) [C](#c) [D](#d) [E](#e) [F](#f) G [H](#h) [I](#i) J [K](#k) [L](#l) [M](#m) [N](#n) [O](#o) -[P](#p) Q [R](#r) [S](#s) [T](#t) [U](#u) [V](#v) W X Y [Z](#z) +### Step 1: To create a payment, Bob clicks send -Are you looking for a zero knowledge, o1js, zkApp, or Mina term that isn't here yet? To let us know, please [create an issue](https://github.com/o1-labs/docs2/issues) or click **EDIT THIS PAGE** to submit a PR. +Any member of the network can create a payment and share it with the Mina network. The payment is cryptographically signed with a private key so that the sender's account can be verified. The payment is then sent out to peers on the network to be processed. The payment, when received by a peer, exists in their local `transaction pool`, which is an in-memory store of all the transactions that peer has heard on the network. -## A +### Step 2: To produce a block, Bob's payment gets put in a todo list -### account +A block producer node is chosen on the network for a given time slot. The currently active producer chooses in-flight payments based on payment fees and places them in a list to be processed called a transition block. Block producers earn mina for building these blocks. The producer generates a SNARK defining the structure of the transition block as compared to the previous block (but not yet verifying these new payments). The producer transmits this new information for SNARK workers to process. -Mina uses accounts to track each public key's state. This is distinct from Bitcoin's UTXO model of maintaining ledger state. +### Step 3: To prove a SNARK transaction, Bob's payment gets SNARK-signed -### archive node +SNARK worker nodes on the network begin performing SNARK calculations on each step of the new transition block. These are individual proofs of each payment and then merge proofs of neighboring payments. Eventually, all the payments are verified. SNARK workers can earn currency by generating these proofs, paid for by block producers from their block rewards. These proofs are transmitted out over the network. -A Mina node that stores the historical chain data to a persistent data source so it can later be retrieved. A zkApp can retrieve events and actions from one or more Mina [archive nodes](/node-operators/archive-node/getting-started). +### Step 4: To verify a payment, Alice and Bob's accounts show the result of the transfer -## B +After the whole block has been proven, the block producer sends out a confirmation of the transition block. Then member nodes on the network apply the changes to their local account balances to reflect these changes. -### Base58 +### Step 5: To achieve a payment confidence level, Alice is confident the transfer is complete -A group of encoding/decoding schemes used to switch data between binary format (hexadecimal) and alphanumeric text format (ASCII). The Base58 alphabet includes numbers (1 to 9) and English letters, except O (uppercase o), I (uppercase i), and l (lowercase L). These letters are omitted to avoid confusion. +With each subsequent block, a recipient has a higher degree of confidence that the payment is actually complete and that the network has consensus about that block. However, like in most blockchains, payments are said to be confirmed after a certain number of blocks, also known as transaction finality. -### Base64 +In the Bitcoin network, a transaction is confirmed after [6 blocks](https://en.bitcoin.it/wiki/Confirmation) (60 mins) with an assumption that an attacker is unlikely to amass more than 10% of the hashrate. -A binary-to-text encoding scheme that represents binary data in a human-readable ASCII string format string. For example, the zero knowledge proof is a Base64 string inside the `authorization` field of a transaction. +With a slot duration of 3 mins and assuming 90% honest stake, the following table shows the finality in blocks, the average time it takes to produce the corresponding number of blocks, and the confidence that payment will be confirmed. -### best tip +| Finality (in blocks) | Average time for finality | Finality confidence (%) | +| -------------------- | ------------------------- | ----------------------- | +| 8 | 33 mins | 98.6709 | +| 15 | 60 mins | 99.9231 | +| 23 | 1hr 32mins | 99.9965 | +| 30 | 2hrs | 99.9998 | +| 38 | 2hrs 32mins | 100 | -The blockchain's latest block with the highest [chain strength](#chain-strength) known to the mina daemon. +Average time is calculated based on consensus constants that determine the number of slots filled per epoch. This is currently set to 75%. -### bitwise operations +The recommended wait time for a transaction to be confirmed is 15 blocks which provides a 99.9% confidence that the transaction will not be reversed. -Generally available in most programming languages, including TypeScript, [bitwise operations](/zkapps/o1js/bitwise-operations) manipulate individual bits within a binary representation of a number. o1js provides versions of bitwise operations that operate on Field elements and result in the necessary circuit constraints to generate a zero knowledge proof of the computation. +## Failure Scenarios -### block +Payments can fail for several reasons. -A set of transactions and consensus information that extend the state of the network. A block in Mina includes a proof that the current state of the network is fully valid. See also [extensional blocks](/glossary#extensional-blocks) and [precomputed blocks](/glossary#precomputed-blocks). See [What's in a Block?](/mina-protocol/whats-in-a-block) +### Transaction is not accepted by the network -### blockchain +Several reasons why a transaction shared with peer nodes might not get accepted: -The data structure that is used in a cryptocurrency to maintain a shared state of all accounts in the network. +- The transaction is not fundamentally valid. For example, the sender's account doesn't exist, the account doesn't have sufficient funds, the signature doesn't match with the account, or the nonce in the transaction was not incremented. +- There could be adversarial nodes in the network that collude to deny service to specific senders in the network. However, this behavior is highly disincentivized and one honest node is enough to prevent this issue. -### block confirmations +### Transaction is not included in a block -The number of blocks added after the reference block. As the number of confirmations increases, the likelihood of a [reorganization](/glossary#reorganization) decreases, thereby increasing the likelihood of all transactions in the reference block being confirmed. +If a transaction is valid and the network is honest, then in all likelihood, a transaction will make it into a block. However, there is one case where a transaction can be dumped from a transaction pool: -### block explorer +- If the transaction pool hits its capacity, or `max_txpool_size`, then it evicts the transaction with the lowest fee in the pool, causing it to be dumped from memory. If this happens, the sender needs to resend the transaction with a higher fee, according to market dynamics at the time. -A web-based tool to extract, visualize, and review blockchain network metrics, including transaction histories, wallet balances, and details about individual blocks and transactions. Block explorers for Mina include: +--- +url: /mina-protocol/proof-of-stake +--- -- https://minascan.io -- https://minataur.net +# Proof of Stake -### block fill rate +The proof of stake consensus mechanism implemented in Mina is a version of the [Ouroboros Praos](https://iohk.io/research/papers/#XJ6MHFXX) protocol, extended and modified slightly for our succinct blockchain. This document will provide an high level overview of how Ouroboros proof of stake works with detailed sections on our changes and additions. For a full description of the Ouroboros protocol, please refer to the original Ouroboros papers: [the original](https://eprint.iacr.org/2016/889.pdf) and [Praos](https://eprint.iacr.org/2017/573.pdf). -The proportion of [slots](#slot) that should contain a block. Some slots are intentionally empty to ensure the network can [catch up](/glossary#catchup) in case of delay of messages. +### A note on Praos, Genesis, and Mina -### block header +Formally, our implementation of Ouroboros is an extension of Praos. There is, however, a newer paper which extends Praos, called Ouroboros Genesis. This extension fixes a vulnerability involving long fork attacks, but this cannot be implemented in a succinct blockchain protocol as described in that paper. Instead, we introduce a new, succinct method for protecting against long fork attacks. -The portion of a block that contains information about the block itself (block metadata), typically includes a timestamp, a hash representation of the block data, the hash of the previous block's header, and a cryptographic nonce (if needed). +## Additions to Praos -### block producer +### Epoch Ledger Optimization -A node that participates in a process to determine what blocks it is allowed to produce and then produces blocks containing transactions that can be broadcast to the network. People who run [block producer](/mina-protocol/block-producers) nodes are also called block producers. +In Ouroboros, nodes need to materialize/keep a ledger at the beginning of the previous epoch in order to VRF evaluations. This is because just having a VRF output is not enough to know that a block was proposed honestly. In order to know that a block was won by the node that proposed it, the VRF output must be such that it is underneath a threshold determined by the proposer's stake, proportional to the total currency in the ledger. In a succinct protocol, such as Mina, materializing/keeping such a ledger in the past is not an easy task. Unlike a non succinct blockchain, nodes cannot arbitrarily request pieces of the chain in the past in order to reconstruct the information they are interested in. This means if we were to implement this feature in a naive manner, we would need to keep in total 3 copies of the entire ledger at any point in time, as well as wait online for at least 2 whole epochs before we could access that information. With some thought, though, we can do much better. -### bootstrap +There is a big difference between how Ouroboros proves block winners and how Mina proves them. In Ouroboros, every node needs to validate that a block proposer actually won a block when they receive it, meaning they must look up the balance for the public key that evaluated the VRF themselves. In Mina, however, the correctness of the VRF evaluation can be calculated in the SNARK, so other nodes only need to verify the SNARK to know that the block was proposed by a winner. Since a proposer proves this itself, it can limit the information it needs to store about epoch ledgers to only the account record and the merkle path for any account it can propose for (itself and all of its delegated accounts). Furthermore, since Ouroboros guarantees us that the point of finality will be reached before two epochs, the proposer can wait to capture this information after finalization, further limiting the information it needs to store. This information is then fed into the SNARK, which proves that: 1) the VRF evaluation is accurate for the provided public key, and 2) the account for the public key exists in the epoch ledger with a balance that creates a VRF threshold greater than the VRF output (using the merkle path to prove the epoch ledger's merkle root from the account). This does not immediately address the issue of a proposer node needing to be online long enough in order to store this necessary information, but it does open up other avenues of the proposer acquiring that information. Mainly, the proposer could request an account record and merkle path proving it's existence at a given epoch ledger. In the current implementation, no nodes store this information and make it available for access, but one could imagine in the future a service being built that would allow proposers to get online and active quicker by providing this information, possibly for some sort of fee. -Part of the [syncing](#syncing) process of a node, bootstrapping gets the current [root](#root-of-transition-frontier) of the [transition frontier](#transition-frontier) from peers. Additional [transitions](#transition) obtained during the [catchup](#catchup) process are applied from this initial root state. + -### breadcrumb +--- +url: /mina-protocol/scan-state +--- -A node in the [transition frontier](#transition-frontier) that contains the external transition, staged ledger, and pending coinbases and is generated by applying the transition to the prior state. +# Scan State -## C +The scan state is a data structure that allows decoupling the production of transaction SNARKs from block producers to SNARK workers. -### catch up {#catchup} +[Block producers](/mina-protocol/block-producers) do not have to produce transaction SNARKs, so the block production time can remain constant regardless of the transaction throughput. The scan state data structure allows the transaction SNARK proof generation to be parallelized and completed by multiple competing [SNARK workers](/mina-protocol/snark-workers). -The final stage of the [syncing](#syncing) process where the node attempts to catch up to the current [best tip](#best-tip) by determining and then downloading all [transitions](#transition) between the transition frontier [root](#root-of-transition-frontier) and the current best tip. First, a node requests the missing transition hashes and a transaction chain proof. This proof proves the path provided is valid, for example, that the provided transition hashes lead from the root to the best tip. After the node has all transition hashes, it requests the full external transition for each transition hash from peers. +The scan state is comprised of a forest of full [binary trees](https://en.wikipedia.org/wiki/Binary_tree), where each node in the tree is a job to be completed by a SNARK worker. The scan state periodically returns a single proof from the top of a tree that attests to the correctness of all transactions at the base of the tree. -With each external transition, the node builds up its transition frontier by applying each to the prior state to construct a [breadcrumb](#breadcrumb). When the catch up stage is complete, the node's local best tip is the same as the network's best tip, and breadcrumbs have been constructed for all transitions from the transition frontier root (best tip - k) to the current tip, and each has been validated. At this point, the node is [synced](#syncing). +The block producers include the emitted ledger proof in the blockchain SNARK they generate that proves both the chain's current state is valid and attests to the validity of all transactions included in the SNARKed ledger. -A catch up can be triggered at any time if the node sees a disjoint transition in the same path that indicates there are missing transitions. +As a result, block times can remain constant regardless of the transaction throughput. The scan state is capable of adjusting to match a desired transaction throughput. -### chain strength +:::tip -Full history is not available in Mina, so a newly connected node to the network cannot sync from genesis by applying all prior transitions. To allow a node to determine the strongest chain, a minimum chain density is stored for a sliding window of time. As a result, honest nodes can choose the blockchain with the higher minimum density or chain strength. +In a steady state, when all slots are filled and all the required proofs are completed, a ledger proof is emitted every block. -### cold wallet +::: -A wallet is "cold" if the private key is not, and never has been, available on the internet. Cold storage is preferred for wallets associated with meaningful stake as it is harder to hack into cold wallet systems if they never have been on the internet. This could be as easy as generating a key pair on a laptop with the internet turned off or using a hardware wallet, like a [Ledger](https://shop.ledger.com/) device. +### Including transactions -### compressing +When constructing a block, a [block producer](/mina-protocol/block-producers) can include transactions up to the maximum defined by the [scan state constants](#scan-state-constants). Block producers can pick up any available transaction fees and pay themselves a coinbase reward by including transactions. Each transaction they add is transformed into new base jobs and added to the scan state. -Generating a SNARK for a computation output can be thought of as compressing that output, as the proofs are fixed size. For example, Mina maintains a succinct blockchain by compressing all the historical data in a blockchain into a zk-SNARK. However, this is computationally different from lossy compression. The term _compress_ is used to more figuratively describe the process of reducing the size of required data. +For every transaction added, a block producer must include an equivalent amount of completed SNARK work corresponding to a sequence of jobs already existing in the scan state. When added to the scan state, these completed jobs create new merge jobs, except for the root node, in which case the proof is returned as a result. -### consensus +The block producer, rather than completing the work themselves, can purchase the completed work from any SNARK workers from bids available in the SNARK pool (snarketplace). -A process through which all the peers of a blockchain network reach a common agreement about the present state of the distributed ledger. A consensus algorithm or set of rules that Mina nodes all agree upon when deciding to update the state of the network. Rules can include what data a new block can contain and how nodes are selected and rewarded for adding a block. Mina implements the [Ouroboros Samisika](/glossary#ouroboros-samisika) consensus mechanism. +### Scan state constants -### consensus node +The following constants dictate the structure and behavior of the scan state: -A participant in the Mina network that performs the consensus function, for example, a [block producer](#block-producer). +- `transaction_capacity_log_2` +- `work_delay` -### cryptocurrency +The `transaction_capacity_log_2` constant defines the maximum number of transactions that can be included in a block: -A digital asset or currency that uses cryptographic primitives to secure financial transactions and to verify ownership by using public/private key pairs. +``` +max_no_of_transactions = 2^{transaction_capacity_log_2} +``` -## D +The work delay ensures there is enough time for the SNARK work to be completed by the SNARK workers. The block producer cannot include any transactions if no completed proofs are available. With the work delay, the maximum number of trees that can exist in the scan state is defined by: -### daemon +``` +max_number_of_trees = (transaction_capacity_log_2 + 1) * (work_delay + 1) + 1 +``` -The Mina daemon is a background process that implements the Mina protocol and runs on a node locally so a local client or wallet can talk to the Mina network. For example, when a CLI is used to issue a command to send a transaction, this request is made to the Mina daemon, which then broadcasts it to the peer-to-peer network. The daemon also listens for events like new blocks and relays this to the client by using a publish-subcribe model. +The maximum number of proofs that can be included per block is defined by: -### DAO +``` +max_number_of_proofs = 2^{transaction\_capacity_log_2 + 1} - 1 +``` -A decentralized autonomous organization (DAO) operates based on rules that are encoded on a blockchain and executed through smart contracts. DAOs are an organizational structure built with blockchain technology. +These scan state constraints ensure that: -### Dapp +- Only a single proof can be emitted per block +- The merge node to be updated after adding proofs corresponding to its children is always empty. -A decentralized application (Dapp) runs on a blockchain or decentralized network and offers benefits such as transparency, security, and censorship resistance. In the Mina ecosystem, Dapps are known as [zkApps](#zkapps). +While the maximum number of transactions can be fixed, this number can dynamically adjust to the transaction throughput. As such, the scan state can handle an unlimited transaction throughput, albeit at the cost of increasing (logarithmically) the transaction proof latency. -### delegating +### Example -Because staking MINA requires nodes to be online, some nodes delegate their MINA to another node that runs a staking service. This process is called delegating a stake. The service provider or staking pool operator can charge a fee that is deducted any time the delegator gets selected to be a block producer. +Consider a scan state with `max_no_of_transactions = 4`, and `work_delay = 1`. Accordingly, this means there can be a maximum amount of work to complete equal to 7 and a maximum of 7 trees. At **genesis**, the scan state is empty. -### deploy alias +Block 0 -Created with the zkApp CLI, a [deploy alias](/zkapps/tutorials/deploying-to-a-network#deploy-alias) in your project `config.json` file contains the details to manage deployments. +**Block 1**: A block producer includes four transactions into the scan state labeled `B1`. These transactions fill the base of the first tree. -### Devnet +Block 1 -Dedicated for developers building on top of the Mina protocol, Devnet is designed for testing and experimentation so you can test tooling and integrations before going live on [Mainnet](#mainnet). See [Connect to Mainnet or Devnet](/node-operators/validator-node/connecting-to-the-network). +**Block 2**: At the second block, a block producer adds another four transactions (`B2`). These are added to a second tree, once again filling the base. There are no proofs required due to the work delay of 1 block. -### distributed ledger technology (DLT) +Block 2 -A digital system for recording the transaction of assets in which the transactions and their details are recorded in multiple places at the same time. In contrast to traditional databases, distributed ledgers have no central data store or administration functionality. +**Block 3**: At the third block, a block producer adds four `B3` transactions to the third tree but must include four proofs for the first tree. As a result of including these completed base proofs, two new `M3` merge jobs are created. -## E +Block 3 -## ECDSA +:::tip -Elliptic Curve Digital Signature Algorithm ([ECDSA](/zkapps/o1js/ecdsa)) a cryptographic algorithm used to sign and verify messages. It is used in many blockchains, including Ethereum, to sign transactions. +`B` or `M` indicates a base or merge job, with the number indicating the sequence order of being added to the scan state. -## elliptic curves +::: -Equations with a specific template, including: _y^2 = x^3 + ax^ + b_: [secp256k1](/glossary#secp256k1). +**Block 4**: For the fourth block, a block producer adds another four transactions (`B4`) to the base of the fourth tree. They must include four proofs corresponding to the work added in block 2. Again, two `M4` merge jobs are created as a result. -### elliptic-curve cryptography (ECC) +Block 4 -An approach to public key cryptography based on the algebraic structure of elliptic curves over finite fields. ECC is the basis of how Ethereum and other cryptocurrencies use private keys and digital signatures. +:::tip -### epoch +Any pending work (displayed in orange) is work for the SNARK workers to complete. The SNARK workers submit completed work to the SNARK pool. Multiple SNARK workers can complete the work, but only the lowest fee remains in the SNARK pool that can be purchased by the block producers. -A unit of time equal to 7140 slots at Mainnet. An epoch is divided into [slots](#slot) of 3 minutes each. +::: -### extensional blocks +**Block 5**: In the fifth block, another four transactions are included to fill the base of tree five (`B5`), and six proofs must be included (`B3`s and `M3`s). The `M3` merge jobs result in a final pending merge job for the first tree (`M5`). -Blocks extracted from the `mina-archive` database contain only the information required to restore data to the archive node and are more lightweight than [precomputed blocks](/glossary#precomputed-blocks). +Block 5 -### external port +**Block 6**: In the sixth block, another four transactions (`B6`) are added, filling the base of the sixth tree. Six proofs are included (`B4` and `M4`), and three new merge jobs are created (`M6`). -The port that the Mina daemon uses to connect to other nodes on the network. When starting the daemon, set using `-external-port`. +Block 6 -### external transition +**Block 7**: In the seventh block, the block producer adds a further four transactions (`B7`), filling the base of the seventh tree. Seven trees are the maximum number of trees according to the specified scan state constants. The maximum number of proofs (7) are included (`B5` and `M5`). These included proofs create three new merge jobs (`M7`);additionally, the top `M5` proof is emitted from the scan state. -Also referred to as a [block](#block), an external transition is generated externally, for example, by another block producer, and gossiped to a node. +Block 7 -## F +The proof that is emitted from the first tree is the ledger proof corresponding to the transactions added in block 1. The contents of the tree are then removed to create space for additional transactions. -## fee payer account +Emit proof -A developer account that is funded and can always pay fees immediately. When you configure a zkApp, you can choose to use a stored account or create a new fee payer account. +**Block 8**: In the eighth block, the block producer adds two transactions (`B8`) and includes 4 (`B6`) proofs. These included proofs result in two new merge jobs (`M8`). Note that only four proofs are required for adding two transactions. -### field element +Block 8 -The basic unit of data in zero knowledge proof programming. Each field element can store a number up to almost 256 bits in size. You can think of a field element as a uint256 in Solidity. For the cryptography inclined, the exact max value that a field can store is 28,948,022,309,329,048,855,892,746,252,171,976,963,363,056,481,941,560,715,954,676,764,349,967,630,336. +:::tip -### finality +SNARK work is bundled into a work package typically containing two _workIds_, except for the final root proof of a tree. Prorated work for a transaction is two proofs, ensuring the equality of transactions included and SNARK work to be purchased. -A consensus constant `k` is the point at which chain [reorganizations](#reorganization) are no longer possible. After a block has `k` [block confirmations](#block-confirmations) as defined by the consensus constants, it is considered final. +::: -## foreign field +**Block 9**: The block producer adds three transactions (`B9`) in the ninth block. Three proofs (`M6`) are required to occupy the slots in the currently unfilled tree. Four proofs were added in the previous block, so only three more proofs need to be done (given the maximum work is 7). The `M6` proof from tree two is returned as the ledger proof. The third `B9` transaction goes into the now empty tree, and two `B7` proofs are added. -A finite field different from the native field of the proof system. [Foreign Field Arithmetic](/zkapps/o1js/ecdsa) lets you perform algorithms that connect your zkApp with the outside world of cryptography. +Block 9 -### full node +**Block 10**: In block ten, the block producer adds four transactions and, as a result, includes seven proofs (`B7`, `M7`, and two `B8`s). -A Mina node that is able to verify the state of the network trustlessly. In Mina, every node is a full node since all nodes can receive and verify zk-SNARKs. +Block 10 -# G +**Block 11**: In the eleventh block, the block producer adds three transactions (`B11`) and completes five proofs (`B9`, `B9`, `M8`, `M8`, `M9`) in that order. In addition, the `M9` ledger proof is returned from the fourth tree. -## gadgets +Block 11 -Small, reusable, low-level building blocks that simplify the process of creating new cryptographic primitives. Most [gadgets](/zkapps/o1js/gadgets) build upon custom gates and act as low-level accelerators in the proof system. +:::tip -## H +To view the contents of the scan state, run the `mina advanced snark-job-list` command. -### hash +::: -A mathematical cryptographic function that converts an input of arbitrary length into an encrypted output of a fixed length. Hashing provides security through encryption and is an efficient store of data because the hash is of a fixed size. +### Integration with the SNARK Pool -### hot wallet +Newly added jobs to the scan state are pending jobs for SNARK workers to complete. SNARK workers complete the required transaction SNARKs, submitting bids for their completed work. When a node receives and validates the completed work, SNARK workers add the completed work to the local SNARK pool if it is valid and has the lowest fee for the required work. The work is also gossiped to other peers in the network. -A wallet is "hot" if the private key is available on a machine that is connected to the internet. To mitigate risk in the case of hackers breaking into their systems, careful block producers avoid having hot wallets with substantial stake on them. +:::tip -## I +While multiple SNARK workers can complete the same work, only the lowest fee is included in the SNARK pool. -### internal transition +::: -A [transition](#transition) that is produced locally, for example, by a block producer. The generated transition is applied locally and added to the [transition frontier](#transition-frontier) before being broadcast to peers. +When a block producer includes completed proofs into a block to offset any transactions they add, they may purchase the corresponding work from the SNARK pool. Continuing the previous example, consider the next block (12). If the block producer wants to add three transactions, comprising a coinbase, a user payment, and a fee transfer to the SNARK worker, the block producer must purchase three completed SNARK works. This corresponds to the six `B9`, `B10`s, `M9`, and `M10` (from the seventh tree) proofs, as each SNARK work includes two _workIds_. -## K +During the time the block is generated, the SNARK pool can include completed work and the best bids for the required jobs (0.025, 0.165, 0.1, and 0.5) respectively, in the example. -## Keccak +Submitted work -[Keccak (SHA-3)](/zkapps/o1js/keccak) is a flexible cryptographic hash function that provides more security than traditional SHA hash algorithms. +A block producer considers the price of available work before selecting transactions. -### key pair +- The first transaction a block producer adds is the coinbase transaction for which there is the coinbase reward. +- If transaction fees do not cover the SNARK work fees required for them to be included, the transaction is not added. -A combination of a [private key](#private-key) and [public key](#public-key). Key pairs can be generated by using a running daemon or using a dedicated keygen tool, see [Generating a Key Pair](/node-operators/validator-node/generating-a-keypair). In Mina, public keys start with `B62` and private keys start with `EK` for easy differentiability. +A block producer never purchases work if it is not economical. -### Kimchi +If completed SNARK work is not available to purchase in the order required, then the corresponding transactions are not included in a block. This situation can result in an empty block, but also, for the case where no transactions can be added (including a coinbase transaction), there is no reward for the block producer. -The proof system for Mina, Kimchi is the main machinery that generates the recursive proofs that keep the Mina blockchain small (about 22 KB). Kimchi is a zero knowledge proof system that's a variant of [PLONK](/glossary#plonk) and features a polynomial commitment scheme that supports verifiable computation using traditional Turing machine-based instruction sets. +To view the current SNARK pool, use: -## L +- [GraphQL API](/node-developers/graphql-api) +- Mina CLI `mina advanced snark-pool` command -### layer 1 (L1) +--- +url: /mina-protocol/sending-a-payment +--- -The fundamental, base-level chain in a network. An L1 blockchain provides the essential services to a network, like recording transactions on the public ledger and ensuring adequate security. Mina is a layer 1 blockchain. +# Sending a Payment -### layer 2 (L2) -An off-chain network, system, or technology built on top of a layer 1 blockchain that helps extend the capabilities of the underlying base layer network. -### ledger +How to send a MINA payment using the Mina CLI. -A cryptocurrency public record-keeping system. Mina has three types of ledgers: [staged ledger](/glossary#staged-ledger), [staking ledger](/glossary#staking-ledger), and [SNARKed ledger](/glossary#snarked-ledger). +:::info -### libp2p +Reminder that this section is intended for node operators. If you want to store, send, and receive MINA without running a node, please see the [Install a Wallet](../using-mina/install-a-wallet) page for links to various user-friendly wallets available for Mina. -Mina uses this peer-to-peer networking library to provide things like message broadcast and file sharing. +::: -### Lightnet +In this section, we'll give a brief overview on how to send a transaction with the Mina client and how to get started with interacting with the blockchain. -A lightweight Mina network in a single Docker container. [Lightnet](zkapps/writing-a-zkapp/introduction-to-zkapps/testing-zkapps-lightnet) is a resource-efficient solution with fast startup and syncing times that lets you test your zkApp locally on an accurate representation of Mina blockchain before you test with a live network. +## Using an offline signed-transaction -### lightweight Mina Explorer +If you want to send a transaction without running a node yourself, but by delegating to someone else running a node, keep following along here. If you wish to send the transaction directly with a running node, skip to [using a connected node](#using-a-connected-node). -Provided in Lightnet, a lightweight Mina Explorer lets you monitor transactions on your local network. See [Testing zkApps with Lightnet](zkapps/writing-a-zkapp/introduction-to-zkapps/testing-zkapps-lightnet#lightweight-mina-explorer). +### Using a Ledger device -## M +To generate a signed transaction offline if your private key is on a Ledger device, see [Ledger Hardware Wallet](/using-mina/ledger-hardware-wallet). -### major upgrade +### Using a keypair generated with the generate-keypair tool -Changes to the network that make the old chain incompatible with the new chain. A major upgrade to the Mina network requires all nodes or users to upgrade to the latest version of the protocol software. +A better tool is coming soon: https://github.com/MinaProtocol/mina/issues/8928. For now, please use the [workaround](https://github.com/MinaProtocol/mina/issues/8928#issuecomment-857095846) provided in a comment on that issue. -### Mainnet +### Using a keypair generated with the offline client-sdk -The live version of the Mina blockchain network that is fully operational. On the Mina Mainnet public blockchain, real-world transactions are performed. See [Connect to the Mina Network](node-operators/validator-node/connecting-to-the-network). A Mainnet is different from a [Testnet](#testnet) and [Devnet](#devnet) which are used for development and testing. +Use the [Mina Signer](/mina-signer). See the [Mina Signer documentation](/mina-signer) for installation and usage instructions. -### MINA +### Send the transaction -The unit of the cryptocurrency that is exchanged by participating nodes on the Mina network. MINA is the exclusive currency of the [snarketplace](#snarketplace). +You can use a hosted service to broadcast your signed transaction. Sending your signed transaction _does not_ leak your private key. -### Mina +Transactions signed with the Mina Signer can use: [https://minascan.io/mainnet/broadcast/payment](https://minascan.io/mainnet/broadcast/payment) -The underlying protocol and the network infrastructure that the system depends on. +Transactions signed with the Ledger hardware wallet can use: [https://minascan.io/mainnet/broadcast/ledger-payment](https://minascan.io/mainnet/broadcast/ledger-payment) -### Mina CLI +## Using a connected node -The primary way for users to interact with the Mina network. The [Mina CLI](/node-operators/reference/mina-cli-reference) command line tool provides standard client functionality to create accounts, send transactions, and participate in consensus and advanced client and daemon commands for power users. +We are assuming in the rest of section that you have the Mina client installed on your system, if you do not have Mina installed please see the [Getting Started](/node-operators/block-producer-node/getting-started). -The Mina CLI is installed when you [install Mina](/node-operators/validator-node/installing-on-ubuntu-and-debian). +## Import your account -### Mina nodes +Once our node is synced, we'll need to import our public/private keypair so that we can sign transactions and generate an address to receive payments. For security reasons, we'll want to put the keys under a directory that is harder for attackers to access. -Mina nodes fulfill different roles within the network, including [block producers](#block-producer) and [SNARK coordinators](#snark-coordinator). +Run the following command to import your [previously generated](/node-operators/validator-node/generating-a-keypair) keypair file: -## N + mina accounts import --privkey-path ~/keys/my-wallet -### node +You will be prompted for the password you entered when the account was created. -A machine running the Mina daemon. +:::caution -### node operators +The public key can be shared freely with anyone, but be very careful with your private key file. Never share this private key with anyone, as it is the equivalent of a password for your funds. -People who run Mina nodes. Node operators participate in consensus to create new blocks and help compress data by generating zk-SNARKs. +::: -### non-consensus node +The response from this command will look like this: -A [full node](#full-node) in the Mina protocol that does not participate in consensus but can still fully verify the zero knowledge proof to trustlessly validate the state of the chain. The size of Mina as 22 KB is in reference to non-consensus nodes. + 😄 Imported account! + Public key: B62qjaA4N9843FKM5FZk1HmeuDiojG42cbCDyZeUDQVjycULte9PFkC -### non-upgradeable +Additionally you can use the `mina accounts create` command to generate new accounts to send and receive transactions. -If the verification key cannot be changed, a zkApp smart contract is considered non-upgradeable. You can make a smart contract upgradeable or not upgradeable using [permissions](/zkapps/writing-a-zkapp/feature-overview/permissions#upgradeability-of-smart-contracts). +Since the public key is quite long and difficult to remember, let's save it as an environment variable. Use the following command but replace `` with the public key output from the previous command: -### nonce + export MINA_PUBLIC_KEY=`` -An incrementing number attached to a transaction used to prevent a replay of a transaction on the network. Transactions are always included in blocks in the sequential order of the nonce. +Now we can access this everywhere as `$MINA_PUBLIC_KEY` -- check if it saved properly by trying `echo $MINA_PUBLIC_KEY`. -## O +Note that these environment variables will only be saved for the current shell session, so if you want to save them for future use, you can add them to `~/.profile` or `~/.bash_profile`. -### o1js +:::tip -A TypeScript library for zk-SNARKs and zkApps. Use o1js to write zk smart contracts based on zero knowledge proofs for the Mina Protocol. +If you are running the node on a cloud virtual machine, make sure to export and save the key file. You can export the key with: -### off-chain + mina accounts export --public-key `` --privkey-path `` -A transfer of value or data, including transactions, that occurs outside a given blockchain network. These transfers do not need blockchain network confirmation, which speeds up the transaction process and reduces lag time. zkApps use an off-chain execution and mostly off-chain state model that allows for private computation and state that can be either private or public. +Then save it to your local machine, maybe using [scp](https://linux.die.net/man/1/scp): -### off-chain state + scp `` `` -State stored anywhere other than the Mina blockchain. +Later, when starting up a new VM, you can upload the key and then import it: -### on-chain + mina accounts import --privkey-path `` -A transfer of value or data, including transactions, that exist on and have been verified to a blockchain network. All relevant information is timestamped and stored on the public ledger. On-chain transactions are recorded on the blockchain and need network confirmation before they are completed. +::: -### on-chain state +If you ever forget what keypairs you've already created, you can see them all with: -State that lives on the Mina blockchain. Each zkApp account provides eight fields of 32 bytes each of arbitrary storage for the on-chain state. + mina accounts list -### oracle +## Check account balance -Specialized software or services that act as intermediaries between blockchain smart contracts and external data sources. [Oracles](/zkapps/tutorials/oracle) connect zkApp smart contracts with the outside world to get data on-chain. +We can check the balance of all our accounts using this command: -### Ouroboros Samisika + mina accounts list -Mina builds on this provably secure proof of stake (PoS) protocol that combines the best features of each iteration of Ouroboros to deliver a PoS consensus mechanism that can resolve long-range forks without requiring history or risking centralization by relying on trusted third parties to provide fork information. +You might see `Balance: 0 mina` for your account. Depending on the traffic in the network, it may take a few blocks before your transaction goes through. -## P +:::tip -## pasta curves +You can run `mina client status` to see the current block height updating. -A collective term for the Pallas and Vesta elliptic curves that are used by the Mina Protocol to generate proofs. See [Pasta Curves](https://o1-labs.github.io/proof-systems/specs/pasta.html?highlight=curves#pasta-curves) in the Mina book. +::: -### peer-to-peer networks +## Make a payment -Networking systems that rely on peer nodes to distribute information amongst each other, are often distributed, and do not rely on any centralized resource broker. +Finally, we get to the good stuff–sending our first transaction! Before you send a payment, you'll need to unlock your account: -### Pickles + mina accounts unlock --public-key $MINA_PUBLIC_KEY -Mina's inductive zk-SNARK composition system. See [Pickles](https://o1-labs.github.io/proof-systems/specs/pickles.html). +For testing purposes, we will specify your public key as the receiver and sender. This just means that we are sending a transaction to ourselves, you can see your public key by issuing the following command: -### Pickles SNARK - -A proof system and associated toolkit that is the first deployed [SNARK](#snark) capable of recursive composition with no trusted setup. Pickles serves as the basis for developers to build private, scalable smart contracts on Mina. [Meet Pickles SNARK: Enabling Smart Contracts on Mina Protocol](https://medium.com/minaprotocol/meet-pickles-snark-enabling-smart-contract-on-coda-protocol-7ede3b54c250). +``` +echo $MINA_PUBLIC_KEY +``` -### PLONK +:::caution -Permutations over Lagrange-bases for Oecumenical Noninteractive arguments of Knowledge (PLONK) is a general-purpose zero knowledge proof scheme. +If the receiving account has not received any transactions, there will be an additional Account Creation Fee of `1 MINA` that will be deducted from the transaction amount. -### polynomial commitment +::: -A commitment scheme that allows a committer to commit to a polynomial with a short string that can be used by a verifier to confirm claimed evaluations of the committed polynomial. +Let's send some of our Mina to ourselves to see what a payment looks like: -### Poseidon + mina client send-payment \ + --amount 1.5 \ + --receiver $MINA_PUBLIC_KEY \ + --fee 0.1 \ + --sender $MINA_PUBLIC_KEY -A family of [hash](#hash) functions that can efficiently run in a zk circuit. Poseidon operates over the native [Pallas base field](https://electriccoin.co/blog/the-pasta-curves-for-halo-2-and-beyond/) and uses parameters generated specifically for Mina, which makes Poseidon the most efficient hash function available in o1js. See [Poseidon: ZK-friendly Hashing](https://www.poseidon-hash.info/). Poseidon is a sponge construction based on the Hades permutation, with a state composed of field elements and a permutation based on field element operation (addition and exponentiation). +If you're wondering what we passed in to the commands above: -### precomputed blocks +- For `amount`, we're sending a test value of `1.5` mina which is enough to cover the Account Creation Fee +- The `receiver` is the public key of the account receiving the transaction, eg. `B62qjaA4N9843FKM5FZk...` +- For `fee`, let's use 0.1 mina +- The `sender` is the public key of the account sending the transaction, eg. `B62qjaA4N9843FKM5FZk...` -Precomputed blocks are [blocks](/glossary#block) logged by mina to disk, log file, or to cloud storage. +If this command is formatted properly, we should get a response that looks like the following: -### preconditions + Dispatched payment with ID 3XCgvAHLAqz9VVbU7an7f2L5ffJtZoFega7jZpVJrPCYA4j5HEmUAx51BCeMc232eBWVz6q9t62Kp2cNvQZoNCSGqJ1rrJpXFqMN6NQe7x987sAC2Sd6wu9Vbs9xSr8g1AkjJoB65v3suPsaCcvvCjyUvUs8c3eVRucH4doa2onGj41pjxT53y5ZkmGaPmPnpWzdJt4YJBnDRW1GcJeyqj61GKWcvvrV6KcGD25VEeHQBfhGppZc7ewVwi3vcUQR7QFFs15bMwA4oZDEfzSbnr1ECoiZGy61m5LX7afwFaviyUwjphtrzoPbQ2QAZ2w2ypnVUrcJ9oUT4y4dvDJ5vkUDazRdGxjAA6Cz86bJqqgfMHdMFqpkmLxCdLbj2Nq3Ar2VpPVvfn2kdKoxwmAGqWCiVhqYbTvHkyZSc4n3siGTEpTGAK9usPnBnqLi53Z2bPPaJ3PuZTMgmdZYrRv4UPxztRtmyBz2HdQSnH8vbxurLkyxK6yEwS23JSZWToccM83sx2hAAABNynBVuxagL8aNZF99k3LKX6E581uSVSw5DAJ2S198DvZHXD53QvjcDGpvB9jYUpofkk1aPvtW7QZkcofBYruePM7kCHjKvbDXSw2CV5brHVv5ZBV9DuUcuFHfcYAA2TVuDtFeNLBjxDumiBASgaLvcdzGiFvSqqnzmS9MBXxYybQcmmz1WuKZHjgqph99XVEapwTsYfZGi1T8ApahcWc5EX9 + Receipt chain hash is now A3gpLyBJGvcpMXny2DsHjvE5GaNFn2bbpLLQqTCHuY3Nd7sqy8vDbM6qHTwHt8tcfqqBkd36LuV4CC6hVH6YsmRqRp4Lzx77WnN9gnRX7ceeXdCQUVB7B2uMo3oCYxfdpU5Q2f2KzJQ46 -Conditions that must be true for the account update to be applied. Corresponds to assertions in an o1js method. +You may not see the `Receipt chain hash` on the first transaction from the account, but in following transactions, this will show you the head of the receipt chain hash list. -### private key +## Staking and Snarking -A component of public key cryptography, private keys are held privately, while public keys can be issued publicly. Only the holder of the public key's corresponding private key can attest to ownership of the public key. This allows for signing transactions to prove that you are the honest holder of any funds associated with any given public key. In Mina, private keys start with `EK` for easy differentiability from [public keys](#public-key). +Once you feel comfortable with the basics of creating an address, and sending & receiving mina, we can move on to the truly unique parts of the Mina network like [participating in consensus and helping compress the blockchain](/node-operators/validator-node/staking-and-snarking). -### protocol state +## Advanced -The state of the network that comprises the previous protocol state hash to link blocks together and a body that contains the genesis state hash, blockchain state, consensus state, and consensus constants. +### Sending Many Transactions -### protocol state hash +Sometimes you may wish to send many transactions: for example, to payout rewards to those delegating to you if you're running a staking pool. -The hash of hashes of the previous state and [protocol state](#protocol-state) body. Acts as a unique identifier for a block. +All information here is relevant as of the 3.0.3 build: -### proof of liabilities (PoL) +### Rate limiting -A cryptographic primitive to prove the size of funds a bank, or centralized exchange (CEX), owes to its customers in a decentralized manner and can be used for solvency audits with better privacy guarantees. +Currently, nodes on the network will rate limit receiving messages from a given node. As of the 3.0.3 build, your node will also follow this rate limit when sending transactions. Specifically, the limit is currently set at 10 transactions every 15 seconds computed over a 5 minute window. If you attempt to send transactions faster than this rate, your node will queue them up and flush them as older transactions expire from the window upon which the rate limit is computed. You do not need to throttle sending these transactions yourself. -### proof of stake (PoS) +Note that older releases of the mina daemon do not perform this rate limiting; if you are running an older version, you should manually limit the number of transactions. Due to overheads from rebroadcasting transactions, we do not recommend sending more than 50 transactions every 5 minutes if you need to manually rate limit. -The Mina consensus algorithm that allows nodes to agree on the state of the network. PoS allows nodes to [stake](/node-operators/validator-node/staking-and-snarking) MINA on the network to increase their chance of being selected as the next block producer. The winning validators are compensated with a percentage yield of the crypto they have staked as an incentive for engaging in this process. See [Proof-of-Work vs Proof-of-Stake](https://minaprotocol.com/blog/proof-of-work-vs-proof-of-stake). +### My node crashed or disconnected before I could finish sending transactions -### proof of work (PoW) +The Mina daemon does _not_ currently persist the transaction pool. This means that the transactions that your node will be unaware of any transactions that you've sent so far if your node crashes in the middle of this process. As of the 3.0.3 build, you can resend all transactions (exactly in the same manner as before) and they will be rebroadcasted on the network. -The original consensus process used by Bitcoin, the first cryptocurrency. PoW achieves the decentralized consensus needed to add new blocks to a blockchain by using machines to compete against one another by guessing the answer to math problems that have no feasible faster solution. Computational power is a requirement for the PoW protocol success and assumes that those contributing more resources (energy, supercomputers, and infrastructure) to a problem are less likely to want to destroy the protocol. As an incentive to participate in this consensus process, miners are rewarded with tokens. +If you believe you were temporarily disconnected from the network, but your node stayed online (i.e. the gossip network may have missed one or more of your transactions), as of the 3.0.3 build, you can resend any of the transactions locally and they will be broadcasted again to the network even if your node thinks they've already been shared. -### prover function +### Cancelling a transaction and setting a new fee -The function that generates a zero knowledge proof from the smart contract logic. +To cancel a transaction, you'll need to have all the transactions that haven't been committed to the chain yet before in your local transaction mempool. This means if your node crashed (see above) you'll need to resend those earlier transactions. Finally, to cancel a transaction, all you need to do is send a transaction with the same nonce of the one you want to cancel with a larger fee. There is no minimum increment, it just needs to be slightly larger (and large enough such that a block producer will choose your transaction). -### public key +--- +url: /mina-protocol/snark-workers +--- -A component of public key cryptography, public keys can be widely shared with the world and can be thought of as _addresses_ or identifiers for the person who holds the corresponding private key. In Mina, public keys start with `B62` for easy differentiability from [private keys](#private-key). +# SNARK Workers -### publish-subscribe (pub-sub) +While most protocols have just one primary group of node operators (often called miners, validators, or block producers), Mina has a second group — the **SNARK worker**. -A messaging pattern where message senders broadcast messages and notifies any listeners that have previously subscribed to that sender's messages. Mina utilizes pub-sub as a way to notify clients when a new block has been added to the chain. This event can be heard by all listeners, so each listener does not need to independently poll for new data. +SNARK workers are integral to the Mina network's health because these nodes are responsible for snarking, or producing SNARK proofs, of transactions in the network. By producing these proofs, snark workers help maintain the succinctness of the Mina blockchain. -## R +Read on to learn why SNARK workers are needed, how the economic incentives align, and operational details of performing SNARK work. Feel free to click through to any of the sections that are most relevant to your needs. -### recursion +Note: The theory of zk-SNARKs is not covered. Deep knowledge of SNARKs is not required to read this section, but it is helpful to understand in general how SNARKs work and what they are useful for. To learn more, check out this + [What are zk-SNARKs?](https://minaprotocol.com/blog/what-are-zk-snarks) primer first. -The layer 1 architecture of Mina Protocol is based on recursive composition, which means that each block in the blockchain is a tiny snapshot of the entire state of the network. This approach enables Mina to maintain a constant blockchain size of only 22KB regardless of the number of transactions processed. With recursion, you can realize composability between zero knowledge proofs to unlock many powerful technical abilities, such as creating high-throughput applications, creating proofs of large computations, and constructing multi-party proofs. +## How Mina Compresses the Blockchain -### reorganization +The Mina protocol is unique because nodes are not required to maintain the full history of the blockchain like other cryptocurrency protocols. By recursively using cryptographic proofs, the Mina protocol effectively compresses the blockchain to constant size. This compression reduces terabytes of data to a few kilobytes. -When a competing fork of the blockchain increases in length relative to the main branch, the blockchain undergoes a reorganization to reflect the stronger fork as the main branch. After a reorganization, the transactions on the dropped branch are no longer guaranteed inclusion into the blockchain and must be added to new blocks on the longest branch. +However, this isn't data encoding or compression in the traditional sense. Mina nodes _compress_ data in the network by generating cryptographic proofs. Node operators play a crucial role in this process by designating themselves as [SNARK workers](/glossary#snark-worker) that generate [zk-SNARKs](/glossary#zk-snark) for transactions that have been added to blocks. -### root of transition frontier +You can connect a SNARK worker to the network or use a SNARK coordinator. [SNARK coordinators](/glossary#snark-coordinator) can be used to coordinate and distribute work to any [SNARK workers](/glossary#snark-worker) that you are running. If you are running a SNARK coordinator, connect the SNARK coordinator to the network, then connect SNARK workers to the SNARK coordinator. -The root of the [transition frontier](#transition-frontier) is the block `k` blocks from the [best tip](#best-tip). The root is obtained from peers during [bootstrap](#bootstrap). After a new best tip is seen, the root is moved, so only `k` blocks are persisted. The root is the point where the block has been [finalized](#finality) due to consensus. +## Why SNARK Workers? -### remote procedure call (RPC) +Mina's unique property is the succinct blockchain. Each block producer, when they propose a new block to the network, must also include a zk-SNARK along with that block. This allows nodes to discard all historical data that's been finalized, and retain just the SNARK. If you are unfamiliar with the Mina protocol, [this video is a good start](https://www.youtube.com/watch?v=eWVGATxEB6M). -An [RPC](https://en.wikipedia.org/wiki/Remote_procedure_call) is used to communicate between nodes on the network and to interact with the running [daemon](#daemon). +However, it is not only sufficient for the block producers in Mina to generate SNARK proofs of blocks. Transactions also need to be SNARKed. The reason is because the blockchain SNARK does not make any statements of the validity of the transactions included in the block. -## S +For example — let's say the current head of the blockchain has a state hash `a6f8792226...` , and we receive a new block with a state hash `0ffdcf284f...` . This block will contain all the transactions that the block producer has chosen to include in this block, and associated metadata. We will also receive an accompanying SNARK that verifies the statement: -### scan state +> "There exists a block with a state hash 0ffdcf284f which extends the blockchain +> with a previous best tip with state hash a6f8792226." -A data structure that allows decoupling the production of transaction SNARKs from block producers to [SNARK workers](#snark-worker). See [Scan State](/mina-protocol/scan-state). +Notice that this statement says nothing about the validity of the transactions included in the new block. If we were to believe this SNARK, and do nothing else, we may be tricked by a malicious block producer sending this block. Luckily, we have the raw block and we can check each transaction to ensure it is valid. But what about nodes in the network that may just want to receive the proof and not verify each block? -### secp256k1 +## Snarking Transactions -The elliptic curve using this equation _y²=x³+7, a=0 b=7_, constructed in a special non-random way to allow for especially efficient computation. secp256k1 can have a key size of up to 256 bits. All points on this curve are valid public keys. +In order to ensure that nodes can operate without trust on the Mina blockchain, it is important that each node can verify the state of the chain without needing to replay the transactions. In order for this to work, the blockchain SNARK is not enough. We need to know that the transactions are also valid. Well, since SNARKs are good for exactly that, the naive suggestion might be to generate a SNARK of each transaction as they come in, and then combine them. -### seed nodes +However, generating SNARK proofs is computationally expensive — if we had to compute SNARKs serially for each transaction, throughput would be very low and block times would skyrocket. Furthermore, transactions in a real world environment arrive asynchronously, so it would be very tough to predict when to perform the next item of work. -A Mina node that keeps a record of nodes in the network and enables nodes that are joining the network to connect to peer nodes. +Lucky for us, we can leverage two properties about SNARKs: -### SHA-2 +1. proofs can be merged - two proofs can be combined to form a _merge proof_ +2. merges are associative - merge proofs are identical, regardless of the order merged -Secure Hash Algorithm 2 (SHA-2) is a family of two similar hash functions with different block sizes known as SHA-256 and SHA-512 that differ in word size. SHA-256 uses 32-bit words and SHA-512 uses 64-bit words. + + + + SNARK Workers + -### SHA-256 +What these two properties essentially allow us to do is take advantage of parallelism. If proofs can be merged, and it doesn't matter how they're combined, then SNARK proofs can be generated in parallel. Whichever proof is finished first can be combined later with the proofs in progress. This can be envisioned as a binary tree, where the bottom row (the leaves) consists of the individual transaction proofs, and each parent row, the set of respective merge proofs. We can combine these all the way to the root, which represents a state update performed by applying all the transactions. -SHA-256, a part of the SHA-2 family, is a cryptographic hash function that generates a 256-bit (32-byte) hash output and is widely used for traditional Web2 applications and protocols as well as blockchain technology. +In addition, because the SNARK proofs don't depend on each other and we can exploit parallelism, this means anyone can do the work! The end result is that the distributed work pool is permission-less. Anyone with spare compute can join the network as SNARK workers, observe transactions that need to be SNARKed, and contribute their compute. And of course, they will be compensated for their work in what we affectionately call **the snarketplace**. -### SHA-3 +Note: To learn more about the details of how this SNARK work scheme evolved, it is highly recommended to watch this video: [High Throughput with Slow Snarks](https://www.youtube.com/watch?v=NZmq1V-Te0E). If you're interested in functional programming and the details of the scan state (the tree structure described above), we have [a video](https://www.youtube.com/watch?v=ztH_Z5TCe9I) covering the technical details. -Secure Hash Algorithm 3 (SHA-3) is the latest member of the Secure Hash Algorithm family of standards, released by NIST on August 5, 2015. [Keccak](#keccak) was standardized as SHA-3. +## The Snarketplace -### signature +The key dynamic to understand about SNARK work is: -Short for digital signature, a way to establish authenticity or ownership of digitally signed messages. +_Block producers use their block rewards to purchase SNARK work from SNARK workers._ -### simulated local blockchain +As a SNARK worker, you get to share some of the block rewards for each block that includes your compressed transactions. The block producer is responsible for gathering compressed transactions before including them into a block and is incentivized by the protocol to reward SNARK workers. -The local testing blockchain you use in the first phase of testing. Using a simulated local blockchain speeds up development and tests the behavior of your smart contract locally. See [Testing zkApps Locally](zkapps/writing-a-zkapp/introduction-to-zkapps/testing-zkapps-locally) and get step-by-steps guidance in [Tutorial 1: Hello World](/zkapps/tutorials/hello-world#simulated-local-blockchain). +There is no protocol involvement in pricing snarks, nor are there any protocol level rewards for SNARK workers to produce snarks. The incentives are purely peer-to-peer, and dynamically established in a public marketplace, aka the snarketplace. -### slot +You may ask, why does a block producer need to buy SNARKs? Fair question — the reason is because of what we mentioned earlier. In order to know for sure the state at the head of the Mina blockchain is valid, the transactions need to be SNARKed. But if we keep adding more transactions without snarking them at an equal rate, then over time we accumulate work that never gets finished. In order to reach a steady state equilibrium, we need work to be processed at roughly the same rate that work is added. -A unit of time in the Mina network. As of Mainnet launch, a slot in Mina is 3minutes long. An [epoch](#epoch) is divided into slots. Block producers can find eligible slots to produce blocks in to earn rewards. +Since block producers profit from including transactions in a block (through transaction fees and the coinbase transaction), they are responsible for offsetting the transactions by purchasing an equal number of completed SNARK work, thereby creating demand for SNARK work. However, their imperative is to purchase SNARK work for the lowest price from the snarketplace. Conversely, the SNARK workers want to maximize their profit while also being able to sell their SNARK work. These two roles act as the two sides of the marketplace, and over time establish an equilibrium at a market price for SNARK work. -### smart contract +### How to price SNARK work -A tamper-proof program that runs on a blockchain network when certain predefined conditions are satisfied. On Mina, smart contracts (zkApps) are written with [o1js](#o1js). +We anticipate the snarketplace to dynamically rebalance — eg. follow the simple laws of [supply and demand](https://en.wikipedia.org/wiki/Supply_and_demand). While each SNARK work applies to a different transaction, seen from a larger perspective, SNARK work is largely a commodity (meaning it doesn't matter which SNARK worker produces the good — it will be the same). However, there are some nuances, so it may help to have some heuristics for pricing strategy: -### SNARK +- if market price is X, it is likely effective to sell SNARK work for any price below X (eg. X - 1), provided it is profitable after operating expenses. +- block producers are incentivized to purchase more units of SNARK work from the same SNARK worker because there will only be one _fee transfer_ transaction they have to include in the block. + - Basically, the way a block producer pays a SNARK worker is through a special type of transaction called a fee transfer. The BP's incentive is to minimize the number of fee transfers, as each is a discrete transaction that needs to be added to a block (and consequently offset by more SNARK work). Thus, the best case scenario is to buy a bundle of SNARK work from the same SNARK worker. +- some SNARK work will be more important to complete ahead of other work, as it would free up an entire tree worth of memory (see the video above for more details). This is made possible by different work selection methods. Currently, the three methods supported natively are sequential, random and sequential with a random offset. Neither of these however takes advantage of dynamic markets, which is an area of improvement that the Mina community can develop solutions for. -An acronym for succinct non-interactive argument of knowledge. See [zk-SNARK](/glossary#zk-snark). +Since all the data around snarks and prices are public, there are several ways to inspect the snarketplace. One example is [using the GraphQL API](https://youtu.be/XQlfX-LnK_A), and other options include using the CLI, or rolling a custom solution that tracks snarks in the SNARK mempool. -### SNARK coordinator +Stay tuned for more detailed analysis on snarketplace dynamics. We will also be releasing an economic whitepaper shortly that will provide more context. -A role on a mina node in the Mina network. SNARK coordinators generate proofs of transactions by distributing work to a series of [SNARK workers](/node-operators/snark-workers). SNARK coordinators then submit that work to the network, and the proofs are sold to block producers. +See also: [SNARKs and SNARK Workers FAQ](/node-operators/faq#snarks-and-snark-workers) -### SNARK pool +--- +url: /mina-protocol/time-locked-accounts +--- -Referred to as the _snarketplace_, the pool that contains work completed by [SNARK workers](#snark-worker) for required work in the [scan state](#scan-state). The SNARK pool contains only the cheapest work offered by SNARK workers for each work bundle. Multiple SNARK workers compete for the same SNARK work, with only the lowest fee for each being included in the SNARK pool to be bought by block producers. +# Time-Locked Accounts -### SNARK worker +A time-locked account disallows payments that would reduce the balance below a minimum, which depends on the block height. -External processes that connect to a Mina node on the network and create zk-SNARK proofs of transactions to compress the transactions so they can be folded into the tiny blockchain proof. The SNARK worker is incentivized with MINA as compensation to help compress transactions. See [What are SNARK Workers and the Snarketplace?](https://minaprotocol.com/blog/what-are-snark-workers-and-the-snarketplace). +To create a time-lock, you must provide the configuration when creating a new account. This can happen only in the genesis ledger at the beginning of a network. In this section, we'll explore the mechanism behind time-locks and see how to interact with time-locked accounts. -### SNARKed ledger +:::tip -The ledger that contains only the transactions that have an associated proof. The SNARKed ledger is updated after a proof has been emitted from the [scan state](#scan-state). + For the current release, values for time-locked accounts were assigned based on the order in which you signed up. -### snarketplace +::: -Similar to a marketplace where people, or nodes, exchange services for a fee. It revolves around a fixed-size buffer like a queue or shelf of work to do. Block producers add work to this shelf in the form of transactions that need to be SNARKed, and then SNARK workers take the work off the shelf and create SNARKs out of them to process the transactions. Block producers purchase SNARK work for the lowest price from the snarketplace. Conversely, the SNARK workers want to maximize their profit while also being able to sell their SNARK work. These two roles act as the two sides of the marketplace and, over time, establish an equilibrium at a market price for SNARK work. +## Understanding time-locks -### soft fork +A time-lock consists of the following fields `initial_minimum_balance`, `cliff` time, a `vesting_period` time, and a `vesting_increment`. -An upgrade to the network that is backward-compatible. +You can still use an account if it has a time-lock, as long as the account holds enough funds. The amount of funds that are time-locked starts off as `initial_minimum_balance` at the beginning of the network. Once the network reaches a block height equal to the `cliff`, the time-locked amount begins to decrease by the `vesting_increment` amount every `vesting_period`. -### staged ledger +For a more technical explanaition of this process, please see [RFC-0025](https://github.com/MinaProtocol/mina/blob/master/rfcs/0025-time-locked-accounts.md) which has a more in-depth overview. -The current account state that contains a pending accounts ledger and also a pending queue of un-SNARKed transactions known as the [scan state](#scan-state). +### Liquid Balance Details: -### staking +If you'd like to expose liquid balances for vesting accounts at some particular time period it is governed by the following function (Note: this computes the locked portion of an account): -Staking MINA allows nodes on the network to increase their chances of being selected as a block producer in accordance with the consensus mechanism. +``` +(* + * uint32 global_slot -- the "clock" it starts at 0 at the genesis block and ticks up every 3minutes. + * uint32 cliff_time -- the slot where the cliff is (similar to startup equity vesting) + * uint32 cliff_amount -- the amount that unlocks at the cliff + * amount vesting_increment -- unlock this amount every "period" + * uint32 vesting_period -- the period that we increment the unlocked amount + * balance initial_minimum_balance -- the total locked amount until the cliff + *) +let min_balance_at_slot ~global_slot ~cliff_time ~cliff_amount ~vesting_period + ~vesting_increment ~initial_minimum_balance = + let open Unsigned in + if Global_slot.(global_slot < cliff_time) then initial_minimum_balance + else + match Balance.(initial_minimum_balance - cliff_amount) with + | None -> + Balance.zero + | Some min_balance_past_cliff -> ( + (* take advantage of fact that global slots are uint32's *) + let num_periods = + UInt32.( + Infix.((global_slot - cliff_time) / vesting_period) + |> to_int64 |> UInt64.of_int64) + in + let vesting_decrement = + UInt64.Infix.(num_periods * Amount.to_uint64 vesting_increment) + |> Amount.of_uint64 + in + match Balance.(min_balance_past_cliff - vesting_decrement) with + | None -> + Balance.zero + | Some amt -> + amt ) -### staking ledger +``` -The ledger used to determine block producers for a slot, as the probability of finding eligible slots to produce blocks in is proportional to the amount of stake. +## Creating a time-locked account -### staking pool +As of the current release, the only way to create a time-locked account is with the genesis ledger. In future releases we may add commands to `mina client` and the GraphQL API that will allow you to create a new time-locked account. -A pool of delegated funds that is run by a staking pool owner. To avoid the requirement of being online, other nodes can choose to delegate funds to a staking pool. +--- +url: /mina-protocol/whats-in-a-block +--- -### state +# What's in a Block? -The current status or snapshot of all data stored within the blockchain. +A block is a set of transactions and consensus information that extend the state of the network. A block in Mina includes a proof that the current state of the network is fully valid. -### syncing +A block in Mina is constituted of: -To successfully produce a block that extends the Mina blockchain, a node requires the current state. To achieve this, a node initializes to connect to peers, [bootstrap](#bootstrap), and then performs a [catchup](#catchup). +* [Protocol state](#protocol-state) +* [Protocol state proof](#protocol-state-proof) +* [Staged ledger diff](#staged-ledger-diff) +* [Delta transition chain proof](#delta-transition-chain-proof) +* Current protocol version +* Proposed protocol version -The syncing process builds the node's [transition frontier](#transition-frontier) by creating [breadcrumbs](#breadcrumb) for all transitions between the transition frontier's [root](#root-of-transition-frontier) to the current [best tip](#best-tip). When complete, the node is synced. +When a node receives a block from a peer, it is first validated, applied to the existing state, and added to the node's transition frontier. If, according to the [consensus](https://minaprotocol.com/blog/what-is-ouroboros-samasika) rules, it results in increasing the length of the blockchain, the node's best tip is updated, and the root of the transition frontier is moved up to only maintain `k` blocks in the transition frontier. -## T +:::tip -### Testnet +In Mina, blocks are synonymous with "transitions". When this transition (block) is received from a peer, it is referenced as an external transition, whereas one generated and applied locally is referred to as an internal transition. -An instance of a blockchain used for testing and experimenting where MINA tokens have no real value. For example, for [zkApp Developer Tutorials](/zkapps/tutorials) and other development, you can request tMINA funds from the Testnet Faucet to fund your [fee payer account](#fee-payer-account). Mina's public Testnet is feature-complete and is called the `Devnet`. +::: -### time-locked accounts +### Protocol State -An account with a non-vested amount of tokens that cannot be moved until a specific condition has been met, like a number of blocks that has been produced. See [Time-Locked Accounts](/zkapps/writing-a-zkapp/feature-overview/time-locked-accounts). +The protocol state is comprised of the **previous protocol state hash** and a body that contains: -### tMINA - -MINA tokens that have no real-world value. You can request tMINA funds from the Testnet Faucet to fund your [fee payer account](#fee-payer-account) during development. - -### tokens - -A digital asset that typically represents an asset, utility, or value in a particular blockchain ecosystem. The native cryptocurrency of the Mina blockchain is MINA. Tokens can serve a variety of purposes, including accessing platform features (utility tokens), representing ownership of assets (security tokens), or facilitating smart contracts and decentralized applications (Dapps). - -### transaction pool - -An in-memory store of all the transactions that peer has heard on the network. Sometimes referred to as the mempool, each node has a local list of all pending transactions that have been gossiped to the node and validated. - -### transition - -A transition in Mina is synonymous with a [block](#block). - -### transition frontier - -A local data store that contains the last `k` blocks of the network. With a rose tree-type data structure, each node of the tree is a [breadcrumb](#breadcrumb) and can have multiple children (forks). - -### TypeScript - -A superset of JavaScript that adds compile-time type safety. [o1js](https://www.npmjs.com/package/o1js) is a TypeScript library for zk-SNARKs and zkApps. See the official [TypeScript docs](https://www.typescriptlang.org). - -## U - -### user transaction - -A transaction that is issued by a user, like a payment or a delegation change. - -## V - -### verification key - -A piece of data that is generated by the smart contract build process. When a smart contract is deployed, a transaction that contains the verification key is sent to an address on the Mina blockchain. Sending a verification key to a zkApp account allows Mina to verify zero knowledge proofs that were generated by a smart contract's prover function. - -### verifier function - -The function that verifies a zero knowledge proof using the [verification key](#verification-key). - -### verifiable random function (VRF) - -A function that generates an output that can be cryptographically verified as random. Mina uses VRF to select a block producer for a slot, taking as input a random seed that is derived from the previous epoch's VRF outputs, a public key, and the current staking ledger. VRF is deterministic, so the same output is returned regardless of how often it is run. - -## Z - -### zero knowledge proof - -A proof by which one party (a prover) can prove to another party (a verifier) that they have knowledge of something, without giving away that specific knowledge. Mina uses zero knowledge proofs, specifically zk-SNARKs, to generate a proof attesting to the blockchain's validity and allows any node on the network to verify the validity. - -### zkApps - -Zero knowledge apps ([zkApps](/zkapps/writing-a-zkapp)) are Mina Protocol's smart contracts powered by zero-knowledge proofs, specifically using zk-SNARKs. zkApps provide powerful and unique characteristics such as unlimited off-chain execution, privacy for private data inputs that are never seen by the blockchain, the ability to write smart contracts in TypeScript, and more. The easiest way to write zk programs is using [o1js](https://www.npmjs.com/package/o1js). - -### zkApp CLI - -A command line tool that zkApp developers use to scaffold and deploy smart contracts. Install the [zkApp CLI](https://www.npmjs.com/package/zkapp-cli). - -### zkApp account - -A zkApp account is an account on the Mina blockchain where a zkApp smart contract is deployed. Each zkApp account provides 8 fields of 32 bytes each of arbitrary storage. When a Mina address contains a verification key, it acts as a zkApp account. - -### zkApp manager account - -A specific type of smart contract that manages a particular thing. For example, the zkApp manager account for a token controls all properties of token accounts and determines rules for token minting, burning, and transfer. - -### zkBridge - -Technology that the Mina network uses to connect to other chains. - -### zk-SNARK - -A zero knowledge proof. zk-SNARK is the acronym for zero knowledge succinct non-interactive argument of knowledge. Specific properties of interest in Mina's implementation of SNARKs are succinctness and non-interactivity, which allow for any node to quickly verify the state of the network. SNARK workers create [zk-SNARKs](https://minaprotocol.com/blog/what-are-zk-snarks) for each transaction. zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of a block, and in turn, these zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of the network. - -[A](#a) [B](#b) [C](#c) [D](#d) [E](#e) [F](#f) G [H](#h) [I](#i) J [K](#k) [L](#l) [M](#m) [N](#n) [O](#o) [P](#p) Q [R](#r) [S](#s) [T](#t) [U](#u) [V](#v) W X Y [Z](#z) - ---- -url: /mina-protocol/block-producers ---- - -# Block Producers - -The role of a block producer in Mina is to achieve [consensus](https://minaprotocol.com/blog/what-is-ouroboros-samasika) and provide security to the blockchain. The block producer is responsible for creating new blocks that include recent transactions broadcast on the network and a blockchain proof that proves the current state of the chain is valid. - -In Mina, anyone can become a block producer. There is an unbounded number of participants with the chance of producing a block proportional to the funds staked. Funds are not locked and are not subject to slashing. - -In return for staking funds and generating the required blockchain proofs, blocks that are produced and included in the canonical chain are rewarded in the form of a coinbase and transaction fees, less any fees paid to purchase required [transaction SNARK work](./snark-workers). - -To successfully produce a block, a block producer must have the current state of the blockchain. A block producer must have enough available compute to produce a blockchain SNARK within the slot time and be connected to peers to broadcast the generated block within an acceptable delay as defined by the network consensus parameters. - -### Select a block producer - -The opportunity to produce a block for a slot is determined by a [verifiable random function](/glossary#verifiable-random-function-vrf) (VRF). Think of this function as a lottery. Each block producer independently runs this VRF for each slot and if the output is greater than a threshold proportional to the producer's stake, they have the chance to produce a block at the designated slot. +* [Genesis state hash](#genesis-state-hash) +* [Blockchain state](#blockchain-state) +* [Consensus state](#consensus-state) +* [Consensus constants](#consensus-constants) -This process is secret so that only the private key holder can determine the VRF output and only they know when they are to produce a block. This selection process aids security as it is impossible for an adversary to target a known block producer at a certain slot, e.g., by a denial of service or targeted attack. As a result, multiple producers can be selected for the same slot. When multiple producers produce a valid block for the same slot, a short-range fork is produced where the consensus rules select the longest chain. +Each block contains the protocol state hash of the previous block, such that blocks may be linked together to form an immutable chain. -The stake distribution is determined from the SNARKed ledger at the last block of `current epoch-2`, so there is a delay for any recently acquired or [delegated stake](#stake-delegation). For example, if the current epoch is 10, the staking distribution is determined from the SNARKed ledger of the last block of the 8th epoch. +The protocol state hash is determined from the hash of hashes of the previous state and body and acts as a unique identifier for each block. -To view the output of the VRF in the logs, look for `Checking VRF evaluations`. +#### Genesis State Hash -### Generating a block +The genesis state hash is the protocol state hash for the genesis protocol state. -When a block producer is selected to produce a block for a slot, they perform the following actions: +#### Blockchain State -- Choose the current best tip from their transition frontier (local store of blocks) on which to build the new block. +The blockchain state is comprised of: -- Select transactions and any SNARK work required from the transaction and SNARK pools. +* Staged ledger hash +* Genesis ledger hash +* Ledger proof statement +* Timestamp +* Body reference - A block producer must purchase SNARK work at least in equal quantity to the transactions they add to a block. - In addition to any user transactions, a block producer must also add a coinbase transaction as a reward for producing the block and any fee transfers to pay the SNARK workers. +##### Ledger proof statement -- Generate the proposed next state of the blockchain. +The ledger proof statement is comprised of: - - Create a diff of the staged ledger that includes the account ledger and scan state (a queue of transactions yet to have proofs). - - Apply this diff to the existing staged ledger to produce the new state. +* Snarked ledger hash +* Signed amount +* Pending coinbase stack +* Fee excess +* Sok digest +* Local state -- Create a blockchain proof to prove that the new state is valid. +#### Consensus State - This SNARK additionally validates the prior protocol state proof. +The consensus state is comprised of: -- Create a delta transition chain proof that proves the validity of the block if it is received within an acceptable network delay as defined by the network consensus parameters. +* Blockchain length +* Epoch count +* Min window density +* Sub window density +* Last VRF output +* Total currency +* Current global slot +* Global slot since genesis +* Staking epoch data +* Next epoch data +* Has ancestor in same checkpoint window +* Block stake winner +* Block creator +* Coinbase receiver +* Superchage coinbase -- Apply this newly generated state locally and add it into the existing transition frontier. +#### Consensus Constants -- Broadcast the block (call an external transition) to its peers. +Constants define the consensus parameters: -### Stake delegation +* k +* delta +* slots_per_sub_window +* slots_per_window +* sub_windows_per_window +* slots_per_epoch +* grace period slots +* grace period end +* checkpoint_window_slots_per_year +* checkpoint_window_size_in_slots +* block_window_duration_ms +* slot_duration_ms +* epoch_duration +* delta_duration +* genesis_state_timestamp -Delegated funds are not spendable and can be undelegated at any time by re-delegating the stake back to the original account. +### Protocol State Proof ---- -url: /mina-protocol ---- +The protocol state proof is a blockchain proof proving that the new protocol state generated by the block producer is valid. Due to the use of recursive SNARKs, this protocol state proof proves the entire history of the chain is valid. -# Introduction +### Staged Ledger Diff -The Mina Protocol is a layer one protocol designed to deliver on the original promise of blockchain, true decentralization, scale and security. +When a [block producer](/mina-protocol/block-producers) wins a slot to produce a block, they select transactions and any SNARK work required from the transaction and SNARK pools. They create the proposed next state of the blockchain, which comprises creating a diff of the staged ledger. A diff consists of: -Mina offers an elegant solution: replacing the blockchain with an easily verifiable, consistent-sized cryptographic proof. Mina dramatically reduces the amount of data each user needs to download. Instead of verifying the entire chain from the beginning of time, participants fully verify the network and transactions using recursive zero knowledge proofs (or zk-SNARKs). Nodes can then store the small proof, as opposed to the entire chain. Because it’s a consistent size, Mina stays accessible even as it scales to many users and accumulates years of transaction data. +* Transactions included in the block +* A list of SNARK proofs generated by [SNARK workers](/mina-protocol/snark-workers) for prior transactions added +* Pending coinbase -## The Mina Protocol +A staged ledger can be regarded as a pending accounts database that has transactions(payments, coinbase, and proof fee payments) applied for which there are no SNARKs available yet. A staged ledger consists of the accounts state (a ledger) and a transaction queue for transactions without SNARK proofs, which is the [scan state](/mina-protocol/scan-state). -There are three public Mina Protocol networks: +### Delta Transition Chain Proof -1. `mainnet` - the production network -2. `devnet` - the test network based on the same software versions as the Mainnet -3. `berkeley` - a development network where new features are trialed +There is an allowed network delay when broadcasting or gossiping newly produced blocks around the network to allow for adverse network conditions. The delta transition chain proof proves that the block was produced within the allotted slot time. -You check the identity of the network with this graphQL query: +### Example Block ``` -query MyQuery { - networkID -} -``` - -This section describes how the Mina Protocol works. - -- [Proof Of Stake](/mina-protocol/proof-of-stake) -- [What's in a Block](/mina-protocol/whats-in-a-block) -- [Block Producers](/mina-protocol/block-producers) -- [SNARK Workers](/mina-protocol/snark-workers) -- [Scan State](/mina-protocol/scan-state) -- [Time-Locked Accounts](/mina-protocol/time-locked-accounts) -- [Sending a Payment](/mina-protocol/sending-a-payment) -- [Lifecycle of a Payment](/mina-protocol/lifecycle-of-a-payment) - -## Node Operators - -[Node Operators](../node-operators) describe how to run Mina nodes on a Mina network. Mina nodes fulfill different roles within the network. - -## Node Developers - -[Node Developers](../node-developers) describes how developers can add to and improve Mina nodes. - -## Exchange Operators - -[Exchange Operators](../exchange-operators/faq) describes how to connect to Mina networks and provide access to the MINA token. - ---- -url: /mina-protocol/lifecycle-of-a-payment ---- - -# Lifecycle of a Payment - -In Mina, payments pass through several steps before they are considered verified and complete. - -This document walks through what happens to a single payment in a simplified overview to help you understand how Mina payments work. It it not a comprehensive technical overview, but instead a simplified walkthrough for users. - -## Gossip Protocol - -Mina uses a gossip protocol to ensure that messages can be reliably transmitted to all other members of the network in a timely manner. - -## Payments - -A payment is a type of transaction, requesting to transfer value from one account to another account, and the associated fee the sender is willing to pay for the payment to go through. - -This scenario walks through a scenario where a sender, Bob, wants to send some MINA to a receiver, Alice. - -### Step 1: To create a payment, Bob clicks send - -Any member of the network can create a payment and share it with the Mina network. The payment is cryptographically signed with a private key so that the sender's account can be verified. The payment is then sent out to peers on the network to be processed. The payment, when received by a peer, exists in their local `transaction pool`, which is an in-memory store of all the transactions that peer has heard on the network. - -### Step 2: To produce a block, Bob's payment gets put in a todo list - -A block producer node is chosen on the network for a given time slot. The currently active producer chooses in-flight payments based on payment fees and places them in a list to be processed called a transition block. Block producers earn mina for building these blocks. The producer generates a SNARK defining the structure of the transition block as compared to the previous block (but not yet verifying these new payments). The producer transmits this new information for SNARK workers to process. - -### Step 3: To prove a SNARK transaction, Bob's payment gets SNARK-signed - -SNARK worker nodes on the network begin performing SNARK calculations on each step of the new transition block. These are individual proofs of each payment and then merge proofs of neighboring payments. Eventually, all the payments are verified. SNARK workers can earn currency by generating these proofs, paid for by block producers from their block rewards. These proofs are transmitted out over the network. - -### Step 4: To verify a payment, Alice and Bob's accounts show the result of the transfer - -After the whole block has been proven, the block producer sends out a confirmation of the transition block. Then member nodes on the network apply the changes to their local account balances to reflect these changes. - -### Step 5: To achieve a payment confidence level, Alice is confident the transfer is complete - -With each subsequent block, a recipient has a higher degree of confidence that the payment is actually complete and that the network has consensus about that block. However, like in most blockchains, payments are said to be confirmed after a certain number of blocks, also known as transaction finality. +{ + "external_transition": { + "protocol_state": { + "previous_state_hash": "3NLKJLNbD7rBAbGdjZz3tfNBPYxUJJaLmwCP9jMKR65KSz4RKV6b", + "body": { + "genesis_state_hash": "3NLxYrjb7zmHdoFgBrubCN8ijM8v7eT8kvLiPLc9DHt3M8XrDDEG", + "blockchain_state": { + "staged_ledger_hash": { + "non_snark": { + "ledger_hash": "jxV4SS44wHUVrGEucCsfxLisZyUC5QddsiokGH3kz5xm2hJWZ25", + "aux_hash": "UmosfM82dH5xzqdckXgA1JoAvJ5tLxch2wsty4sXmiEPKnPTPq", + "pending_coinbase_aux": "WLo8mDN6oBUTSyBkFCy7Fky7Na5fN4R6oGq4HMf3YoHCAj4cwY" + }, + "pending_coinbase_hash": "2mze7iXKwA9JAqVDC1MVvgWfJDgvbgSexKtuShdkgqMfv1tjATQQ" + }, + "ledger_proof_statement": { + "connecting_ledger_right": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", + "sok_digest": null, + "target": { + "local_state": { + "full_transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000", + "call_stack": "0x0000000000000000000000000000000000000000000000000000000000000000", + "token_id": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf", + "excess": { + "sgn": [ + "Pos" + ], + "magnitude": "0" + }, + "success": true, + "stack_frame": "0x0641662E94D68EC970D0AFC059D02729BBF4A2CD88C548CCD9FB1E26E570C66C", + "will_succeed": true, + "account_update_index": "0", + "supply_increase": { + "sgn": [ + "Pos" + ], + "magnitude": "0" + }, + "ledger": "jw6bz2wud1N6itRUHZ5ypo3267stk4UgzkiuWtAMPRZo9g4Udyd", + "failure_status_tbl": [], + "transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000" + }, + "pending_coinbase_stack": { + "state": { + "init": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej", + "curr": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej" + }, + "data": "4QNrZFBTDQCPfEZqBZsaPYx8qdaNFv1nebUyCUsQW9QUJqyuD3un" + }, + "first_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", + "second_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S" + }, + "fee_excess": [ + { + "token": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf", + "amount": { + "sgn": [ + "Pos" + ], + "magnitude": "0" + } + }, + { + "amount": { + "sgn": [ + "Pos" + ], + "magnitude": "0" + }, + "token": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf" + } + ], + "source": { + "second_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", + "local_state": { + "account_update_index": "0", + "supply_increase": { + "sgn": [ + "Pos" + ], + "magnitude": "0" + }, + "stack_frame": "0x0641662E94D68EC970D0AFC059D02729BBF4A2CD88C548CCD9FB1E26E570C66C", + "ledger": "jw6bz2wud1N6itRUHZ5ypo3267stk4UgzkiuWtAMPRZo9g4Udyd", + "transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000", + "token_id": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf", + "excess": { + "sgn": [ + "Pos" + ], + "magnitude": "0" + }, + "call_stack": "0x0000000000000000000000000000000000000000000000000000000000000000", + "will_succeed": true, + "full_transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000", + "failure_status_tbl": [], + "success": true + }, + "first_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", + "pending_coinbase_stack": { + "data": "4QNrZFBTDQCPfEZqBZsaPYx8qdaNFv1nebUyCUsQW9QUJqyuD3un", + "state": { + "curr": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej", + "init": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej" + } + } + }, + "supply_increase": { + "sgn": [ + "Pos" + ], + "magnitude": "0" + }, + "connecting_ledger_left": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S" + }, + "body_reference": "b94b2580ca80f27c9655289579a0d71df0b7604dfa7c404e6c309cccf7730d2f", + "snarked_ledger_hash": "jx9171AbMApHNG1guAcKct1E6nyUFweA7M4ZPCjBZpgNNrE21Nj", + "genesis_ledger_hash": "jxX6VJ84HaafrKozFRA4qjnni4aPXqXC2H5vQLKSryNpKTXuz1R", + "snarked_next_available_token": "2", + "timestamp": "1611691710000" + }, + "consensus_state": { + "blockchain_length": "3852", + "epoch_count": "1", + "min_window_density": "1", + "sub_window_densities": [ + "3", + "1", + "3", + "1", + "4", + "2", + "1", + "2", + "2", + "4", + "5" + ], + "last_vrf_output": "g_1vrXSXLhvn1e4Ap1Ey5e8yh3PFMJT0vZyhZLlTBAA=", + "total_currency": "167255800000001000", + "curr_global_slot": { + "slot_number": "12978", + "slots_per_epoch": "7140" + }, + "global_slot_since_genesis": "12978", + "staking_epoch_data": { + "ledger": { + "hash": "jxX6VJ84HaafrKozFRA4qjnni4aPXqXC2H5vQLKSryNpKTXuz1R", + "total_currency": "165950000000001000" + }, + "seed": "2vb1Mjvydod6sEwn7qpbejKCfRqugMgyG3MHXXRKcAkwQLRs9fj8", + "start_checkpoint": "3NK2tkzqqK5spR2sZ7tujjqPksL45M3UUrcA4WhCkeiPtnugyE2x", + "lock_checkpoint": "3NK5G8Xqn1Prh3XoTyZ2tqntJC6X2nVwruv5mEJCL3GaTk7jKUNo", + "epoch_length": "1769" + }, + "next_epoch_data": { + "ledger": { + "hash": "jx7XXjRfJj2mGXmiHQmpm6ZgTxz14udpugyFtw4DefJFpie7apN", + "total_currency": "166537000000001000" + }, + "seed": "2vavBR2GfJWvWkpC7yGJQFnts18nHaFjdVEr84r1Y9DQXvnJRhmd", + "start_checkpoint": "3NLdAqxtBRYxYbCWMXxGu6j1hGDrpQwGkBDF9QvGxmtpziXQDADu", + "lock_checkpoint": "3NL4Eis1pS1yrPdfCbiJcpCCYsHuXY3ZgEzHojPnFWfMK9gKmhZh", + "epoch_length": "2084" + }, + "has_ancestor_in_same_checkpoint_window": true, + "block_stake_winner": "B62qpBrUYW8SHcKTFWLbHKD7d3FqYFvGRBaWRLQCgsr3V9pwsPSd7Ms", + "block_creator": "B62qpBrUYW8SHcKTFWLbHKD7d3FqYFvGRBaWRLQCgsr3V9pwsPSd7Ms", + "coinbase_receiver": "B62qpBrUYW8SHcKTFWLbHKD7d3FqYFvGRBaWRLQCgsr3V9pwsPSd7Ms", + "supercharge_coinbase": true + }, + "constants": { + "k": "290", + "slots_per_epoch": "7140", + "slots_per_sub_window": "7", + "delta": "0", + "genesis_state_timestamp": "1609355670000" + } + } + }, + "protocol_state_proof": "", + "staged_ledger_diff": "", + "delta_transition_chain_proof": "", + "current_protocol_version": "1.1.0", + "proposed_protocol_version": "" + } +} +``` -In the Bitcoin network, a transaction is confirmed after [6 blocks](https://en.bitcoin.it/wiki/Confirmation) (60 mins) with an assumption that an attacker is unlikely to amass more than 10% of the hashrate. +--- +url: /mina-security +--- -With a slot duration of 3 mins and assuming 90% honest stake, the following table shows the finality in blocks, the average time it takes to produce the corresponding number of blocks, and the confidence that payment will be confirmed. +# Mina Security -| Finality (in blocks) | Average time for finality | Finality confidence (%) | -| -------------------- | ------------------------- | ----------------------- | -| 8 | 33 mins | 98.6709 | -| 15 | 60 mins | 99.9231 | -| 23 | 1hr 32mins | 99.9965 | -| 30 | 2hrs | 99.9998 | -| 38 | 2hrs 32mins | 100 | +Mina Protocol is built with ZK from the ground up so anyone can quickly sync and verify the network, exponentially increasing participation, true decentralization, censorship resistance, and network security. -Average time is calculated based on consensus constants that determine the number of slots filled per epoch. This is currently set to 75%. +However it doesn't stop there. Check out some other resources to see what measures are in place to keep Mina secure. -The recommended wait time for a transaction to be confirmed is 15 blocks which provides a 99.9% confidence that the transaction will not be reversed. +## Audits -## Failure Scenarios +### Protocol -Payments can fail for several reasons. +- [August 27, 2024 o1js](https://github.com/o1-labs/o1js/blob/a09c5167c4df64f879684e5af14c59cf7a6fce11/audits/VAR_o1js_240318_o1js_V3.pdf) by Veridise +- [December 12, 2023 Pickles](https://minaprotocol.com/wp-content/uploads/Least-Authority-Pickles-Final-Audit-Report.pdf) by Least Authority +- [August 28, 2023 Transaction Logic and Transaction Pool](https://minaprotocol.com/blog/least-authority-concludes-security-audit-of-mina-protocols-transaction-logic-and-transaction-pool) by Least Authority +- [October 16, 2022 Mina codebase, ecosystem projects](https://minaprotocol.com/wp-content/uploads/Mina-Security-Assessment-2022.pdf) by Mo Ashouri +- [February 22, 2022 Mina Client SDK, Signature Library and Base Components](https://www.nccgroup.com/us/research-blog/public-report-o-1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review/?sq=mina) by NCC Group +- [December 2020 Mina Protocol and Staking Economics](https://gauntlet.network/reports/mina) by Gauntlet Network +- [May 14, 2020 Coda Protocol ](https://minaprotocol.com/blog/ncc-group-security-audit-results-of-coda-protocol) by NCC Group -### Transaction is not accepted by the network +### Tools +- [Feb 4, 2022 Mina Ledger Application](https://minaprotocol.com/blog/ledger-nanox-nanos-developer-mode) by Least Authority +- [Sept 28, 2021 StakingPower Wallet ](https://minaprotocol.com/blog/least-authority-concludes-security-audit-on-stakingpower-wallet) by Least Authority +- [August 9, 2021 Auro Wallet](https://minaprotocol.com/blog/least-authority-concludes-security-audit-on-auro-wallet) by Least Authority +- [July 16, 2021 Clor.io Wallet](https://minaprotocol.com/blog/clorio-wallet-audit) by Least Authority -Several reasons why a transaction shared with peer nodes might not get accepted: +### Auditors +- [Veridise](https://veridise.com/) +- [Least Authority](https://leastauthority.com/) +- [NCC Group](https://www.nccgroup.com/us/) +- [Gauntlet Network](https://www.gauntlet.xyz/) -- The transaction is not fundamentally valid. For example, the sender's account doesn't exist, the account doesn't have sufficient funds, the signature doesn't match with the account, or the nonce in the transaction was not incremented. -- There could be adversarial nodes in the network that collude to deny service to specific senders in the network. However, this behavior is highly disincentivized and one honest node is enough to prevent this issue. -### Transaction is not included in a block +## Current Community Security Programs +- [Bug Bounty Program](https://minaprotocol.com/blog/re-launching-the-mina-ecosystem-bug-bounty-program) +- [Testworld Mission 2.0](https://minaprotocol.com/blog/testworld-2-protocol-performance-testing-program) -If a transaction is valid and the network is honest, then in all likelihood, a transaction will make it into a block. However, there is one case where a transaction can be dumped from a transaction pool: -- If the transaction pool hits its capacity, or `max_txpool_size`, then it evicts the transaction with the lowest fee in the pool, causing it to be dumped from memory. If this happens, the sender needs to resend the transaction with a higher fee, according to market dynamics at the time. +## Learn + +- [Solving for Blockchain’s Security Flaw — Accessible Nodes](https://minaprotocol.com/blog/solving-for-blockchains-security-flaw?utm_medium=gitHub&utm_source=social&utm_campaign=evergreen) +- [What is Ouroboros Samisika?](https://minaprotocol.com/blog/how-ouroboros-samasika-upholds-minas-goals-of-decentralization?utm_medium=github&utm_source=social&utm_campaign=updates) +- [How Ouroboros Samasika Upholds Mina’s Goals of Decentralization](https://minaprotocol.com/blog/how-ouroboros-samasika-upholds-minas-goals-of-decentralization?utm_medium=github&utm_source=social&utm_campaign=updates) +- [Mina’s Mainnet Launch Marks a New Era for Internet Privacy and Data Security](https://minaprotocol.com/blog/minas-mainnet-launch-marks-a-new-era-for-internet-privacy-and-data-security?utm_medium=github&utm_source=social&utm_campaign=updates) --- -url: /mina-protocol/proof-of-stake +url: /mina-signer --- -# Proof of Stake - -The proof of stake consensus mechanism implemented in Mina is a version of the [Ouroboros Praos](https://iohk.io/research/papers/#XJ6MHFXX) protocol, extended and modified slightly for our succinct blockchain. This document will provide an high level overview of how Ouroboros proof of stake works with detailed sections on our changes and additions. For a full description of the Ouroboros protocol, please refer to the original Ouroboros papers: [the original](https://eprint.iacr.org/2016/889.pdf) and [Praos](https://eprint.iacr.org/2017/573.pdf). +# Mina Signer -### A note on Praos, Genesis, and Mina +Mina Signer is a NodeJS/Browser compatible JavaScript library tailored for the Mina Protocol. This library aids developers in seamlessly signing transactions and generating keys. A noteworthy feature is the ability to sign transactions offline, allowing for their broadcasting to the network whenever required. It also supports functionalities such as signing zkApp transactions, verifying these transactions, generating nullifiers, and more. -Formally, our implementation of Ouroboros is an extension of Praos. There is, however, a newer paper which extends Praos, called Ouroboros Genesis. This extension fixes a vulnerability involving long fork attacks, but this cannot be implemented in a succinct blockchain protocol as described in that paper. Instead, we introduce a new, succinct method for protecting against long fork attacks. +## Installation -## Additions to Praos +To incorporate Mina Signer into your project: -### Epoch Ledger Optimization - -In Ouroboros, nodes need to materialize/keep a ledger at the beginning of the previous epoch in order to VRF evaluations. This is because just having a VRF output is not enough to know that a block was proposed honestly. In order to know that a block was won by the node that proposed it, the VRF output must be such that it is underneath a threshold determined by the proposer's stake, proportional to the total currency in the ledger. In a succinct protocol, such as Mina, materializing/keeping such a ledger in the past is not an easy task. Unlike a non succinct blockchain, nodes cannot arbitrarily request pieces of the chain in the past in order to reconstruct the information they are interested in. This means if we were to implement this feature in a naive manner, we would need to keep in total 3 copies of the entire ledger at any point in time, as well as wait online for at least 2 whole epochs before we could access that information. With some thought, though, we can do much better. - -There is a big difference between how Ouroboros proves block winners and how Mina proves them. In Ouroboros, every node needs to validate that a block proposer actually won a block when they receive it, meaning they must look up the balance for the public key that evaluated the VRF themselves. In Mina, however, the correctness of the VRF evaluation can be calculated in the SNARK, so other nodes only need to verify the SNARK to know that the block was proposed by a winner. Since a proposer proves this itself, it can limit the information it needs to store about epoch ledgers to only the account record and the merkle path for any account it can propose for (itself and all of its delegated accounts). Furthermore, since Ouroboros guarantees us that the point of finality will be reached before two epochs, the proposer can wait to capture this information after finalization, further limiting the information it needs to store. This information is then fed into the SNARK, which proves that: 1) the VRF evaluation is accurate for the provided public key, and 2) the account for the public key exists in the epoch ledger with a balance that creates a VRF threshold greater than the VRF output (using the merkle path to prove the epoch ledger's merkle root from the account). This does not immediately address the issue of a proposer node needing to be online long enough in order to store this necessary information, but it does open up other avenues of the proposer acquiring that information. Mainly, the proposer could request an account record and merkle path proving it's existence at a given epoch ledger. In the current implementation, no nodes store this information and make it available for access, but one could imagine in the future a service being built that would allow proposers to get online and active quicker by providing this information, possibly for some sort of fee. - - - ---- -url: /mina-protocol/scan-state ---- +```sh +npm install mina-signer +``` -# Scan State +## Mina Protocol Usage -The scan state is a data structure that allows decoupling the production of transaction SNARKs from block producers to SNARK workers. +Mina Signer offers a wide range of features for the Mina Protocol: -[Block producers](/mina-protocol/block-producers) do not have to produce transaction SNARKs, so the block production time can remain constant regardless of the transaction throughput. The scan state data structure allows the transaction SNARK proof generation to be parallelized and completed by multiple competing [SNARK workers](/mina-protocol/snark-workers). +- Generate keys +- Sign transactions +- Verify transactions -The scan state is comprised of a forest of full [binary trees](https://en.wikipedia.org/wiki/Binary_tree), where each node in the tree is a job to be completed by a SNARK worker. The scan state periodically returns a single proof from the top of a tree that attests to the correctness of all transactions at the base of the tree. +Additionally, it ensures compatibility across various networks, like mainnet and testnet. -The block producers include the emitted ledger proof in the blockchain SNARK they generate that proves both the chain's current state is valid and attests to the validity of all transactions included in the SNARKed ledger. +### Specifying the network -As a result, block times can remain constant regardless of the transaction throughput. The scan state is capable of adjusting to match a desired transaction throughput. +When importing and initializing Mina Signer, it's imperative to designate the desired network. +Different networks may employ varying cryptographic methods. This specification is executed by supplying the network parameter during the constructor's invocation. +Possible values are `mainnet` and `testnet`. :::tip +By default, if no network is explicitly chosen, `mainnet` is the default choice. For the Berkeley network, use `testnet`. +::: -In a steady state, when all slots are filled and all the required proofs are completed, a ledger proof is emitted every block. +```js -::: +const MainnetClient = new Client({ network: 'mainnet' }); // Specify mainnet +const TestnetClient = new Client({ network: 'testnet' }); // Specify testnet (Berkeley) +``` -### Including transactions +### Generating keys -When constructing a block, a [block producer](/mina-protocol/block-producers) can include transactions up to the maximum defined by the [scan state constants](#scan-state-constants). Block producers can pick up any available transaction fees and pay themselves a coinbase reward by including transactions. Each transaction they add is transformed into new base jobs and added to the scan state. +With Mina Signer, generating keypairs is straightforward. -For every transaction added, a block producer must include an equivalent amount of completed SNARK work corresponding to a sequence of jobs already existing in the scan state. When added to the scan state, these completed jobs create new merge jobs, except for the root node, in which case the proof is returned as a result. +```js -The block producer, rather than completing the work themselves, can purchase the completed work from any SNARK workers from bids available in the SNARK pool (snarketplace). +const client = new Client({ network: 'mainnet' }); // Specify mainnet +const keypair = client.genKeys(); // Generates a public and private keypair +``` -### Scan state constants +### Signing & Verifying Transactions -The following constants dictate the structure and behavior of the scan state: +Mina Signer facilitates both transaction and stake delegation signing and verification. To sign a transaction, the sender's private must be provided. +Conversely, for verification, the sender's public key must be provided. Post-signing, the Mina Daemon can be utilized to broadcast the payment or delegation. -- `transaction_capacity_log_2` -- `work_delay` +#### Payments -The `transaction_capacity_log_2` constant defines the maximum number of transactions that can be included in a block: +Payments are transactions that transfer funds from one account to another. To sign a payment, the following parameters must be provided: -``` -max_no_of_transactions = 2^{transaction_capacity_log_2} -``` +```js -The work delay ensures there is enough time for the SNARK work to be completed by the SNARK workers. The block producer cannot include any transactions if no completed proofs are available. With the work delay, the maximum number of trees that can exist in the scan state is defined by: +const client = new Client({ network: 'mainnet' }); +const keypair = client.genKeys(); -``` -max_number_of_trees = (transaction_capacity_log_2 + 1) * (work_delay + 1) + 1 +const payment = client.signPayment( + { + to: keypair.publicKey, // Public key of the recipient + from: keypair.publicKey, // Public key of the sender + amount: '1', // Amount to be sent (in nano MINA) + fee: '1', // Fee to be paid (in nano MINA) + nonce: '0', // Nonce of the sender + }, + keypair.privateKey +); + +const verifiedPayment = client.verifyPayment(payment); ``` -The maximum number of proofs that can be included per block is defined by: +#### Delegations -``` -max_number_of_proofs = 2^{transaction\_capacity_log_2 + 1} - 1 -``` +Stake delegations are a way for users to delegate their stake to a validator. This allows the validator to produce blocks on behalf of the delegator. To sign a stake delegation, the following parameters must be provided: -These scan state constraints ensure that: +```js -- Only a single proof can be emitted per block -- The merge node to be updated after adding proofs corresponding to its children is always empty. +const client = new Client({ network: 'mainnet' }); +const keypair = client.genKeys(); -While the maximum number of transactions can be fixed, this number can dynamically adjust to the transaction throughput. As such, the scan state can handle an unlimited transaction throughput, albeit at the cost of increasing (logarithmically) the transaction proof latency. +const delegation = client.signStakeDelegation( + { + to: keypair.publicKey, // Public key of the validator + from: keypair.publicKey, // Public key of the delegator + fee: '1', // Fee to be paid (in nano MINA) + nonce: '0', // Nonce of the delegator + }, + keypair.privateKey +); -### Example +const verifiedDelegation = client.verifyStakeDelegation(delegation); +``` -Consider a scan state with `max_no_of_transactions = 4`, and `work_delay = 1`. Accordingly, this means there can be a maximum amount of work to complete equal to 7 and a maximum of 7 trees. At **genesis**, the scan state is empty. +#### Generic Signing -Block 0 +Mina Signer can accept a generic payload and determine the most apt signing approach via `signTransaction()`. +This functionality is especially beneficial for applications that support different types of transactions. -**Block 1**: A block producer includes four transactions into the scan state labeled `B1`. These transactions fill the base of the first tree. +```js -Block 1 +const client = new Client({ network: 'mainnet' }); +const keypair = client.genKeys(); -**Block 2**: At the second block, a block producer adds another four transactions (`B2`). These are added to a second tree, once again filling the base. There are no proofs required due to the work delay of 1 block. +// Sign a payment +client.signTransaction( + { + to: keypair.publicKey, + from: keypair.publicKey, + amount: '1', + fee: '1', + nonce: '0', + }, + keypair.privateKey +); -Block 2 +// Sign a delegation +client.signTransaction( + { + to: keypair.publicKey, + from: keypair.publicKey, + fee: '1', + nonce: '0', + }, + keypair.privateKey +); -**Block 3**: At the third block, a block producer adds four `B3` transactions to the third tree but must include four proofs for the first tree. As a result of including these completed base proofs, two new `M3` merge jobs are created. -Block 3 +// Sign a zkApp transaction +client.signTransaction( + { + zkappCommand: ..., + feePayer: ... + }, + keypair.privateKey +); -:::tip +// Sign a simple string payload +client.signTransaction('Hello World', keypair.privateKey); +``` -`B` or `M` indicates a base or merge job, with the number indicating the sequence order of being added to the scan state. +### Broadcasting a Signed Payment -::: +After signing a payment, you can broadcast it to the network via a Mina Node GraphQL endpoint: -**Block 4**: For the fourth block, a block producer adds another four transactions (`B4`) to the base of the fourth tree. They must include four proofs corresponding to the work added in block 2. Again, two `M4` merge jobs are created as a result. +```javascript -Block 4 -:::tip +const client = new Client({ network: 'mainnet' }); -Any pending work (displayed in orange) is work for the SNARK workers to complete. The SNARK workers submit completed work to the SNARK pool. Multiple SNARK workers can complete the work, but only the lowest fee remains in the SNARK pool that can be purchased by the block producers. +const senderPrivateKey = 'EKFd1Gx...'; +const senderPublicKey = 'B62qrDM...'; -::: +let payment = { + from: senderPublicKey, + to: 'B62qkBw...', + amount: 100, + nonce: 1, + fee: 1000000, +}; -**Block 5**: In the fifth block, another four transactions are included to fill the base of tree five (`B5`), and six proofs must be included (`B3`s and `M3`s). The `M3` merge jobs result in a final pending merge job for the first tree (`M5`). +const signedPayment = client.signPayment(payment, senderPrivateKey); -Block 5 +const url = 'https://your-mina-node/graphql'; -**Block 6**: In the sixth block, another four transactions (`B6`) are added, filling the base of the sixth tree. Six proofs are included (`B4` and `M4`), and three new merge jobs are created (`M6`). +const sendPaymentMutationQuery = ` +mutation SendPayment($input: SendPaymentInput!, $signature: SignatureInput!) { + sendPayment(input: $input, signature: $signature) { + payment { + hash + } + } +} +`; +const graphQlVariables = { + input: signedPayment.data, + signature: signedPayment.signature, +}; +const body = JSON.stringify({ + query: sendPaymentMutationQuery, + variables: graphQlVariables, + operationName: 'SendPayment', +}); -Block 6 +const paymentResponse = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body, +}); -**Block 7**: In the seventh block, the block producer adds a further four transactions (`B7`), filling the base of the seventh tree. Seven trees are the maximum number of trees according to the specified scan state constants. The maximum number of proofs (7) are included (`B5` and `M5`). These included proofs create three new merge jobs (`M7`);additionally, the top `M5` proof is emitted from the scan state. +const paymentResponseJson = await paymentResponse.json(); +if (paymentResponse.ok) { + console.log( + `Transaction hash: ${paymentResponseJson.data.sendPayment.payment.hash}` + ); +} else { + console.error(JSON.stringify(paymentResponseJson)); +} +``` -Block 7 +### Payment & Delegation Transaction Hashes -The proof that is emitted from the first tree is the ledger proof corresponding to the transactions added in block 1. The contents of the tree are then removed to create space for additional transactions. +In addition to signing/verifying payments/delegations for the Mina Protocol, Mina Signer allows you to compute the hash that will be used to identify the transaction on the blockchain. This is useful for applications that require the transaction hash before the transaction is broadcasted to the network. -Emit proof +```js -**Block 8**: In the eighth block, the block producer adds two transactions (`B8`) and includes 4 (`B6`) proofs. These included proofs result in two new merge jobs (`M8`). Note that only four proofs are required for adding two transactions. +const client = new Client({ network: 'mainnet' }); +const keypair = client.genKeys(); -Block 8 - -:::tip - -SNARK work is bundled into a work package typically containing two _workIds_, except for the final root proof of a tree. Prorated work for a transaction is two proofs, ensuring the equality of transactions included and SNARK work to be purchased. - -::: - -**Block 9**: The block producer adds three transactions (`B9`) in the ninth block. Three proofs (`M6`) are required to occupy the slots in the currently unfilled tree. Four proofs were added in the previous block, so only three more proofs need to be done (given the maximum work is 7). The `M6` proof from tree two is returned as the ledger proof. The third `B9` transaction goes into the now empty tree, and two `B7` proofs are added. - -Block 9 - -**Block 10**: In block ten, the block producer adds four transactions and, as a result, includes seven proofs (`B7`, `M7`, and two `B8`s). - -Block 10 +const payment = client.signTransaction( + { + to: keypair.publicKey, + from: keypair.publicKey, + amount: '1', + fee: '1', + nonce: '0', + }, + keypair.privateKey +); +const hashedPayment = client.hashPayment(payment); -**Block 11**: In the eleventh block, the block producer adds three transactions (`B11`) and completes five proofs (`B9`, `B9`, `M8`, `M8`, `M9`) in that order. In addition, the `M9` ledger proof is returned from the fourth tree. +const delegation = client.signTransaction( + { + to: keypair.publicKey, + from: keypair.publicKey, + fee: '1', + nonce: '0', + }, + keypair.privateKey +); +const hashedDelegation = client.hashStakeDelegation(delegation); +``` -Block 11 +### Rosetta Integration -:::tip +For those developing with [Rosetta](https://www.rosetta-api.org/), Mina Signer provides an avenue to transform a signed Rosetta transaction into a Mina-compliant transaction, ready for broadcasting through the Mina Daemon. -To view the contents of the scan state, run the `mina advanced snark-job-list` command. +```js -::: +const client = new Client({ network: 'mainnet' }); -### Integration with the SNARK Pool +const signedRosettaTx = '...'; +const signedGraphQLCommand = + client.signedRosettaTransactionToSignedCommand(signedRosettaTx); +``` -Newly added jobs to the scan state are pending jobs for SNARK workers to complete. SNARK workers complete the required transaction SNARKs, submitting bids for their completed work. When a node receives and validates the completed work, SNARK workers add the completed work to the local SNARK pool if it is valid and has the lowest fee for the required work. The work is also gossiped to other peers in the network. +For detailed Rosetta usage including the offline signer CLI tool and `signRosettaTransaction`, see the [Mina Signer for Node Operators](/node-operators/mina-signer) documentation. -:::tip +## o1js Integration -While multiple SNARK workers can complete the same work, only the lowest fee is included in the SNARK pool. +Mina Signer can seamlessly integrate with [o1js](/zkapps/o1js), delivering an array of features for zkApps like: -::: +- zkApp transaction signing and verification +- Field payload signing and verification +- Nullifier generation -When a block producer includes completed proofs into a block to offset any transactions they add, they may purchase the corresponding work from the SNARK pool. Continuing the previous example, consider the next block (12). If the block producer wants to add three transactions, comprising a coinbase, a user payment, and a fee transfer to the SNARK worker, the block producer must purchase three completed SNARK works. This corresponds to the six `B9`, `B10`s, `M9`, and `M10` (from the seventh tree) proofs, as each SNARK work includes two _workIds_. +### Signing & Verifying zkApp transactions -During the time the block is generated, the SNARK pool can include completed work and the best bids for the required jobs (0.025, 0.165, 0.1, and 0.5) respectively, in the example. +Mina Signer supports signing and verifying zkApp transactions. o1js itself can be used to sign zkApp transactions, +but Mina Signer offers the ability to sign a zkApp transaction that can easily be broadcasted with a Mina Daemon. This can be very useful for wallet applications that want to support zkApps. -Submitted work +```js -A block producer considers the price of available work before selecting transactions. -- The first transaction a block producer adds is the coinbase transaction for which there is the coinbase reward. -- If transaction fees do not cover the SNARK work fees required for them to be included, the transaction is not added. -A block producer never purchases work if it is not economical. +const client = new Client({ network: 'testnet' }); +const keypair = client.genKeys(); -If completed SNARK work is not available to purchase in the order required, then the corresponding transactions are not included in a block. This situation can result in an empty block, but also, for the case where no transactions can be added (including a coinbase transaction), there is no reward for the block producer. +const zkAppTransaction = await Mina.transaction(feePayerAddress, () => { + // ... Interact with a zkApp inside this block to produce a zkApp transaction +}); -To view the current SNARK pool, use: +// Sign the zkApp transaction with Mina Signer +const signedZkAppTransaction = client.signZkappCommand( + { + zkappCommand: JSON.parse(JSON.stringify(txn.transaction)), + feePayer: { + feePayer: keypair.publicKey, + fee: '1', + nonce: '0', + memo: 'memo', + }, + }, + keypair.privateKey +); -- [GraphQL API](/node-developers/graphql-api) -- Mina CLI `mina advanced snark-pool` command +// Verify the zkApp transaction with Mina Signer +const verifiedZkAppTransaction = client.verifyZkappCommand( + signedZkAppTransaction +); +``` ---- -url: /mina-protocol/sending-a-payment ---- +Firstly, when supplying the input parameters for `signZkappCommand()`, we must first parse the zkApp transaction into a string and then into a JSON object. This is because the types generated from `Mina.transaction()` are not compatible with the types used by Mina Signer. +Secondly, we specify the `feePayer` object which contains the public key of the fee payer, the fee to be paid, the nonce of the fee payer, and the memo of the transaction. The `feePayer` object is used to sign the zkApp transaction. -# Sending a Payment +:::tip +Use o1js to sign zkApp transactions if you can, as it's more ergonomic and easier to use. Only use `Mina Signer` if you need to sign zkApp transactions offline and broadcast at a later time (e.g. wallet software). +::: +### Signing/Verifying Field payloads +Mina Signer can sign and validate Field payloads. This is invaluable when ensuring a Field payload's authenticity, as it confirms the payload remains untampered by external parties. -How to send a MINA payment using the Mina CLI. +```js -:::info +const client = new Client({ network: 'testnet' }); +const keypair = client.genKeys(); -Reminder that this section is intended for node operators. If you want to store, send, and receive MINA without running a node, please see the [Install a Wallet](../using-mina/install-a-wallet) page for links to various user-friendly wallets available for Mina. +const fields = [10n, 20n, 30n, 340817401n, 2091283n, 1n, 0n]; +const signedFields = client.signFields(fields, keypair.privateKey); +const verifiedFields = client.verifyFields(signedFields); +``` -::: +If you are using o1js to generate Field payloads, you must convert the Fields to BigInts before signing/verifying them. +In Mina Signer, the Field type is a BigInt (while in o1js they are a separate data structure), so you must convert the fields from o1js to BigInts before signing/verifying them. -In this section, we'll give a brief overview on how to send a transaction with the Mina client and how to get started with interacting with the blockchain. +```js -## Using an offline signed-transaction -If you want to send a transaction without running a node yourself, but by delegating to someone else running a node, keep following along here. If you wish to send the transaction directly with a running node, skip to [using a connected node](#using-a-connected-node). +const client = new Client({ network: 'testnet' }); +const keypair = client.genKeys(); -### Using a Ledger device +const fields = [Field(10), Field(20)].map((f) => f.toBigInt()); +const signedFields = client.signFields(fields, keypair.privateKey); +const verifiedFields = client.verifyFields(signedFields); +``` -To generate a signed transaction offline if your private key is on a Ledger device, see [Ledger Hardware Wallet](/using-mina/ledger-hardware-wallet). +### Nullifiers -### Using a keypair generated with the generate-keypair tool +Mina Signer supports generating nullifiers for zkApp transactions. In the world of cryptography, nullifiers play a pivotal role. +They stand as unique markers, maintaining anonymity yet ensuring account reliability, and staving off illicit undertakings like double-spends. +To generate a nullifier, provide a message (an array of BigInts) and the sender's private key. -A better tool is coming soon: https://github.com/MinaProtocol/mina/issues/8928. For now, please use the [workaround](https://github.com/MinaProtocol/mina/issues/8928#issuecomment-857095846) provided in a comment on that issue. +```js -### Using a keypair generated with the offline client-sdk +const client = new Client({ network: 'testnet' }); +const keypair = client.genKeys(); -Use the [Mina Signer](/mina-signer). See the [Mina Signer documentation](/mina-signer) for installation and usage instructions. +const message = [10n, 20n, 30n, 340817401n, 2091283n, 1n, 0n]; +const nullifier = client.createNullifier(message, keypair.privateKey); +``` -### Send the transaction +--- +url: /network-upgrades/berkeley/appendix +--- -You can use a hosted service to broadcast your signed transaction. Sending your signed transaction _does not_ leak your private key. +# Appendix -Transactions signed with the Mina Signer can use: [https://minascan.io/mainnet/broadcast/payment](https://minascan.io/mainnet/broadcast/payment) +## Migration from o1labs/client-sdk to mina-signer -Transactions signed with the Ledger hardware wallet can use: [https://minascan.io/mainnet/broadcast/ledger-payment](https://minascan.io/mainnet/broadcast/ledger-payment) +The signing library `o1labs/client-sdk` was deprecated some time ago and will stop working after the Mina mainnet upgrade. All users should upgrade to use the [mina-signer](https://www.npmjs.com/package/mina-signer) library. -## Using a connected node +Below you will find an example of how to use the `mina-signer` library. Please keep in mind the following: -We are assuming in the rest of section that you have the Mina client installed on your system, if you do not have Mina installed please see the [Getting Started](/node-operators/block-producer-node/getting-started). +1. Make sure to adjust the `nonce` to the correct nonce on the account you want to use as "sender" +1. Update the `url` variable with an existing Mina Node GraphQL endpoint -## Import your account +```javascript -Once our node is synced, we'll need to import our public/private keypair so that we can sign transactions and generate an address to receive payments. For security reasons, we'll want to put the keys under a directory that is harder for attackers to access. -Run the following command to import your [previously generated](/node-operators/validator-node/generating-a-keypair) keypair file: +// create the client and define the keypair - mina accounts import --privkey-path ~/keys/my-wallet +const client = new Client({ network: 'testnet' }); // Mind the `network` client configuration option -You will be prompted for the password you entered when the account was created. +const senderPrivateKey = 'EKFd1Gx...'; // Sender's private key +const senderPublicKey = 'B62qrDM...'; // Sender's public key, perhaps derived from the private key using `client.derivePublicKey(senderPrivateKey)`; -:::caution +// define and sign payment -The public key can be shared freely with anyone, but be very careful with your private key file. Never share this private key with anyone, as it is the equivalent of a password for your funds. +let payment = { + from: senderPublicKey, + to: 'B62qkBw...', // Recipient public key + amount: 100, + nonce: 1, + fee: 1000000, +}; -::: +const signedPayment = client.signPayment(payment, senderPrivateKey); -The response from this command will look like this: +// send payment to graphql endpoint - 😄 Imported account! - Public key: B62qjaA4N9843FKM5FZk1HmeuDiojG42cbCDyZeUDQVjycULte9PFkC +const url = 'https://qanet.minaprotocol.network/graphql'; -Additionally you can use the `mina accounts create` command to generate new accounts to send and receive transactions. +const sendPaymentMutationQuery = ` +mutation SendPayment($input: SendPaymentInput!, $signature: SignatureInput!) { + sendPayment(input: $input, signature: $signature) { + payment { + hash + } + } +} +`; +const graphQlVariables = { + input: signedPayment.data, + signature: signedPayment.signature, +}; +const body = JSON.stringify({ + query: sendPaymentMutationQuery, + variables: graphQlVariables, + operationName: 'SendPayment', +}); -Since the public key is quite long and difficult to remember, let's save it as an environment variable. Use the following command but replace `` with the public key output from the previous command: +const paymentResponse = await fetch(url, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body +}); - export MINA_PUBLIC_KEY=`` +const paymentResponseJson = await paymentResponse.json(); +if (paymentResponse.ok) { + console.log(`Transaction hash: ${paymentResponseJson.data.sendPayment.payment.hash}`); +} else { + console.error(JSON.stringify(paymentResponseJson)); +} -Now we can access this everywhere as `$MINA_PUBLIC_KEY` -- check if it saved properly by trying `echo $MINA_PUBLIC_KEY`. -Note that these environment variables will only be saved for the current shell session, so if you want to save them for future use, you can add them to `~/.profile` or `~/.bash_profile`. +``` -:::tip +--- +url: /network-upgrades/berkeley/archive-migration/appendix +--- -If you are running the node on a cloud virtual machine, make sure to export and save the key file. You can export the key with: +# Appendix - mina accounts export --public-key `` --privkey-path `` +## Archive node schema changes -Then save it to your local machine, maybe using [scp](https://linux.die.net/man/1/scp): +If you are using the Archive Node database directly for your system integrations, then you should understand all the changes that might impact your applications. The most important change is that the `balances` table in the Berkeley schema will no longer exist. In the new schema, it is replaced with the table `accounts_accessed` - from an application semantics point of view, the data in `accounts_accessed` is still the same. - scp `` `` +In the Berkeley protocol, accounts can now have the same public key but a different token_id. This means accounts are identified by both their public key and token_id, not just the public key. Consequently, the foreign key for the account in all tables is account_identifier_id instead of public_key_id. -Later, when starting up a new VM, you can upload the key and then import it: +### Schema differences +- **Removed Types** + - The options `create_token`, `create_account`, and `mint_tokens` have been removed from the user_command_type enumeration. +- Indexes Dropped + - We've removed several indexes from tables, this may affect how you search and organize data: + - `idx_public_keys_id` + - `idx_public_keys_value` + - `idx_snarked_ledger_hashes_value` + - `idx_blocks_id` + - `idx_blocks_state_hash` +- **Table Removed** + - The `balances` table is no longer available. +- **New Tables Added** + - We've introduced the following new tables: + - `tokens` + - `token_symbols` + - `account_identifiers` + - `voting_for` + - `protocol_versions` + - `accounts_accessed` + - `accounts_created` + - `zkapp_commands` + - `blocks_zkapp_commands` + - `zkapp_field` + - `zkapp_field_array` + - `zkapp_states_nullable` + - `zkapp_states` + - `zkapp_action_states` + - `zkapp_events` + - `zkapp_verification_key_hashes` + - `zkapp_verification_keys` + - `zkapp_permissions` + - `zkapp_timing_info` + - `zkapp_uris` + - `zkapp_updates` + - `zkapp_balance_bounds` + - `zkapp_nonce_bounds` + - `zkapp_account_precondition` + - `zkapp_accounts` + - `zkapp_token_id_bounds` + - `zkapp_length_bounds` + - `zkapp_amount_bounds` + - `zkapp_global_slot_bounds` + - `zkapp_epoch_ledger` + - `zkapp_epoch_data` + - `zkapp_network_precondition` + - `zkapp_fee_payer_body` + - `zkapp_account_update_body` + - `zkapp_account_update` + - `zkapp_account_update_failures` +- **Updated Tables** + - The following tables have been updated + - `timing_info` + - `user_commands` + - `internal_commands` + - `epoch_data` + - `blocks` + - `blocks_user_commands` + - `blocks_internal_commands` - mina accounts import --privkey-path `` +### Differences per table +- **`timing_info`** + - Removed columns: + - `token` + - `initial_balance` +- **`user_commands`** + - Removed columns: + - `fee_token` + - `token` +- **`internal_commands`** + - Removed columns: + - `token` + - Renamed column + - `command_type` to `type` +- **`epoch_data`** + - Added columns: + - `total_currency` + - `start_checkpoint` + - `lock_checkpoint` + - `epoch_length` +- **`blocks`** + - Added columns: + - `last_vrf_output` + - `min_window_density` + - `sub_window_densities` + - `total_currency` + - `global_slot_since_hard_fork` + - `global_slot_since_genesis` + - `protocol_version_id` + - `proposed_protocol_version_id` + - Removed column: + - `global_slot` +- **`blocks_user_commands`** + - Removed columns: + - `fee_payer_account_creation_fee_paid` + - `receiver_account_creation_fee_paid` + - `created_token` + - `fee_payer_balance` + - `source_balance` + - `receiver_balance` + - Added index: + - `idx_blocks_user_commands_sequence_no` +- **`blocks_internal_commands`** + - Removed columns: + - `receiver_account_creation_fee_paid` + - `receiver_balance` + - Added indexes: + - `idx_blocks_internal_commands_sequence_no` + - `idx_blocks_internal_commands_secondary_sequence_no` -::: +### Rosetta API new operations -If you ever forget what keypairs you've already created, you can see them all with: +The Berkeley upgrade introduces two new operation types: +- `zkapp_fee_payer_dec` +- `zkapp_balance_change` - mina accounts list +--- +url: /network-upgrades/berkeley/archive-migration/archive-migration-installation +--- -## Check account balance +The archive node Berkeley migration package is sufficient for satisfying the migration from Devnet/Mainnet to Berkeley. +However, it has some limitations. For example, the migration package does not migrate a non-canonical chain and it skips orphaned blocks that are not part of a canonical chain. -We can check the balance of all our accounts using this command: +To mitigate these limitations, the archive node maintenance package is available for use by archive node operators who want to maintain a copy of their Devnet and Mainnet databases for historical reasons. - mina accounts list +## Install with Google Cloud SDK -You might see `Balance: 0 mina` for your account. Depending on the traffic in the network, it may take a few blocks before your transaction goes through. +The Google Cloud SDK installer does not always register a `google-cloud-sdk` apt package. The best way to install gsutil is using the apt repostory: -:::tip +```sh +curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg +echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list +sudo apt-get update && sudo apt-get install google-cloud-sdk +``` -You can run `mina client status` to see the current block height updating. +## Download the o1labs Mainnet archive database -::: +We strongly encourage you to perform the migration on your own data to preserve the benefits of decentralization. However, if you want to use the archive data that o1labs runs (for example, to bootstrap a new archive from SQL without waiting all day for the chain to download and replay), you can use the following steps: -## Make a payment +1. Download the Devnet/Mainnet archive data using cURL or gsutil: -Finally, we get to the good stuff–sending our first transaction! Before you send a payment, you'll need to unlock your account: + - cURL: - mina accounts unlock --public-key $MINA_PUBLIC_KEY + For Devnet: + ```sh + curl https://storage.googleapis.com/mina-archive-dumps/devnet-archive-dump-{date}_0000.sql.tar.gz + ``` -For testing purposes, we will specify your public key as the receiver and sender. This just means that we are sending a transaction to ourselves, you can see your public key by issuing the following command: + For Mainnet: + ```sh + curl https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-{date}_0000.sql.tar.gz + ``` -``` -echo $MINA_PUBLIC_KEY -``` + To filter the dumps by date, replace `{date}` using the required `yyyy-dd-mm` format. For example, for March 15, 2024, use `2024-03-15`. -:::caution + :warning: The majority of backups have the `0000` suffix. If a download with that name suffix is not available, try incrementing it. For example, `0001`, `0002`, and so on. -If the receiving account has not received any transactions, there will be an additional Account Creation Fee of `1 MINA` that will be deducted from the transaction amount. + - gsutil: -::: + ```sh + gsutil cp gs://mina-archive-dumps/mainnet-archive-dump-2024-01-15* . + ``` -Let's send some of our Mina to ourselves to see what a payment looks like: +2. Extract the tar package. - mina client send-payment \ - --amount 1.5 \ - --receiver $MINA_PUBLIC_KEY \ - --fee 0.1 \ - --sender $MINA_PUBLIC_KEY + ```sh + tar -xvzf {network}-archive-dump-{date}_0000.sql.tar.gz {network}-archive-dump-{date}_0000.sql + ``` -If you're wondering what we passed in to the commands above: +3. Import the Devnet/Mainnet archive dump into the Berkeley database. -- For `amount`, we're sending a test value of `1.5` mina which is enough to cover the Account Creation Fee -- The `receiver` is the public key of the account receiving the transaction, eg. `B62qjaA4N9843FKM5FZk...` -- For `fee`, let's use 0.1 mina -- The `sender` is the public key of the account sending the transaction, eg. `B62qjaA4N9843FKM5FZk...` + Run this command at the database server: -If this command is formatted properly, we should get a response that looks like the following: + ```sh + psql -U {user} -f {network}-archive-dump-{date}_0000.sql + ``` - Dispatched payment with ID 3XCgvAHLAqz9VVbU7an7f2L5ffJtZoFega7jZpVJrPCYA4j5HEmUAx51BCeMc232eBWVz6q9t62Kp2cNvQZoNCSGqJ1rrJpXFqMN6NQe7x987sAC2Sd6wu9Vbs9xSr8g1AkjJoB65v3suPsaCcvvCjyUvUs8c3eVRucH4doa2onGj41pjxT53y5ZkmGaPmPnpWzdJt4YJBnDRW1GcJeyqj61GKWcvvrV6KcGD25VEeHQBfhGppZc7ewVwi3vcUQR7QFFs15bMwA4oZDEfzSbnr1ECoiZGy61m5LX7afwFaviyUwjphtrzoPbQ2QAZ2w2ypnVUrcJ9oUT4y4dvDJ5vkUDazRdGxjAA6Cz86bJqqgfMHdMFqpkmLxCdLbj2Nq3Ar2VpPVvfn2kdKoxwmAGqWCiVhqYbTvHkyZSc4n3siGTEpTGAK9usPnBnqLi53Z2bPPaJ3PuZTMgmdZYrRv4UPxztRtmyBz2HdQSnH8vbxurLkyxK6yEwS23JSZWToccM83sx2hAAABNynBVuxagL8aNZF99k3LKX6E581uSVSw5DAJ2S198DvZHXD53QvjcDGpvB9jYUpofkk1aPvtW7QZkcofBYruePM7kCHjKvbDXSw2CV5brHVv5ZBV9DuUcuFHfcYAA2TVuDtFeNLBjxDumiBASgaLvcdzGiFvSqqnzmS9MBXxYybQcmmz1WuKZHjgqph99XVEapwTsYfZGi1T8ApahcWc5EX9 - Receipt chain hash is now A3gpLyBJGvcpMXny2DsHjvE5GaNFn2bbpLLQqTCHuY3Nd7sqy8vDbM6qHTwHt8tcfqqBkd36LuV4CC6hVH6YsmRqRp4Lzx77WnN9gnRX7ceeXdCQUVB7B2uMo3oCYxfdpU5Q2f2KzJQ46 + The database in the dump **archive_balances_migrated** is created with the Devnet/Mainnet archive schema. -You may not see the `Receipt chain hash` on the first transaction from the account, but in following transactions, this will show you the head of the receipt chain hash list. + Note: This database does not have any Berkeley changes. -## Staking and Snarking +## Ensure the location of Google Cloud bucket with the Devnet/Mainnet precomputed blocks -Once you feel comfortable with the basics of creating an address, and sending & receiving mina, we can move on to the truly unique parts of the Mina network like [participating in consensus and helping compress the blockchain](/node-operators/validator-node/staking-and-snarking). +The recommended method is to perform migration on your own data to preserve the benefits of decentralization. -## Advanced +`gsutil cp gs://mina_network_block_data/{network}-*.json .` -### Sending Many Transactions +:warning: Precomputed blocks for the Mainnet network take ~800 GB of disk space. Plan for adequate time to download these blocks. The Berkeley migration app downloads them incrementally only when needed. -Sometimes you may wish to send many transactions: for example, to payout rewards to those delegating to you if you're running a staking pool. +## Validate the Devnet/Mainnet database -All information here is relevant as of the 3.0.3 build: +The correct Devnet/Mainnet database state is crucial for a successful migration. -### Rate limiting +[Missing blocks](/network-upgrades/berkeley/archive-migration/mainnet-database-maintenance#missing-blocks) is one the most frequent issues when dealing with the Devnet/Mainnet archive. Although this step is optional, it is strongly recommended that you verify the archive condition before you start the migration process. -Currently, nodes on the network will rate limit receiving messages from a given node. As of the 3.0.3 build, your node will also follow this rate limit when sending transactions. Specifically, the limit is currently set at 10 transactions every 15 seconds computed over a 5 minute window. If you attempt to send transactions faster than this rate, your node will queue them up and flush them as older transactions expire from the window upon which the rate limit is computed. You do not need to throttle sending these transactions yourself. +To learn how to maintain archive data, see [Devnet/Mainnet database maintenance](/network-upgrades/berkeley/archive-migration/mainnet-database-maintenance). -Note that older releases of the mina daemon do not perform this rate limiting; if you are running an older version, you should manually limit the number of transactions. Due to overheads from rebroadcasting transactions, we do not recommend sending more than 50 transactions every 5 minutes if you need to manually rate limit. +## Download the migration applications -### My node crashed or disconnected before I could finish sending transactions +Migration applications are distributed as part of the archive migration Docker and Debian packages. -The Mina daemon does _not_ currently persist the transaction pool. This means that the transactions that your node will be unaware of any transactions that you've sent so far if your node crashes in the middle of this process. As of the 3.0.3 build, you can resend all transactions (exactly in the same manner as before) and they will be rebroadcasted on the network. +Choose the packages that are appropriate for your environment. -If you believe you were temporarily disconnected from the network, but your node stayed online (i.e. the gossip network may have missed one or more of your transactions), as of the 3.0.3 build, you can resend any of the transactions locally and they will be broadcasted again to the network even if your node thinks they've already been shared. +### Debian packages -### Cancelling a transaction and setting a new fee +To get the Debian packages: -To cancel a transaction, you'll need to have all the transactions that haven't been committed to the chain yet before in your local transaction mempool. This means if your node crashed (see above) you'll need to resend those earlier transactions. Finally, to cancel a transaction, all you need to do is send a transaction with the same nonce of the one you want to cancel with a larger fee. There is no minimum increment, it just needs to be slightly larger (and large enough such that a block producer will choose your transaction). +``` +CODENAME=bullseye +CHANNEL=stable +VERSION=3.0.1-e848ecb ---- -url: /mina-protocol/snark-workers ---- +echo "deb [trusted=yes] http://packages.o1test.net $CODENAME $CHANNEL" | tee /etc/apt/sources.list.d/mina.list +apt-get update +apt-get install --allow-downgrades -y "mina-archive-migration=$VERSION" +``` -# SNARK Workers +### Docker image -While most protocols have just one primary group of node operators (often called miners, validators, or block producers), Mina has a second group — the **SNARK worker**. +To get the Docker image: -SNARK workers are integral to the Mina network's health because these nodes are responsible for snarking, or producing SNARK proofs, of transactions in the network. By producing these proofs, snark workers help maintain the succinctness of the Mina blockchain. +``` +docker pull gcr.io/o1labs-192920/mina-archive-migration:3.0.1-e848ecb-{codename} +``` -Read on to learn why SNARK workers are needed, how the economic incentives align, and operational details of performing SNARK work. Feel free to click through to any of the sections that are most relevant to your needs. +Where supported codenames are: +- bullseye +- focal +- buster -Note: The theory of zk-SNARKs is not covered. Deep knowledge of SNARKs is not required to read this section, but it is helpful to understand in general how SNARKs work and what they are useful for. To learn more, check out this - [What are zk-SNARKs?](https://minaprotocol.com/blog/what-are-zk-snarks) primer first. -## How Mina Compresses the Blockchain +## Devnet/Mainnet genesis ledger -The Mina protocol is unique because nodes are not required to maintain the full history of the blockchain like other cryptocurrency protocols. By recursively using cryptographic proofs, the Mina protocol effectively compresses the blockchain to constant size. This compression reduces terabytes of data to a few kilobytes. +The Mina Devnet/Mainnet genesis ledger is stored in GitHub in the `mina` repository under the `genesis_ledgers` subfolder. However, if you are already running a daemon that is connected to the Mina Mainnet or the Devnet network, you already have the genesis ledger locally. -However, this isn't data encoding or compression in the traditional sense. Mina nodes _compress_ data in the network by generating cryptographic proofs. Node operators play a crucial role in this process by designating themselves as [SNARK workers](/glossary#snark-worker) that generate [zk-SNARKs](/glossary#zk-snark) for transactions that have been added to blocks. +## Berkeley database schema files -You can connect a SNARK worker to the network or use a SNARK coordinator. [SNARK coordinators](/glossary#snark-coordinator) can be used to coordinate and distribute work to any [SNARK workers](/glossary#snark-worker) that you are running. If you are running a SNARK coordinator, connect the SNARK coordinator to the network, then connect SNARK workers to the SNARK coordinator. +You can get the Berkeley schema files from different locations: -## Why SNARK Workers? +- GitHub repository from the `berkeley` branch. -Mina's unique property is the succinct blockchain. Each block producer, when they propose a new block to the network, must also include a zk-SNARK along with that block. This allows nodes to discard all historical data that's been finalized, and retain just the SNARK. If you are unfamiliar with the Mina protocol, [this video is a good start](https://www.youtube.com/watch?v=eWVGATxEB6M). + Note: The `berkeley` branch can contain new updates regarding schema files, so always get the latest schema files instead of using an already downloaded schema. -However, it is not only sufficient for the block producers in Mina to generate SNARK proofs of blocks. Transactions also need to be SNARKed. The reason is because the blockchain SNARK does not make any statements of the validity of the transactions included in the block. +- Archive/Rosetta Docker from `berkeley` version -For example — let's say the current head of the blockchain has a state hash `a6f8792226...` , and we receive a new block with a state hash `0ffdcf284f...` . This block will contain all the transactions that the block producer has chosen to include in this block, and associated metadata. We will also receive an accompanying SNARK that verifies the statement: +### Example: Downloading schema sources from GitHub -> "There exists a block with a state hash 0ffdcf284f which extends the blockchain -> with a previous best tip with state hash a6f8792226." + ```sh + wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql -Notice that this statement says nothing about the validity of the transactions included in the new block. If we were to believe this SNARK, and do nothing else, we may be tricked by a malicious block producer sending this block. Luckily, we have the raw block and we can check each transaction to ensure it is valid. But what about nodes in the network that may just want to receive the proof and not verify each block? + wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql + ``` -## Snarking Transactions +## Next steps -In order to ensure that nodes can operate without trust on the Mina blockchain, it is important that each node can verify the state of the chain without needing to replay the transactions. In order for this to work, the blockchain SNARK is not enough. We need to know that the transactions are also valid. Well, since SNARKs are good for exactly that, the naive suggestion might be to generate a SNARK of each transaction as they come in, and then combine them. +Congratulations on completing the essential preparation and verification steps. You are now ready to perform the migration steps in [Migrating Devnet/Mainnet Archive to Berkeley Archive](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley). -However, generating SNARK proofs is computationally expensive — if we had to compute SNARKs serially for each transaction, throughput would be very low and block times would skyrocket. Furthermore, transactions in a real world environment arrive asynchronously, so it would be very tough to predict when to perform the next item of work. +--- +url: /network-upgrades/berkeley/archive-migration/archive-migration-prerequisites +--- -Lucky for us, we can leverage two properties about SNARKs: +To successfully migrate the archive database into the Berkeley version of the Mina network, you must ensure that your environment meets the foundational requirements. -1. proofs can be merged - two proofs can be combined to form a _merge proof_ -2. merges are associative - merge proofs are identical, regardless of the order merged +## Migration host - - - - SNARK Workers - +- PostgreSQL database for database server +- If you use Docker, then any of the supported OS by Mina (bullseye, focal, or buster) with at least 32 GB of RAM +- gsutil application from Google Cloud Suite in version 5 or later +- (Optional) Docker in version 23.0 or later -What these two properties essentially allow us to do is take advantage of parallelism. If proofs can be merged, and it doesn't matter how they're combined, then SNARK proofs can be generated in parallel. Whichever proof is finished first can be combined later with the proofs in progress. This can be envisioned as a binary tree, where the bottom row (the leaves) consists of the individual transaction proofs, and each parent row, the set of respective merge proofs. We can combine these all the way to the root, which represents a state update performed by applying all the transactions. +## (Optional) Devnet/Mainnet database -In addition, because the SNARK proofs don't depend on each other and we can exploit parallelism, this means anyone can do the work! The end result is that the distributed work pool is permission-less. Anyone with spare compute can join the network as SNARK workers, observe transactions that need to be SNARKed, and contribute their compute. And of course, they will be compensated for their work in what we affectionately call **the snarketplace**. +One of the most obvious prerequisites is a Mainnet database. -Note: To learn more about the details of how this SNARK work scheme evolved, it is highly recommended to watch this video: [High Throughput with Slow Snarks](https://www.youtube.com/watch?v=NZmq1V-Te0E). If you're interested in functional programming and the details of the scan state (the tree structure described above), we have [a video](https://www.youtube.com/watch?v=ztH_Z5TCe9I) covering the technical details. +If you don't have an existing database with Devnet/Mainnet archive data, you can always download it from the Google Cloud bucket. However, we strongly encourage you to perform migration on your own data to preserve the benefits of decentralization. +You can use any gsutil-compatible alternative to Google Cloud or a gsutil wrapper program. -## The Snarketplace +## (Optional) Google Cloud bucket with Devnet/Mainnet precomputed blocks -The key dynamic to understand about SNARK work is: +Precomputed blocks are the JSON files that a correctly configured node updloads to the Google Cloud bucket. +The Devnet/Mainnet to Berkeley archive data migration requires access to precomputed blocks that are uploaded by daemons that are connected to the Devnet or Mainnet networks. -_Block producers use their block rewards to purchase SNARK work from SNARK workers._ +The **berkeley-migration** app uses the gsutil app to download blocks. If you didn't store precomputed blocks during the first phase of migration, you can use the precomputed blocks provided by Mina Foundation. +However, it is strongly recommended that you perform migration on your own data to preserve the benefits of decentralization. -As a SNARK worker, you get to share some of the block rewards for each block that includes your compressed transactions. The block producer is responsible for gathering compressed transactions before including them into a block and is incentivized by the protocol to reward SNARK workers. +For Devnet blocks: -There is no protocol involvement in pricing snarks, nor are there any protocol level rewards for SNARK workers to produce snarks. The incentives are purely peer-to-peer, and dynamically established in a public marketplace, aka the snarketplace. +```sh +gsutil cp gs://mina_network_block_data/devnet-*.json . +``` -You may ask, why does a block producer need to buy SNARKs? Fair question — the reason is because of what we mentioned earlier. In order to know for sure the state at the head of the Mina blockchain is valid, the transactions need to be SNARKed. But if we keep adding more transactions without snarking them at an equal rate, then over time we accumulate work that never gets finished. In order to reach a steady state equilibrium, we need work to be processed at roughly the same rate that work is added. +For Mainnet blocks: -Since block producers profit from including transactions in a block (through transaction fees and the coinbase transaction), they are responsible for offsetting the transactions by purchasing an equal number of completed SNARK work, thereby creating demand for SNARK work. However, their imperative is to purchase SNARK work for the lowest price from the snarketplace. Conversely, the SNARK workers want to maximize their profit while also being able to sell their SNARK work. These two roles act as the two sides of the marketplace, and over time establish an equilibrium at a market price for SNARK work. +```sh +gsutil cp gs://mina_network_block_data/mainnet-*.json . +``` -### How to price SNARK work +:warning: Precomputed blocks for the Mainnet network take ~800 GB of disk space. Plan for adequate time to download these blocks. The Berkeley migration app downloads them incrementally only when needed. You can instead download a 100 GB bundle of only the canonical Mainnet blocks that unpacks into ~220 GB: -We anticipate the snarketplace to dynamically rebalance — eg. follow the simple laws of [supply and demand](https://en.wikipedia.org/wiki/Supply_and_demand). While each SNARK work applies to a different transaction, seen from a larger perspective, SNARK work is largely a commodity (meaning it doesn't matter which SNARK worker produces the good — it will be the same). However, there are some nuances, so it may help to have some heuristics for pricing strategy: +```sh +gsutil cp gs://mina_network_block_data/mainnet-bundle-2024-03-20.tar.zst . ; tar -xf mainnet-bundle-2024-03-20.tar.zst +``` -- if market price is X, it is likely effective to sell SNARK work for any price below X (eg. X - 1), provided it is profitable after operating expenses. -- block producers are incentivized to purchase more units of SNARK work from the same SNARK worker because there will only be one _fee transfer_ transaction they have to include in the block. - - Basically, the way a block producer pays a SNARK worker is through a special type of transaction called a fee transfer. The BP's incentive is to minimize the number of fee transfers, as each is a discrete transaction that needs to be added to a block (and consequently offset by more SNARK work). Thus, the best case scenario is to buy a bundle of SNARK work from the same SNARK worker. -- some SNARK work will be more important to complete ahead of other work, as it would free up an entire tree worth of memory (see the video above for more details). This is made possible by different work selection methods. Currently, the three methods supported natively are sequential, random and sequential with a random offset. Neither of these however takes advantage of dynamic markets, which is an area of improvement that the Mina community can develop solutions for. +:warning: Precomputed blocks for the Devnet network take several hundred GBs. Plan for adequate time to download these blocks. Instead, you can download a ~50 GB bundle of only the canonical Devnet blocks that unpacks into ~90 GB: -Since all the data around snarks and prices are public, there are several ways to inspect the snarketplace. One example is [using the GraphQL API](https://youtu.be/XQlfX-LnK_A), and other options include using the CLI, or rolling a custom solution that tracks snarks in the SNARK mempool. +```sh +gsutil cp gs://mina_network_block_data/devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz . ; tar -xf devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz +``` -Stay tuned for more detailed analysis on snarketplace dynamics. We will also be releasing an economic whitepaper shortly that will provide more context. +These bundles are partial. Updated documentation with the new links and final data will be provided _after_ the Berkeley major upgrade is completed. -See also: [SNARKs and SNARK Workers FAQ](/node-operators/faq#snarks-and-snark-workers) +The best practice is to collect precomputed blocks by yourself or by other third parties to preserve the benefits of decentralization. --- -url: /mina-protocol/time-locked-accounts +url: /network-upgrades/berkeley/archive-migration/debian-example --- -# Time-Locked Accounts +# Debian example -A time-locked account disallows payments that would reduce the balance below a minimum, which depends on the block height. +You can follow these steps that can be copy-pasted directly into a fresh Debian 11. -To create a time-lock, you must provide the configuration when creating a new account. This can happen only in the genesis ledger at the beginning of a network. In this section, we'll explore the mechanism behind time-locks and see how to interact with time-locked accounts. +This example uses an altered two-step version of the [full simplified workflow](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley#simplified-approach). -:::tip +```sh +apt update && apt install lsb-release sudo postgresql curl wget gpg # debian:11 is surprisingly light - For the current release, values for time-locked accounts were assigned based on the order in which you signed up. +curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg +echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list +sudo apt-get update && sudo apt-get install google-cloud-sdk -::: +sudo rm /etc/apt/sources.list.d/mina*.list +sudo echo "deb [trusted=yes] http://packages.o1test.net $(lsb_release -cs) unstable" | sudo tee /etc/apt/sources.list.d/mina.list +sudo apt-get update && sudo apt-get install --allow-downgrades -y mina-archive-migration=3.0.0-rc1-4277e73 -## Understanding time-locks +mkdir -p mina-migration-workdir +cd mina-migration-workdir -A time-lock consists of the following fields `initial_minimum_balance`, `cliff` time, a `vesting_period` time, and a `vesting_increment`. +gsutil cp gs://mina_network_block_data/devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz . +tar -xf devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz -You can still use an account if it has a time-lock, as long as the account holds enough funds. The amount of funds that are time-locked starts off as `initial_minimum_balance` at the beginning of the network. Once the network reaches a block height equal to the `cliff`, the time-locked amount begins to decrease by the `vesting_increment` amount every `vesting_period`. +wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql +wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql -For a more technical explanaition of this process, please see [RFC-0025](https://github.com/MinaProtocol/mina/blob/master/rfcs/0025-time-locked-accounts.md) which has a more in-depth overview. +# this next step is required only if you don't have an archive yet +createdb devnet_balances_migrated +createdb devnet_really_migrated -### Liquid Balance Details: +psql -d devnet_really_migrated -f create_schema.sql -If you'd like to expose liquid balances for vesting accounts at some particular time period it is governed by the following function (Note: this computes the locked portion of an account): +gsutil cp gs://mina-archive-dumps/devnet-archive-dump-2024-03-22_0000.sql.tar.gz . +tar -xf devnet-archive-dump-2024-03-22_0000.sql.tar.gz +# the next step ensures you don't accidentally merge mainnet and devnet data +sed -i -e s/archive_balances_migrated/devnet_balances_migrated/g devnet-archive-dump-2024-03-22_0000.sql +psql -d devnet_balances_migrated -f devnet-archive-dump-2024-03-22_0000.sql -``` -(* - * uint32 global_slot -- the "clock" it starts at 0 at the genesis block and ticks up every 3minutes. - * uint32 cliff_time -- the slot where the cliff is (similar to startup equity vesting) - * uint32 cliff_amount -- the amount that unlocks at the cliff - * amount vesting_increment -- unlock this amount every "period" - * uint32 vesting_period -- the period that we increment the unlocked amount - * balance initial_minimum_balance -- the total locked amount until the cliff - *) -let min_balance_at_slot ~global_slot ~cliff_time ~cliff_amount ~vesting_period - ~vesting_increment ~initial_minimum_balance = - let open Unsigned in - if Global_slot.(global_slot < cliff_time) then initial_minimum_balance - else - match Balance.(initial_minimum_balance - cliff_amount) with - | None -> - Balance.zero - | Some min_balance_past_cliff -> ( - (* take advantage of fact that global slots are uint32's *) - let num_periods = - UInt32.( - Infix.((global_slot - cliff_time) / vesting_period) - |> to_int64 |> UInt64.of_int64) - in - let vesting_decrement = - UInt64.Infix.(num_periods * Amount.to_uint64 vesting_increment) - |> Amount.of_uint64 - in - match Balance.(min_balance_past_cliff - vesting_decrement) with - | None -> - Balance.zero - | Some amt -> - amt ) +mina-berkeley-migration-script initial \ + --genesis-ledger /var/lib/coda/devnet.json \ + --source-db postgres:///devnet_balances_migrated \ + --target-db postgres:///devnet_really_migrated \ + --blocks-batch-size 100 --blocks-bucket mina_network_block_data \ + --network devnet -``` +# now, do a final migration -## Creating a time-locked account +gsutil cp gs://mina-archive-dumps/devnet-archive-dump-2024-03-22_2050.sql.tar.gz . +tar -xf devnet-archive-dump-2024-03-22_2050.sql.tar.gz +# the next step ensures you don't accidentally merge mainnet and devnet data +sed -i -e s/archive_balances_migrated/devnet_balances_migrated/g devnet-archive-dump-2024-03-22_2050.sql +psql -d devnet_balances_migrated -f devnet-archive-dump-2024-03-22_2050.sql -As of the current release, the only way to create a time-locked account is with the genesis ledger. In future releases we may add commands to `mina client` and the GraphQL API that will allow you to create a new time-locked account. +curl -O https://gist.githubusercontent.com/ghost-not-in-the-shell/cfe629a15702e7bae7b0c1415fe0d85e/raw/8d8bff2814c1d0c15deb70b388dea8a28a485184/genesis.json + +mina-berkeley-migration-script final \ + --genesis-ledger /var/lib/coda/devnet.json \ + --source-db postgres:///devnet_balances_migrated \ + --target-db postgres:///devnet_really_migrated \ + --blocks-batch-size 100 --blocks-bucket mina_network_block_data \ + --network devnet \ + --replayer-checkpoint migration-checkpoint-437195.json \ + --fork-state-hash 3NKoUJX87VrfmNAoUdqoWUykVvt66ztm5rzruDQR7ihwYaWsdJKq \ + --fork-config genesis.json \ + --prefetch-blocks +``` --- -url: /mina-protocol/whats-in-a-block +url: /network-upgrades/berkeley/archive-migration/docker-example --- -# What's in a Block? +# Docker example -A block is a set of transactions and consensus information that extend the state of the network. A block in Mina includes a proof that the current state of the network is fully valid. +You can follow these steps that can be copy-pasted directly into a OS running Docker. -A block in Mina is constituted of: +This example performs a Mainnet initial migration following the [debian-example](/network-upgrades/berkeley/archive-migration/debian-example) -* [Protocol state](#protocol-state) -* [Protocol state proof](#protocol-state-proof) -* [Staged ledger diff](#staged-ledger-diff) -* [Delta transition chain proof](#delta-transition-chain-proof) -* Current protocol version -* Proposed protocol version +```sh -When a node receives a block from a peer, it is first validated, applied to the existing state, and added to the node's transition frontier. If, according to the [consensus](https://minaprotocol.com/blog/what-is-ouroboros-samasika) rules, it results in increasing the length of the blockchain, the node's best tip is updated, and the root of the transition frontier is moved up to only maintain `k` blocks in the transition frontier. +# Create a new directory for the migration data +mkdir $(pwd)/mainnet-migration && cd $(pwd)/mainnet-migration -:::tip +# Create Network +docker network create mainnet -In Mina, blocks are synonymous with "transitions". When this transition (block) is received from a peer, it is referenced as an external transition, whereas one generated and applied locally is referred to as an internal transition. +# Launch Local Postgres Database +docker run --name postgres -d -p 5432:5432 --network mainnet -v $(pwd)/mainnet-migration/postgresql/data:/var/lib/postgresql/data -e POSTGRES_USER=mina -e POSTGRES_PASSWORD=minamina -d postgres:13-bullseye -::: +export PGHOST="localhost" +export PGPORT=5432 +export PGUSER="mina" +export PGPASSWORD="minamina" -### Protocol State +# Drop DBs if they exist +psql -c "DROP DATABASE IF EXISTS mainnet_balances_migrated;" +psql -c "DROP DATABASE IF EXISTS mainnet_really_migrated;" -The protocol state is comprised of the **previous protocol state hash** and a body that contains: +# Create DBs +psql -c "CREATE DATABASE mainnet_balances_migrated;" +psql -c "CREATE DATABASE mainnet_really_migrated;" -* [Genesis state hash](#genesis-state-hash) -* [Blockchain state](#blockchain-state) -* [Consensus state](#consensus-state) -* [Consensus constants](#consensus-constants) +# Retrieve Archive Node Backup +wget https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-2024-04-29_0000.sql.tar.gz +tar -xf mainnet-archive-dump-2024-04-29_0000.sql.tar.gz -Each block contains the protocol state hash of the previous block, such that blocks may be linked together to form an immutable chain. +# Replace the database name in the dump +sed -i -e s/archive_balances_migrated/mainnet_balances_migrated/g mainnet-archive-dump-2024-04-29_0000.sql +psql mainnet_balances_migrated -f mainnet-archive-dump-2024-04-29_0000.sql -The protocol state hash is determined from the hash of hashes of the previous state and body and acts as a unique identifier for each block. +# Prepare target +wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql +wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql +psql mainnet_really_migrated -f create_schema.sql -#### Genesis State Hash +# Start migration +docker create --name mainnet-db-migration \ + -v $(pwd)/mainnet-migration:/data \ + --network mainnet minaprotocol/mina-archive-migration:3.0.1-e848ecb-bullseye -- bash -c ' + wget http://673156464838-mina-genesis-ledgers.s3-website-us-west-2.amazonaws.com/mainnet/genesis_ledger.json; mina-berkeley-migration-script initial \ + --genesis-ledger genesis_ledger.json \ + --source-db postgres://mina:minamina@postgres:5432/mainnet_balances_migrated \ + --target-db postgres://mina:minamina@postgres:5432/mainnet_really_migrated \ + --blocks-batch-size 5000 \ + --blocks-bucket mina_network_block_data \ + --checkpoint-output-path /data/checkpoints/. \ + --precomputed-blocks-local-path /data/precomputed_blocks/. \ + --network mainnet' -The genesis state hash is the protocol state hash for the genesis protocol state. +docker start mainnet-db-migration -#### Blockchain State +docker logs -f mainnet-db-migration -The blockchain state is comprised of: +``` -* Staged ledger hash -* Genesis ledger hash -* Ledger proof statement -* Timestamp -* Body reference +--- +url: /network-upgrades/berkeley/archive-migration +--- -##### Ledger proof statement +# Archive Migration -The ledger proof statement is comprised of: +The Berkeley upgrade is a major upgrade that requires all nodes in a network to upgrade to a newer version. It is not backward compatible. -* Snarked ledger hash -* Signed amount -* Pending coinbase stack -* Fee excess -* Sok digest -* Local state +A major upgrade occurs when there are major changes to the core protocol that require all nodes on the network to update to the latest software. -#### Consensus State +## How to prepare for the Berkeley upgrade -The consensus state is comprised of: +The Berkeley upgrade requires upgrading all nodes, including archive nodes. One of the required steps is to migrate archive databases from the current Mainnet format to Berkeley. This migration requires actions and efforts from node operators and exchanges. -* Blockchain length -* Epoch count -* Min window density -* Sub window density -* Last VRF output -* Total currency -* Current global slot -* Global slot since genesis -* Staking epoch data -* Next epoch data -* Has ancestor in same checkpoint window -* Block stake winner -* Block creator -* Coinbase receiver -* Superchage coinbase +Learn about the archive data migration: -#### Consensus Constants +- [Understanding the migration process](/network-upgrades/berkeley/archive-migration/understanding-archive-migration) +- [Prerequisites before migration](/network-upgrades/berkeley/archive-migration/archive-migration-prerequisites) +- [Suggested installation procedure](/network-upgrades/berkeley/archive-migration/archive-migration-installation) +- [How to perform archive migration](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley) -Constants define the consensus parameters: +Finally, see the shell script example that is compatible with a stock Debian 11 container: -* k -* delta -* slots_per_sub_window -* slots_per_window -* sub_windows_per_window -* slots_per_epoch -* grace period slots -* grace period end -* checkpoint_window_slots_per_year -* checkpoint_window_size_in_slots -* block_window_duration_ms -* slot_duration_ms -* epoch_duration -* delta_duration -* genesis_state_timestamp +- [Worked Devnet Debian example using March 22 data](/network-upgrades/berkeley/archive-migration/debian-example) +- [Worked Mainnet Docker example using April 29 data](/network-upgrades/berkeley/archive-migration/docker-example) -### Protocol State Proof +## What will happen with original Devnet/Mainnet data -The protocol state proof is a blockchain proof proving that the new protocol state generated by the block producer is valid. Due to the use of recursive SNARKs, this protocol state proof proves the entire history of the chain is valid. +After the migration, you will have two databases: -### Staged Ledger Diff +- The original Devnet/Mainnet database with small data adjustments (all pending blocks from last canoncial block until the fork block are converted to canoncial blocks) +- A new Berkeley database based on Devnet/Mainnet data, but: + - Without Devnet/Mainnet orphaned blocks + - Without pending blocks that are not in the canonical chain + - With all pending blocks on the canonical chain converted to canonical blocks -When a [block producer](/mina-protocol/block-producers) wins a slot to produce a block, they select transactions and any SNARK work required from the transaction and SNARK pools. They create the proposed next state of the blockchain, which comprises creating a diff of the staged ledger. A diff consists of: +There is no requirement to preserve the original Devnet/Mainnet database after migration. However, if for some reason you want to keep the Mainnet orphaned or non-canonical pending blocks, you can download the archive maintenance package for the Devnet/Mainnet database. -* Transactions included in the block -* A list of SNARK proofs generated by [SNARK workers](/mina-protocol/snark-workers) for prior transactions added -* Pending coinbase +To learn about maintaining archive data, see [Devnet/Mainnet database maintenance](/network-upgrades/berkeley/archive-migration/mainnet-database-maintenance). -A staged ledger can be regarded as a pending accounts database that has transactions(payments, coinbase, and proof fee payments) applied for which there are no SNARKs available yet. A staged ledger consists of the accounts state (a ledger) and a transaction queue for transactions without SNARK proofs, which is the [scan state](/mina-protocol/scan-state). +--- +url: /network-upgrades/berkeley/archive-migration/mainnet-database-maintenance +--- -### Delta Transition Chain Proof +# Devnet/Mainnet database maintenance -There is an allowed network delay when broadcasting or gossiping newly produced blocks around the network to allow for adverse network conditions. The delta transition chain proof proves that the block was produced within the allotted slot time. +After the Berkeley migration, the original Devnet/Mainnet database is not required unless you are interested in +preserving some aspect of the database that is lost during the migration process. -### Example Block +Two databases exist after the successful migration: + +- The original Devnet/Mainnet database with small data adjustments: + - All pending blocks from last canoncial block until the fork block are converted to canonical blocks + +- A new Berkeley database based on Devnet/Mainnet data with these differences: + - Without Devnet/Mainnet orphaned blocks + - Without pending blocks that are not in the canonical chain + - With all pending blocks on the canonical chain converted to canonical blocks + +The o1Labs and Mina Foundation teams have consistently prioritized rigorous testing and the delivery of high-quality software products. + +However, being human entails the possibility of making mistakes. + +## Known issues + +Recently, a few mistakes were identified while working on a version of Mina used on Mainnet. These issues were promptly addressed; however, within the decentralized environment, archive nodes can retain historical issues despite our best efforts. + +Fixes are available for the following known issues: + +- **Missing or invalid nonces** - a historical issue skewed nonces in the `balances` table. Although the issue was resolved, you might still have nonces that are missing or invalid. +- **Incorrect ledger hashes** - a historical issue with the same root cause as 'Missing or invalid nonces'. However, the outcome is that a 'replayer run' operation of validating archive node against daemon ledger shows ledger mismatches and cannot pass problematic blocks. +- **Missing blocks** - This recurring missing blocks issue consistently poses challenges and is a source of concern for all archive node operators. This persistent challenge from disruptions in daemon node operations can potentially lead to incomplete block reception by archive nodes. This situation can compromise chain continuity within the archive database. + +To address these issues, install and use the special archive node maintenance package that includes fixes. + +## Installing the archive node maintenance package + +The package provides support for codenames: + +- bullseye +- buster +- focal + +The following steps describe only the bullseye package installation. Modify the steps as appropriate for your environment. + +### Debian packages + +To get the Debian package: + +```sh +CODENAME=bullseye +CHANNEL=stable +VERSION=1.4.1 -``` -{ - "external_transition": { - "protocol_state": { - "previous_state_hash": "3NLKJLNbD7rBAbGdjZz3tfNBPYxUJJaLmwCP9jMKR65KSz4RKV6b", - "body": { - "genesis_state_hash": "3NLxYrjb7zmHdoFgBrubCN8ijM8v7eT8kvLiPLc9DHt3M8XrDDEG", - "blockchain_state": { - "staged_ledger_hash": { - "non_snark": { - "ledger_hash": "jxV4SS44wHUVrGEucCsfxLisZyUC5QddsiokGH3kz5xm2hJWZ25", - "aux_hash": "UmosfM82dH5xzqdckXgA1JoAvJ5tLxch2wsty4sXmiEPKnPTPq", - "pending_coinbase_aux": "WLo8mDN6oBUTSyBkFCy7Fky7Na5fN4R6oGq4HMf3YoHCAj4cwY" - }, - "pending_coinbase_hash": "2mze7iXKwA9JAqVDC1MVvgWfJDgvbgSexKtuShdkgqMfv1tjATQQ" - }, - "ledger_proof_statement": { - "connecting_ledger_right": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", - "sok_digest": null, - "target": { - "local_state": { - "full_transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000", - "call_stack": "0x0000000000000000000000000000000000000000000000000000000000000000", - "token_id": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf", - "excess": { - "sgn": [ - "Pos" - ], - "magnitude": "0" - }, - "success": true, - "stack_frame": "0x0641662E94D68EC970D0AFC059D02729BBF4A2CD88C548CCD9FB1E26E570C66C", - "will_succeed": true, - "account_update_index": "0", - "supply_increase": { - "sgn": [ - "Pos" - ], - "magnitude": "0" - }, - "ledger": "jw6bz2wud1N6itRUHZ5ypo3267stk4UgzkiuWtAMPRZo9g4Udyd", - "failure_status_tbl": [], - "transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000" - }, - "pending_coinbase_stack": { - "state": { - "init": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej", - "curr": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej" - }, - "data": "4QNrZFBTDQCPfEZqBZsaPYx8qdaNFv1nebUyCUsQW9QUJqyuD3un" - }, - "first_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", - "second_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S" - }, - "fee_excess": [ - { - "token": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf", - "amount": { - "sgn": [ - "Pos" - ], - "magnitude": "0" - } - }, - { - "amount": { - "sgn": [ - "Pos" - ], - "magnitude": "0" - }, - "token": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf" - } - ], - "source": { - "second_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", - "local_state": { - "account_update_index": "0", - "supply_increase": { - "sgn": [ - "Pos" - ], - "magnitude": "0" - }, - "stack_frame": "0x0641662E94D68EC970D0AFC059D02729BBF4A2CD88C548CCD9FB1E26E570C66C", - "ledger": "jw6bz2wud1N6itRUHZ5ypo3267stk4UgzkiuWtAMPRZo9g4Udyd", - "transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000", - "token_id": "wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf", - "excess": { - "sgn": [ - "Pos" - ], - "magnitude": "0" - }, - "call_stack": "0x0000000000000000000000000000000000000000000000000000000000000000", - "will_succeed": true, - "full_transaction_commitment": "0x0000000000000000000000000000000000000000000000000000000000000000", - "failure_status_tbl": [], - "success": true - }, - "first_pass_ledger": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S", - "pending_coinbase_stack": { - "data": "4QNrZFBTDQCPfEZqBZsaPYx8qdaNFv1nebUyCUsQW9QUJqyuD3un", - "state": { - "curr": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej", - "init": "4Yx5U3t3EYQycZ91yj4478bHkLwGkhDHnPbCY9TxgUk69SQityej" - } - } - }, - "supply_increase": { - "sgn": [ - "Pos" - ], - "magnitude": "0" - }, - "connecting_ledger_left": "jwbhUympjiAFLPi7w3Cg9GEeNCfACEpPxrTp45XZt3BDA5UNV8S" - }, - "body_reference": "b94b2580ca80f27c9655289579a0d71df0b7604dfa7c404e6c309cccf7730d2f", - "snarked_ledger_hash": "jx9171AbMApHNG1guAcKct1E6nyUFweA7M4ZPCjBZpgNNrE21Nj", - "genesis_ledger_hash": "jxX6VJ84HaafrKozFRA4qjnni4aPXqXC2H5vQLKSryNpKTXuz1R", - "snarked_next_available_token": "2", - "timestamp": "1611691710000" - }, - "consensus_state": { - "blockchain_length": "3852", - "epoch_count": "1", - "min_window_density": "1", - "sub_window_densities": [ - "3", - "1", - "3", - "1", - "4", - "2", - "1", - "2", - "2", - "4", - "5" - ], - "last_vrf_output": "g_1vrXSXLhvn1e4Ap1Ey5e8yh3PFMJT0vZyhZLlTBAA=", - "total_currency": "167255800000001000", - "curr_global_slot": { - "slot_number": "12978", - "slots_per_epoch": "7140" - }, - "global_slot_since_genesis": "12978", - "staking_epoch_data": { - "ledger": { - "hash": "jxX6VJ84HaafrKozFRA4qjnni4aPXqXC2H5vQLKSryNpKTXuz1R", - "total_currency": "165950000000001000" - }, - "seed": "2vb1Mjvydod6sEwn7qpbejKCfRqugMgyG3MHXXRKcAkwQLRs9fj8", - "start_checkpoint": "3NK2tkzqqK5spR2sZ7tujjqPksL45M3UUrcA4WhCkeiPtnugyE2x", - "lock_checkpoint": "3NK5G8Xqn1Prh3XoTyZ2tqntJC6X2nVwruv5mEJCL3GaTk7jKUNo", - "epoch_length": "1769" - }, - "next_epoch_data": { - "ledger": { - "hash": "jx7XXjRfJj2mGXmiHQmpm6ZgTxz14udpugyFtw4DefJFpie7apN", - "total_currency": "166537000000001000" - }, - "seed": "2vavBR2GfJWvWkpC7yGJQFnts18nHaFjdVEr84r1Y9DQXvnJRhmd", - "start_checkpoint": "3NLdAqxtBRYxYbCWMXxGu6j1hGDrpQwGkBDF9QvGxmtpziXQDADu", - "lock_checkpoint": "3NL4Eis1pS1yrPdfCbiJcpCCYsHuXY3ZgEzHojPnFWfMK9gKmhZh", - "epoch_length": "2084" - }, - "has_ancestor_in_same_checkpoint_window": true, - "block_stake_winner": "B62qpBrUYW8SHcKTFWLbHKD7d3FqYFvGRBaWRLQCgsr3V9pwsPSd7Ms", - "block_creator": "B62qpBrUYW8SHcKTFWLbHKD7d3FqYFvGRBaWRLQCgsr3V9pwsPSd7Ms", - "coinbase_receiver": "B62qpBrUYW8SHcKTFWLbHKD7d3FqYFvGRBaWRLQCgsr3V9pwsPSd7Ms", - "supercharge_coinbase": true - }, - "constants": { - "k": "290", - "slots_per_epoch": "7140", - "slots_per_sub_window": "7", - "delta": "0", - "genesis_state_timestamp": "1609355670000" - } - } - }, - "protocol_state_proof": "", - "staged_ledger_diff": "", - "delta_transition_chain_proof": "", - "current_protocol_version": "1.1.0", - "proposed_protocol_version": "" - } -} +echo "deb [trusted=yes] http://packages.o1test.net $CODENAME $CHANNEL" | tee /etc/apt/sources.list.d/mina.list +apt-get update +apt-get install --allow-downgrades -y "mina-archive-maintenance=$VERSION" ``` ---- -url: /mina-security ---- +### Docker image -# Mina Security +To get the Docker image: -Mina Protocol is built with ZK from the ground up so anyone can quickly sync and verify the network, exponentially increasing participation, true decentralization, censorship resistance, and network security. +```sh +docker pull minaprotocol/mina-archive-maintenance:1.4.1-060f0a5-bullseye +``` -However it doesn't stop there. Check out some other resources to see what measures are in place to keep Mina secure. +## Usage for missing or invalid nonces -## Audits +The replayer application was developed to verify the Devnet/Mainnet archive data. You must run the replayer application against your existing Devnet/Mainnet database to verify the blockchain state. -### Protocol +To run the replayer application: -- [August 27, 2024 o1js](https://github.com/o1-labs/o1js/blob/a09c5167c4df64f879684e5af14c59cf7a6fce11/audits/VAR_o1js_240318_o1js_V3.pdf) by Veridise -- [December 12, 2023 Pickles](https://minaprotocol.com/wp-content/uploads/Least-Authority-Pickles-Final-Audit-Report.pdf) by Least Authority -- [August 28, 2023 Transaction Logic and Transaction Pool](https://minaprotocol.com/blog/least-authority-concludes-security-audit-of-mina-protocols-transaction-logic-and-transaction-pool) by Least Authority -- [October 16, 2022 Mina codebase, ecosystem projects](https://minaprotocol.com/wp-content/uploads/Mina-Security-Assessment-2022.pdf) by Mo Ashouri -- [February 22, 2022 Mina Client SDK, Signature Library and Base Components](https://www.nccgroup.com/us/research-blog/public-report-o-1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review/?sq=mina) by NCC Group -- [December 2020 Mina Protocol and Staking Economics](https://gauntlet.network/reports/mina) by Gauntlet Network -- [May 14, 2020 Coda Protocol ](https://minaprotocol.com/blog/ncc-group-security-audit-results-of-coda-protocol) by NCC Group +```sh +mina-replayer \ + --archive-uri {db_connection_string} \ + --input-file reference_replayer_input.json \ + --output-file replayer_input_file.json \ + --checkpoint-interval 10000 \ + --fix-nonces \ + --set-nonces \ + --dump-repair-script +``` -### Tools -- [Feb 4, 2022 Mina Ledger Application](https://minaprotocol.com/blog/ledger-nanox-nanos-developer-mode) by Least Authority -- [Sept 28, 2021 StakingPower Wallet ](https://minaprotocol.com/blog/least-authority-concludes-security-audit-on-stakingpower-wallet) by Least Authority -- [August 9, 2021 Auro Wallet](https://minaprotocol.com/blog/least-authority-concludes-security-audit-on-auro-wallet) by Least Authority -- [July 16, 2021 Clor.io Wallet](https://minaprotocol.com/blog/clorio-wallet-audit) by Least Authority +where: -### Auditors -- [Veridise](https://veridise.com/) -- [Least Authority](https://leastauthority.com/) -- [NCC Group](https://www.nccgroup.com/us/) -- [Gauntlet Network](https://www.gauntlet.xyz/) +- `archive-uri` - connection string to the archive database +- `input-file` - JSON file that holds the archive database +- `output-file` - JSON file that will hold the ledger with auxiliary information, like global slot and blockchain height, which will be dumped on the last block +- `checkpoint-interval` - frequency of checkpoints expressed in blocks count +- `replayer_input_file.json` - JSON file constructed from the Devnet/Mainnet genesis ledger: + ```sh + jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json + ``` -## Current Community Security Programs -- [Bug Bounty Program](https://minaprotocol.com/blog/re-launching-the-mina-ecosystem-bug-bounty-program) -- [Testworld Mission 2.0](https://minaprotocol.com/blog/testworld-2-protocol-performance-testing-program) +- `--fix-nonces` - adjust nonces values while replaying transactions +- `--set-nonces` - set missing nonces while replaying transactions +- `--dump-repair-script` - path to the output SQL script that will contain all updates to nonces made during the replayer run that can be directly applied to other database instances that contain the same data with invalid nonces +Running a replayer from scratch on a Devnet/Mainnet database can take up to a couple of days. The recommended best practice is to break the replayer into smaller parts by using the checkpoint capabilities of the replayer. +Additionally, running the replayer can exert significant demands on system resources that potentially affect the performance of the archive node. Because of the large resource requirements, we recommend that you execute the replayer in isolation from network connections, preferably within an isolated environment where the Devnet/Mainnet dumps can be imported. -## Learn +## Bad ledger hashes -- [Solving for Blockchain’s Security Flaw — Accessible Nodes](https://minaprotocol.com/blog/solving-for-blockchains-security-flaw?utm_medium=gitHub&utm_source=social&utm_campaign=evergreen) -- [What is Ouroboros Samisika?](https://minaprotocol.com/blog/how-ouroboros-samasika-upholds-minas-goals-of-decentralization?utm_medium=github&utm_source=social&utm_campaign=updates) -- [How Ouroboros Samasika Upholds Mina’s Goals of Decentralization](https://minaprotocol.com/blog/how-ouroboros-samasika-upholds-minas-goals-of-decentralization?utm_medium=github&utm_source=social&utm_campaign=updates) -- [Mina’s Mainnet Launch Marks a New Era for Internet Privacy and Data Security](https://minaprotocol.com/blog/minas-mainnet-launch-marks-a-new-era-for-internet-privacy-and-data-security?utm_medium=github&utm_source=social&utm_campaign=updates) +There is no ultimate fix for this issue because preserving historical ledger hashes is essential to the overall security of the Mina network. Even with this issue, you can validate archive data integrity. ---- -url: /mina-signer ---- +The replayer application has a built-in mechanism to skip errors when the `--continue-on-error` flag is enabled. +However, instead of skipping only blocks with bad ledger hashes, this mode skipped all of the problems with integrity. +With the new archive node maintenance package, you can run the replayer application without a special flag and to correctly handle the bad ledger hashes issue. -# Mina Signer +To run replayer: -Mina Signer is a NodeJS/Browser compatible JavaScript library tailored for the Mina Protocol. This library aids developers in seamlessly signing transactions and generating keys. A noteworthy feature is the ability to sign transactions offline, allowing for their broadcasting to the network whenever required. It also supports functionalities such as signing zkApp transactions, verifying these transactions, generating nullifiers, and more. +```sh +mina-replayer --archive-uri {db_connection_string} --input-file reference_replayer_input.json --output-file reference_replayer_output.json --checkpoint-interval 10000 +``` -## Installation +where: -To incorporate Mina Signer into your project: +- `archive-uri` - connection string to the archive database +- `input-file` - JSON file that holds the archive database +- `output-file` - JSON file that will hold the ledger with auxiliary information, like global slot and blockchain height, which will be dumped on the last block +- `checkpoint-interval` - frequency of checkpoints expressed in blocks count +- `replayer_input_file.json` - JSON file constructed from the Devnet/Mainnet genesis ledger: -```sh -npm install mina-signer -``` + ``` + jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json + ``` -## Mina Protocol Usage +:warning: Running a replayer from scratch on a Devnet/Mainnet database can take up to a couple of days. The recommended best practice is to break the replayer into smaller parts by using the checkpoint capabilities of the replayer. -Mina Signer offers a wide range of features for the Mina Protocol: +:warning: You must run the replayer using the Mainnet version. You can run it from the Docker image at `minaprotocol/mina-archive:3.1.0-ae112d3-bullseye`. -- Generate keys -- Sign transactions -- Verify transactions +## Missing blocks -Additionally, it ensures compatibility across various networks, like mainnet and testnet. +The daemon node unavailability can cause the archive node to miss some of the blocks. This recurring missing blocks issue consistently poses challenges. To address this issue, you can reapply missing blocks. -### Specifying the network +If you uploaded the missing blocks to Google Cloud, the missing blocks can be reapplied from precomputed blocks to preserve chain continuity. -When importing and initializing Mina Signer, it's imperative to designate the desired network. -Different networks may employ varying cryptographic methods. This specification is executed by supplying the network parameter during the constructor's invocation. -Possible values are `mainnet` and `testnet`. +1. To automatically verify and patch missing blocks, use the [download-missing-blocks.sh](https://github.com/MinaProtocol/mina/blob/berkeley/scripts/archive/download-missing-blocks.sh) script. -:::tip -By default, if no network is explicitly chosen, `mainnet` is the default choice. For the Berkeley network, use `testnet`. -::: + The `download-missing-blocks` script uses `localhost` as the database host so the script assumes that psql is running on localhost on port 5432. Modify `PG_CONN` in `download_missing_block.sh` for your environment. -```js +1. Install the required `mina-archive-blocks` and `mina-missing-blocks-auditor` scripts that are packed in the `minaprotocol/mina-archive:3.1.0-ae112d3-bullseye` Docker image. -const MainnetClient = new Client({ network: 'mainnet' }); // Specify mainnet -const TestnetClient = new Client({ network: 'testnet' }); // Specify testnet (Berkeley) -``` +1. Export the `BLOCKS_BUCKET`: -### Generating keys + ```sh + export BLOCKS_BUCKET="https://storage.googleapis.com/my_bucket_with_precomputed_blocks" + ``` -With Mina Signer, generating keypairs is straightforward. +1. Run the `mina-missing-blocks-auditor` script from the database host: -```js + For Devnet: -const client = new Client({ network: 'mainnet' }); // Specify mainnet -const keypair = client.genKeys(); // Generates a public and private keypair -``` + ```sh + download-missing-blocks.sh devnet {db_user} {db_password} + ``` -### Signing & Verifying Transactions + For Mainnet: -Mina Signer facilitates both transaction and stake delegation signing and verification. To sign a transaction, the sender's private must be provided. -Conversely, for verification, the sender's public key must be provided. Post-signing, the Mina Daemon can be utilized to broadcast the payment or delegation. + ```sh + download-missing-blocks.sh mainnet {db_user} {db_password} + ``` +### Using precomputed blocks from O1labs bucket -#### Payments +O1labs maintains a Google bucket containing precomputed blocks from Devnet and Mainnet, accessible at https://storage.googleapis.com/mina_network_block_data/. -Payments are transactions that transfer funds from one account to another. To sign a payment, the following parameters must be provided: +Note: It's important to highlight that precomputed blocks for **Devnet** between heights `2` and `1582` have missing fields or incorrect transaction data. Utilizing these blocks to patch your Devnet archive database will result in failure. For those who rely on precomputed blocks from this bucket, please follow the outlined steps: -```js +1. Download additional blocks from `gs://mina_network_block_data/devnet-extensional-bundle.tar.gz`. +2. Install the necessary `mina-archive-blocks` script contained within the `minaprotocol/mina-archive:3.1.0-ae112d3-bullseye` Docker image. +3. Execute mina-archive-blocks to import the extracted blocks from step 1 using the provided command: -const client = new Client({ network: 'mainnet' }); -const keypair = client.genKeys(); + ```sh + mina-archive-blocks --archive-uri --extensional ./extensional/* + ``` +4. Proceed with patching your Devnet database with blocks having heights other than `2` to `1582` using the available precomputed blocks. -const payment = client.signPayment( - { - to: keypair.publicKey, // Public key of the recipient - from: keypair.publicKey, // Public key of the sender - amount: '1', // Amount to be sent (in nano MINA) - fee: '1', // Fee to be paid (in nano MINA) - nonce: '0', // Nonce of the sender - }, - keypair.privateKey -); +## Next steps -const verifiedPayment = client.verifyPayment(payment); -``` +Now that you have completed the steps to properly maintain the correctness of the archive database, you are ready to perform the archive [migration process](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley). -#### Delegations +--- +url: /network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley +--- -Stake delegations are a way for users to delegate their stake to a validator. This allows the validator to produce blocks on behalf of the delegator. To sign a stake delegation, the following parameters must be provided: +# Migrating Devnet/Mainnet Archive to Berkeley Archive -```js +Before you start the process to migrate your archive database from the current Mainnet or Devnet format to Berkeley, be sure that you: -const client = new Client({ network: 'mainnet' }); -const keypair = client.genKeys(); +- [Understand the Archive Migration](/network-upgrades/berkeley/archive-migration/understanding-archive-migration) +- Meet the foundational requirements in [Archive migration prerequisites](/network-upgrades/berkeley/archive-migration/archive-migration-prerequisites) +- Have successfully installed the [archive migration package](/network-upgrades/berkeley/archive-migration/archive-migration-installation) -const delegation = client.signStakeDelegation( - { - to: keypair.publicKey, // Public key of the validator - from: keypair.publicKey, // Public key of the delegator - fee: '1', // Fee to be paid (in nano MINA) - nonce: '0', // Nonce of the delegator - }, - keypair.privateKey -); +## Migration process -const verifiedDelegation = client.verifyStakeDelegation(delegation); -``` +The Devnet/Mainnet migration can take up to a couple of days. +Therefore, you can achieve a successful migration by using three stages: -#### Generic Signing +- **Stage 1:** Initial migration -Mina Signer can accept a generic payload and determine the most apt signing approach via `signTransaction()`. -This functionality is especially beneficial for applications that support different types of transactions. +- **Stage 2:** Incremental migration -```js +- **Stage 3:** Remainder migration -const client = new Client({ network: 'mainnet' }); -const keypair = client.genKeys(); +Each stage has three migration phases: -// Sign a payment -client.signTransaction( - { - to: keypair.publicKey, - from: keypair.publicKey, - amount: '1', - fee: '1', - nonce: '0', - }, - keypair.privateKey -); +- **Phase 1:** Copying data and precomputed blocks from Devnet/Mainnet database using the **berkeley_migration** app. -// Sign a delegation -client.signTransaction( - { - to: keypair.publicKey, - from: keypair.publicKey, - fee: '1', - nonce: '0', - }, - keypair.privateKey -); +- **Phase 2:** Populating new Berkeley tables using the **replayer app in migration mode** + +- **Phase 3:** Additional validation for migrated database +Review these phases and stages before you start the migration. -// Sign a zkApp transaction -client.signTransaction( - { - zkappCommand: ..., - feePayer: ... - }, - keypair.privateKey -); +## Simplified approach -// Sign a simple string payload -client.signTransaction('Hello World', keypair.privateKey); +For convenience, use the `mina-berkeley-migration-script` app if you do not need to delve into the details of migration or if your environment does not require a special approach to migration. + +### Stage 1: Initial migration + +``` +mina-berkeley-migration-script \ + initial \ + --genesis-ledger ledger.json \ + --source-db postgres://postgres:postgres@localhost:5432/source \ + --target-db postgres://postgres:postgres@localhost:5432/migrated \ + --blocks-bucket mina_network_block_data \ + --blocks-batch-size 500 \ + --checkpoint-interval 10000 \ + --checkpoint-output-path . \ + --precomputed-blocks-local-path . \ + --network NETWORK ``` -### Broadcasting a Signed Payment +where: -After signing a payment, you can broadcast it to the network via a Mina Node GraphQL endpoint: +`-g | --genesis-ledger`: path to the genesis ledger file -```javascript +`-s | --source-db`: connection string to the database to be migrated +`-t | --target-db`: connection string to the database that will hold the migrated data -const client = new Client({ network: 'mainnet' }); +`-b | --blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -const senderPrivateKey = 'EKFd1Gx...'; -const senderPublicKey = 'B62qrDM...'; +`-bs | --blocks-batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up the migration process. -let payment = { - from: senderPublicKey, - to: 'B62qkBw...', - amount: 100, - nonce: 1, - fee: 1000000, -}; +`-n | --network`: network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json`. -const signedPayment = client.signPayment(payment, senderPrivateKey); +`-c | --checkpoint-output-path`: path to folder for replayer checkpoint files -const url = 'https://your-mina-node/graphql'; +`-i | --checkpoint-interval`: frequency of dumping checkpoint expressed in blocks count -const sendPaymentMutationQuery = ` -mutation SendPayment($input: SendPaymentInput!, $signature: SignatureInput!) { - sendPayment(input: $input, signature: $signature) { - payment { - hash - } - } -} -`; -const graphQlVariables = { - input: signedPayment.data, - signature: signedPayment.signature, -}; -const body = JSON.stringify({ - query: sendPaymentMutationQuery, - variables: graphQlVariables, - operationName: 'SendPayment', -}); +`-l | --precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location -const paymentResponse = await fetch(url, { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body, -}); +The command output is the `migration-replayer-XXX.json` file required for the next run. -const paymentResponseJson = await paymentResponse.json(); -if (paymentResponse.ok) { - console.log( - `Transaction hash: ${paymentResponseJson.data.sendPayment.payment.hash}` - ); -} else { - console.error(JSON.stringify(paymentResponseJson)); -} +### Stage 2: Incremental migration + +``` +mina-berkeley-migration-script \ + incremental \ + --genesis-ledger ledger.json \ + --source-db postgres://postgres:postgres@localhost:5432/source \ + --target-db postgres://postgres:postgres@localhost:5432/migrated \ + --blocks-bucket mina_network_block_data \ + --blocks-batch-size 500 \ + --network NETWORK \ + --checkpoint-output-path . \ + --checkpoint-interval 10000 \ + --precomputed-blocks-local-path . \ + --replayer-checkpoint migration-checkpoint-XXX.json ``` -### Payment & Delegation Transaction Hashes +where: -In addition to signing/verifying payments/delegations for the Mina Protocol, Mina Signer allows you to compute the hash that will be used to identify the transaction on the blockchain. This is useful for applications that require the transaction hash before the transaction is broadcasted to the network. +`-g | --genesis-ledger`: path to the genesis ledger file -```js +`-s | --source-db`: connection string to the database to be migrated -const client = new Client({ network: 'mainnet' }); -const keypair = client.genKeys(); +`-t | --target-db`: connection string to the database that will hold the migrated data -const payment = client.signTransaction( - { - to: keypair.publicKey, - from: keypair.publicKey, - amount: '1', - fee: '1', - nonce: '0', - }, - keypair.privateKey -); -const hashedPayment = client.hashPayment(payment); +`-b | --blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -const delegation = client.signTransaction( - { - to: keypair.publicKey, - from: keypair.publicKey, - fee: '1', - nonce: '0', - }, - keypair.privateKey -); -const hashedDelegation = client.hashStakeDelegation(delegation); -``` +`-bs | --blocks-batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. -### Rosetta Integration +`-n | --network`: network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json`. -For those developing with [Rosetta](https://www.rosetta-api.org/), Mina Signer provides an avenue to transform a signed Rosetta transaction into a Mina-compliant transaction, ready for broadcasting through the Mina Daemon. +`-r | --replayer-checkpoint`: path to the latest checkpoint file `migration-checkpoint-XXX.json` -```js +`-c | --checkpoint-output-path`: path to folder for replayer checkpoint files -const client = new Client({ network: 'mainnet' }); +`-i | --checkpoint-interval`: frequency of dumping checkpoint expressed in blocks count -const signedRosettaTx = '...'; -const signedGraphQLCommand = - client.signedRosettaTransactionToSignedCommand(signedRosettaTx); +`-l | --precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location + +### Stage 3: Remainder migration + +``` +mina-berkeley-migration-script \ + final \ + --genesis-ledger ledger.json \ + --source-db postgres://postgres:postgres@localhost:5432/source \ + --target-db postgres://postgres:postgres@localhost:5432/migrated \ + --blocks-bucket mina_network_block_data \ + --blocks-batch-size 500 \ + --network NETWORK \ + --checkpoint-output-path . \ + --checkpoint-interval 10000 \ + --precomputed-blocks-local-path . \ + --replayer-checkpoint migration-checkpoint-XXX.json \ + -fc fork-genesis-config.json ``` -For detailed Rosetta usage including the offline signer CLI tool and `signRosettaTransaction`, see the [Rosetta Offline Signer](/exchange-operators/rosetta/samples/using-signer) documentation. +where: -## o1js Integration +`-g | --genesis-ledger`: path to the genesis ledger file -Mina Signer can seamlessly integrate with [o1js](/zkapps/o1js), delivering an array of features for zkApps like: +`-s | --source-db`: connection string to the database to be migrated -- zkApp transaction signing and verification -- Field payload signing and verification -- Nullifier generation +`-t | --target-db`: connection string to the database that will hold the migrated data -### Signing & Verifying zkApp transactions +`-b | --blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -Mina Signer supports signing and verifying zkApp transactions. o1js itself can be used to sign zkApp transactions, -but Mina Signer offers the ability to sign a zkApp transaction that can easily be broadcasted with a Mina Daemon. This can be very useful for wallet applications that want to support zkApps. +`-bs | --blocks-batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up the migration process. -```js +`-n | --network`: network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json`. +`-r | --replayer-checkpoint`: path to the latest checkpoint file `migration-checkpoint-XXX.json` +`-c | --checkpoint-output-path`: path to folder for replayer checkpoint files -const client = new Client({ network: 'testnet' }); -const keypair = client.genKeys(); +`-i | --checkpoint-interval`: frequency of dumping checkpoint expressed in blocks count -const zkAppTransaction = await Mina.transaction(feePayerAddress, () => { - // ... Interact with a zkApp inside this block to produce a zkApp transaction -}); +`-l | --precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location -// Sign the zkApp transaction with Mina Signer -const signedZkAppTransaction = client.signZkappCommand( - { - zkappCommand: JSON.parse(JSON.stringify(txn.transaction)), - feePayer: { - feePayer: keypair.publicKey, - fee: '1', - nonce: '0', - memo: 'memo', - }, - }, - keypair.privateKey -); +`-fc | --fork-config`: fork genesis config file is the new genesis config that is distributed with the new daemon and is published after the fork block is announced -// Verify the zkApp transaction with Mina Signer -const verifiedZkAppTransaction = client.verifyZkappCommand( - signedZkAppTransaction -); -``` +## Advanced approach -Firstly, when supplying the input parameters for `signZkappCommand()`, we must first parse the zkApp transaction into a string and then into a JSON object. This is because the types generated from `Mina.transaction()` are not compatible with the types used by Mina Signer. -Secondly, we specify the `feePayer` object which contains the public key of the fee payer, the fee to be paid, the nonce of the fee payer, and the memo of the transaction. The `feePayer` object is used to sign the zkApp transaction. +If the simplified berkeley migration script is, for some reason, not suitable for you, it is possible to run the migration using the **berkeley_migration** and **replayer** apps without an interface the script provides. -:::tip -Use o1js to sign zkApp transactions if you can, as it's more ergonomic and easier to use. Only use `Mina Signer` if you need to sign zkApp transactions offline and broadcast at a later time (e.g. wallet software). -::: +### Stage 1: Initial migration -### Signing/Verifying Field payloads +This first stage requires only the initial Berkeley schema, which is the foundation for the next migration stage. This schema populates the migrated database and creates an initial checkpoint for further incremental migration. -Mina Signer can sign and validate Field payloads. This is invaluable when ensuring a Field payload's authenticity, as it confirms the payload remains untampered by external parties. +- Inputs + - Unmigrated Devnet/Mainnet database + - Devnet/Mainnet genesis ledger + - Empty target Berkeley database with the schema created, but without any content -```js +- Outputs + - Migrated Devnet/Mainnet database to the Berkeley format from genesis up to the last canonical block in the original database + - Replayer checkpoint that can be used for incremental migration -const client = new Client({ network: 'testnet' }); -const keypair = client.genKeys(); +#### Phase 1: Berkeley migration app run -const fields = [10n, 20n, 30n, 340817401n, 2091283n, 1n, 0n]; -const signedFields = client.signFields(fields, keypair.privateKey); -const verifiedFields = client.verifyFields(signedFields); +``` +mina-berkeley-migration \ + --batch-size 1000 \ + --config-file ledger.json \ + --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ + --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ + --blocks-bucket mina_network_block_data \ + --precomputed-blocks-local-path . \ + --keep-precomputed-blocks \ + --network NETWORK ``` -If you are using o1js to generate Field payloads, you must convert the Fields to BigInts before signing/verifying them. -In Mina Signer, the Field type is a BigInt (while in o1js they are a separate data structure), so you must convert the fields from o1js to BigInts before signing/verifying them. +where: -```js +`--batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. +`--config-file`: path to the genesis ledger file -const client = new Client({ network: 'testnet' }); -const keypair = client.genKeys(); +`--mainnet-archive-uri`: connection string to the database to be migrated -const fields = [Field(10), Field(20)].map((f) => f.toBigInt()); -const signedFields = client.signFields(fields, keypair.privateKey); -const verifiedFields = client.verifyFields(signedFields); -``` +`--migrated-archive-uri`: connection string to the database that will hold the migrated data -### Nullifiers +`--blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -Mina Signer supports generating nullifiers for zkApp transactions. In the world of cryptography, nullifiers play a pivotal role. -They stand as unique markers, maintaining anonymity yet ensuring account reliability, and staving off illicit undertakings like double-spends. -To generate a nullifier, provide a message (an array of BigInts) and the sender's private key. +`--precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location -```js +`--keep-precomputed-blocks`: keep the precomputed blocks on-disk after the migration is complete -const client = new Client({ network: 'testnet' }); -const keypair = client.genKeys(); +`--network`: the network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -const message = [10n, 20n, 30n, 340817401n, 2091283n, 1n, 0n]; -const nullifier = client.createNullifier(message, keypair.privateKey); +#### Phase 2: Replayer in migration mode run + +Replayer config must contain the Devnet/Mainnet ledger as the starting point. So first, you must prepare the replayer config file: + +``` + jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json ``` ---- -url: /network-upgrades/berkeley/appendix ---- +where: -# Appendix + `genesis_ledger.json` is the genesis file from a daemon bootstrap on a particular network -## Migration from o1labs/client-sdk to mina-signer +Then: +``` + mina-migration-replayer \ + --migration-mode \ + --archive-uri postgres://postgres:postgres@localhost:5432/migrated \ + --input-file replayer_input_config.json \ + --checkpoint-interval 10000 \ + --checkpoint-output-folder . +``` -The signing library `o1labs/client-sdk` was deprecated some time ago and will stop working after the Mina mainnet upgrade. All users should upgrade to use the [mina-signer](https://www.npmjs.com/package/mina-signer) library. +where: -Below you will find an example of how to use the `mina-signer` library. Please keep in mind the following: +`--migration-mode`: flag for migration -1. Make sure to adjust the `nonce` to the correct nonce on the account you want to use as "sender" -1. Update the `url` variable with an existing Mina Node GraphQL endpoint +`--archive-uri`: connection string to the database that will hold the migrated data -```javascript +`--input-file`: path to the replayer input file, see below on how's created +`replayer_input_config.json`: is a file constructed out of network genesis ledger: + ``` + jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json + ``` -// create the client and define the keypair +`--checkpoint-interval`: frequency of checkpoints file expressed in blocks count -const client = new Client({ network: 'testnet' }); // Mind the `network` client configuration option +`--checkpoint-output-folder`: path to folder for replayer checkpoint files -const senderPrivateKey = 'EKFd1Gx...'; // Sender's private key -const senderPublicKey = 'B62qrDM...'; // Sender's public key, perhaps derived from the private key using `client.derivePublicKey(senderPrivateKey)`; +#### Phase 3: Validations -// define and sign payment +Use the **berkeley_migration_verifier** app to perform checks for both the fully migrated and partially migrated databases. -let payment = { - from: senderPublicKey, - to: 'B62qkBw...', // Recipient public key - amount: 100, - nonce: 1, - fee: 1000000, -}; +``` + mina-berkeley-migration-verifier \ + pre-fork \ + --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ + --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated +``` -const signedPayment = client.signPayment(payment, senderPrivateKey); +where: -// send payment to graphql endpoint +`--mainnet-archive-uri`: connection string to the database to be migrated -const url = 'https://qanet.minaprotocol.network/graphql'; +`--migrated-archive-uri`: connection string to the database that will hold the migrated data -const sendPaymentMutationQuery = ` -mutation SendPayment($input: SendPaymentInput!, $signature: SignatureInput!) { - sendPayment(input: $input, signature: $signature) { - payment { - hash - } - } -} -`; -const graphQlVariables = { - input: signedPayment.data, - signature: signedPayment.signature, -}; -const body = JSON.stringify({ - query: sendPaymentMutationQuery, - variables: graphQlVariables, - operationName: 'SendPayment', -}); +### Stage 2: Incremental migration -const paymentResponse = await fetch(url, { - method: 'POST', - headers: { 'Content-Type': 'application/json' }, - body -}); +After the initial migration, the data is migrated data up to the last canonical block. However, Devnet/Mainnet data is progressing with new blocks that must also be migrated again and again until the fork block is announced. -const paymentResponseJson = await paymentResponse.json(); -if (paymentResponse.ok) { - console.log(`Transaction hash: ${paymentResponseJson.data.sendPayment.payment.hash}`); -} else { - console.error(JSON.stringify(paymentResponseJson)); -} +:::info +Incremental migration can, and probably must, be repeated a couple of times until the fork block is announced by Mina Foundation. +Run the incremental migration multiple times with the latest Devnet/Mainnet database and the latest replayer checkpoint file. +::: + +- Inputs + - Latest Devnet/Mainnet database + - Devnet/Mainnet genesis ledger + - Replayer checkpoint from last run + - Migrated berkeley database from initial migration + +- Outputs + - Migrated Devnet/Mainnet database to the Berkeley format up to the last canonical block + - Replayer checkpoint which can be used for the next incremental migration +### Phase 1: Berkeley migration app run ``` +mina-berkeley-migration \ + --batch-size 1000 \ + --config-file ledger.json \ + --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ + --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ + --blocks-bucket mina_network_block_data \ + --precomputed-blocks-local-path . \ + --keep-precomputed-blocks \ + --network NETWORK +``` ---- -url: /network-upgrades/berkeley/archive-migration/appendix ---- +where: -# Appendix +`--batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. -## Archive node schema changes +`--config-file`: path to the genesis ledger file -If you are using the Archive Node database directly for your system integrations, then you should understand all the changes that might impact your applications. The most important change is that the `balances` table in the Berkeley schema will no longer exist. In the new schema, it is replaced with the table `accounts_accessed` - from an application semantics point of view, the data in `accounts_accessed` is still the same. +`--mainnet-archive-uri`: connection string to the database to be migrated -In the Berkeley protocol, accounts can now have the same public key but a different token_id. This means accounts are identified by both their public key and token_id, not just the public key. Consequently, the foreign key for the account in all tables is account_identifier_id instead of public_key_id. +`--migrated-archive-uri`: connection string to the database that will hold the migrated data -### Schema differences -- **Removed Types** - - The options `create_token`, `create_account`, and `mint_tokens` have been removed from the user_command_type enumeration. -- Indexes Dropped - - We've removed several indexes from tables, this may affect how you search and organize data: - - `idx_public_keys_id` - - `idx_public_keys_value` - - `idx_snarked_ledger_hashes_value` - - `idx_blocks_id` - - `idx_blocks_state_hash` -- **Table Removed** - - The `balances` table is no longer available. -- **New Tables Added** - - We've introduced the following new tables: - - `tokens` - - `token_symbols` - - `account_identifiers` - - `voting_for` - - `protocol_versions` - - `accounts_accessed` - - `accounts_created` - - `zkapp_commands` - - `blocks_zkapp_commands` - - `zkapp_field` - - `zkapp_field_array` - - `zkapp_states_nullable` - - `zkapp_states` - - `zkapp_action_states` - - `zkapp_events` - - `zkapp_verification_key_hashes` - - `zkapp_verification_keys` - - `zkapp_permissions` - - `zkapp_timing_info` - - `zkapp_uris` - - `zkapp_updates` - - `zkapp_balance_bounds` - - `zkapp_nonce_bounds` - - `zkapp_account_precondition` - - `zkapp_accounts` - - `zkapp_token_id_bounds` - - `zkapp_length_bounds` - - `zkapp_amount_bounds` - - `zkapp_global_slot_bounds` - - `zkapp_epoch_ledger` - - `zkapp_epoch_data` - - `zkapp_network_precondition` - - `zkapp_fee_payer_body` - - `zkapp_account_update_body` - - `zkapp_account_update` - - `zkapp_account_update_failures` -- **Updated Tables** - - The following tables have been updated - - `timing_info` - - `user_commands` - - `internal_commands` - - `epoch_data` - - `blocks` - - `blocks_user_commands` - - `blocks_internal_commands` +`--blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -### Differences per table -- **`timing_info`** - - Removed columns: - - `token` - - `initial_balance` -- **`user_commands`** - - Removed columns: - - `fee_token` - - `token` -- **`internal_commands`** - - Removed columns: - - `token` - - Renamed column - - `command_type` to `type` -- **`epoch_data`** - - Added columns: - - `total_currency` - - `start_checkpoint` - - `lock_checkpoint` - - `epoch_length` -- **`blocks`** - - Added columns: - - `last_vrf_output` - - `min_window_density` - - `sub_window_densities` - - `total_currency` - - `global_slot_since_hard_fork` - - `global_slot_since_genesis` - - `protocol_version_id` - - `proposed_protocol_version_id` - - Removed column: - - `global_slot` -- **`blocks_user_commands`** - - Removed columns: - - `fee_payer_account_creation_fee_paid` - - `receiver_account_creation_fee_paid` - - `created_token` - - `fee_payer_balance` - - `source_balance` - - `receiver_balance` - - Added index: - - `idx_blocks_user_commands_sequence_no` -- **`blocks_internal_commands`** - - Removed columns: - - `receiver_account_creation_fee_paid` - - `receiver_balance` - - Added indexes: - - `idx_blocks_internal_commands_sequence_no` - - `idx_blocks_internal_commands_secondary_sequence_no` - -### Rosetta API new operations - -The Berkeley upgrade introduces two new operation types: -- `zkapp_fee_payer_dec` -- `zkapp_balance_change` - ---- -url: /network-upgrades/berkeley/archive-migration/archive-migration-installation ---- - -The archive node Berkeley migration package is sufficient for satisfying the migration from Devnet/Mainnet to Berkeley. -However, it has some limitations. For example, the migration package does not migrate a non-canonical chain and it skips orphaned blocks that are not part of a canonical chain. +`--precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location -To mitigate these limitations, the archive node maintenance package is available for use by archive node operators who want to maintain a copy of their Devnet and Mainnet databases for historical reasons. +`--keep-precomputed-blocks`: keep the precomputed blocks on-disk after the migration is complete -## Install with Google Cloud SDK +`--network`: the network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -The Google Cloud SDK installer does not always register a `google-cloud-sdk` apt package. The best way to install gsutil is using the apt repostory: +#### Phase 2: Replayer in migration mode run -```sh -curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg -echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list -sudo apt-get update && sudo apt-get install google-cloud-sdk +``` + mina-migration-replayer \ + --migration-mode \ + --archive-uri postgres://postgres:postgres@localhost:5432/migrated \ + --input-file replayer-checkpoint-XXX.json \ + --checkpoint-interval 10000 \ + --checkpoint-output-folder . ``` -## Download the o1labs Mainnet archive database +where: -We strongly encourage you to perform the migration on your own data to preserve the benefits of decentralization. However, if you want to use the archive data that o1labs runs (for example, to bootstrap a new archive from SQL without waiting all day for the chain to download and replay), you can use the following steps: +`--migration-mode`: flag for migration -1. Download the Devnet/Mainnet archive data using cURL or gsutil: +`--archive-uri`: connection string to the database that will hold the migrated data - - cURL: +`--input-file`: path to the latest checkpoint file `replayer-checkpoint-XXX.json` - For Devnet: - ```sh - curl https://storage.googleapis.com/mina-archive-dumps/devnet-archive-dump-{date}_0000.sql.tar.gz - ``` +`replayer-checkpoint-XXX.json`: the latest checkpoint generated from the previous migration - For Mainnet: - ```sh - curl https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-{date}_0000.sql.tar.gz - ``` +`--checkpoint-interval`: frequency of checkpoints file expressed in blocks count - To filter the dumps by date, replace `{date}` using the required `yyyy-dd-mm` format. For example, for March 15, 2024, use `2024-03-15`. +`--checkpoint-output-folder`: path to folder for replayer checkpoint files - :warning: The majority of backups have the `0000` suffix. If a download with that name suffix is not available, try incrementing it. For example, `0001`, `0002`, and so on. +Incremental migration can be run continuously on top of the initial migration or last incremental until the fork block is announced. - - gsutil: +#### Phase 3: Validations - ```sh - gsutil cp gs://mina-archive-dumps/mainnet-archive-dump-2024-01-15* . - ``` +Use the **berkeley_migration_verifier** app to perform checks for both the fully migrated and partially migrated database. -2. Extract the tar package. +``` + mina-berkeley-migration-verifier \ + pre-fork \ + --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ + --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated +``` - ```sh - tar -xvzf {network}-archive-dump-{date}_0000.sql.tar.gz {network}-archive-dump-{date}_0000.sql - ``` +where: -3. Import the Devnet/Mainnet archive dump into the Berkeley database. +`--mainnet-archive-uri`: connection string to the database to be migrated - Run this command at the database server: +`--migrated-archive-uri`: connection string to the database that will hold the migrated data - ```sh - psql -U {user} -f {network}-archive-dump-{date}_0000.sql - ``` +Note that: you can run incremental migration continuously on top of the initial migration or the last incremental until the fork block is announced. - The database in the dump **archive_balances_migrated** is created with the Devnet/Mainnet archive schema. +### Stage 3: Remainder migration - Note: This database does not have any Berkeley changes. +When the fork block is announced, you must tackle the remainder migration. This is the last migration run +you need to perform. In this stage, you close the migration cycle with the last migration of the remainder blocks between the current last canonical block and the fork block (which can be pending, so you don't need to wait 290 blocks until it would become canonical). +You must use `--fork-state-hash` as an additional parameter to the **berkeley-migration** app. -## Ensure the location of Google Cloud bucket with the Devnet/Mainnet precomputed blocks +- Inputs + - Latest Devnet/Mainnet database + - Devnet/Mainnet genesis ledger + - Replayer checkpoint from last run + - Migrated Berkeley database from last run + - Fork block state hash -The recommended method is to perform migration on your own data to preserve the benefits of decentralization. +- Outputs + - Migrated devnet/mainnet database to berkeley up to fork point + - Replayer checkpoint which can be used for the next incremental migration -`gsutil cp gs://mina_network_block_data/{network}-*.json .` +:::info +The migrated database output from this stage of the final migration is required to initialize your archive nodes on the upgraded network. +::: -:warning: Precomputed blocks for the Mainnet network take ~800 GB of disk space. Plan for adequate time to download these blocks. The Berkeley migration app downloads them incrementally only when needed. +#### Phase 1: Berkeley migration app run -## Validate the Devnet/Mainnet database +``` +mina-berkeley-migration \ + --batch-size 1000 \ + --config-file ledger.json \ + --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ + --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ + --blocks-bucket mina_network_block_data \ + --precomputed-blocks-local-path \ + --keep-precomputed-blocks \ + --network NETWORK \ + --fork-state-hash {fork-state-hash} +``` -The correct Devnet/Mainnet database state is crucial for a successful migration. +where: -[Missing blocks](/network-upgrades/berkeley/archive-migration/mainnet-database-maintenance#missing-blocks) is one the most frequent issues when dealing with the Devnet/Mainnet archive. Although this step is optional, it is strongly recommended that you verify the archive condition before you start the migration process. +`--batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. -To learn how to maintain archive data, see [Devnet/Mainnet database maintenance](/network-upgrades/berkeley/archive-migration/mainnet-database-maintenance). +`--config-file`: path to the genesis ledger file -## Download the migration applications +`--mainnet-archive-uri`: connection string to the database to be migrated -Migration applications are distributed as part of the archive migration Docker and Debian packages. +`--migrated-archive-uri`: connection string to the database that will hold the migrated data -Choose the packages that are appropriate for your environment. +`--blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -### Debian packages +`--precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location -To get the Debian packages: +`--keep-precomputed-blocks`: keep the precomputed blocks on-disk after the migration is complete -``` -CODENAME=bullseye -CHANNEL=stable -VERSION=3.0.1-e848ecb +`--network`: the network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -echo "deb [trusted=yes] http://packages.o1test.net $CODENAME $CHANNEL" | tee /etc/apt/sources.list.d/mina.list -apt-get update -apt-get install --allow-downgrades -y "mina-archive-migration=$VERSION" -``` +`--fork-state-hash`: fork state hash -### Docker image +:::info +When you run the **berkeley-migration** app with fork-state-hash, there is no requirement for the fork state block to be canonical. +The tool automatically converts all pending blocks in the subchain, including the fork block, to canonical blocks. +::: -To get the Docker image: +#### Phase 2: Replayer in migration mode run ``` -docker pull gcr.io/o1labs-192920/mina-archive-migration:3.0.1-e848ecb-{codename} +mina-migration-replayer \ + --migration-mode \ + --archive-uri postgres://postgres:postgres@localhost:5432/migrated \ + --input-file replayer-checkpoint-XXX.json \ + --checkpoint-interval 10000 \ + --checkpoint-output-folder . ``` -Where supported codenames are: -- bullseye -- focal -- buster +where: +`--migration-mode`: flag for migration -## Devnet/Mainnet genesis ledger +`--archive-uri`: connection string to the database that will hold the migrated data -The Mina Devnet/Mainnet genesis ledger is stored in GitHub in the `mina` repository under the `genesis_ledgers` subfolder. However, if you are already running a daemon that is connected to the Mina Mainnet or the Devnet network, you already have the genesis ledger locally. +`--input-file`: path to the latest checkpoint file `replayer-checkpoint-XXX.json` from stage 1 -## Berkeley database schema files +`replayer-checkpoint-XXX.json`: the latest checkpoint generated from the previous migration -You can get the Berkeley schema files from different locations: +`--checkpoint-interval`: frequency of checkpoints file expressed in blocks count -- GitHub repository from the `berkeley` branch. +`--checkpoint-output-folder`: path to folder for replayer checkpoint files - Note: The `berkeley` branch can contain new updates regarding schema files, so always get the latest schema files instead of using an already downloaded schema. +#### Phase 3: Validations -- Archive/Rosetta Docker from `berkeley` version +Use the **berkeley_migration_verifier** app to perform checks for both the fully migrated and partially migrated databases. -### Example: Downloading schema sources from GitHub +``` + mina-berkeley-migration-verifier \ + post-fork \ + --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ + --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ + --fork-config-file fork_genesis_config.json \ + --migrated-replayer-output replayer-checkpoint-XXXX.json +``` - ```sh - wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql +where: - wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql - ``` +`--mainnet-archive-uri`: connection string to the database to be migrated -## Next steps +`--migrated-archive-uri`: connection string to the database that will hold the migrated data -Congratulations on completing the essential preparation and verification steps. You are now ready to perform the migration steps in [Migrating Devnet/Mainnet Archive to Berkeley Archive](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley). +`--migrated-replayer-output`: path to the latest checkpoint file `replayer-checkpoint-XXX.json` ---- -url: /network-upgrades/berkeley/archive-migration/archive-migration-prerequisites ---- +`--fork-config`: fork genesis config file is the new genesis config that is distributed with the new daemon and is published after the fork block is announced -To successfully migrate the archive database into the Berkeley version of the Mina network, you must ensure that your environment meets the foundational requirements. +### Example migration steps using Mina Foundation data for Devnet using Debian -## Migration host +See: [Worked example using March 22 data](/network-upgrades/berkeley/archive-migration/debian-example) -- PostgreSQL database for database server -- If you use Docker, then any of the supported OS by Mina (bullseye, focal, or buster) with at least 32 GB of RAM -- gsutil application from Google Cloud Suite in version 5 or later -- (Optional) Docker in version 23.0 or later +### Example migration steps using Mina Foundation data for Mainnet using Docker -## (Optional) Devnet/Mainnet database +See: [Worked example using March 22 data](/network-upgrades/berkeley/archive-migration/docker-example) -One of the most obvious prerequisites is a Mainnet database. +## How to verify a successful migration -If you don't have an existing database with Devnet/Mainnet archive data, you can always download it from the Google Cloud bucket. However, we strongly encourage you to perform migration on your own data to preserve the benefits of decentralization. -You can use any gsutil-compatible alternative to Google Cloud or a gsutil wrapper program. - -## (Optional) Google Cloud bucket with Devnet/Mainnet precomputed blocks - -Precomputed blocks are the JSON files that a correctly configured node updloads to the Google Cloud bucket. -The Devnet/Mainnet to Berkeley archive data migration requires access to precomputed blocks that are uploaded by daemons that are connected to the Devnet or Mainnet networks. - -The **berkeley-migration** app uses the gsutil app to download blocks. If you didn't store precomputed blocks during the first phase of migration, you can use the precomputed blocks provided by Mina Foundation. -However, it is strongly recommended that you perform migration on your own data to preserve the benefits of decentralization. +o1Labs and Mina Foundation make every effort to provide reliable tools of high quality. However, it is not possible to eliminate all errors and test all possible Mainnet archive variations. +All important checks are implemented in the `mina-berkeley-migration-verifier` application. +However, you can use the following checklist if you want to perform the checks manually: -For Devnet blocks: +1. All transaction (user command and internal command) hashes are left intact. -```sh -gsutil cp gs://mina_network_block_data/devnet-*.json . -``` + Verify that the `user_command` and `internal_command` tables have the Devnet/Mainnet format of hashes. For example, `CkpZirFuoLVV...`. -For Mainnet blocks: +2. Parent-child block relationship is preserved -```sh -gsutil cp gs://mina_network_block_data/mainnet-*.json . -``` + Verify that a given block in the migrated archive has the same parent in the Devnet/Mainnet archive (`state_hash` and `parent_hash` columns) that was used as input. -:warning: Precomputed blocks for the Mainnet network take ~800 GB of disk space. Plan for adequate time to download these blocks. The Berkeley migration app downloads them incrementally only when needed. You can instead download a 100 GB bundle of only the canonical Mainnet blocks that unpacks into ~220 GB: +3. Account balances remain the same -```sh -gsutil cp gs://mina_network_block_data/mainnet-bundle-2024-03-20.tar.zst . ; tar -xf mainnet-bundle-2024-03-20.tar.zst -``` + Verify the same balance exists for a given block in Mainnet and the migrated databases. -:warning: Precomputed blocks for the Devnet network take several hundred GBs. Plan for adequate time to download these blocks. Instead, you can download a ~50 GB bundle of only the canonical Devnet blocks that unpacks into ~90 GB: +## Tips and tricks -```sh -gsutil cp gs://mina_network_block_data/devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz . ; tar -xf devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz -``` +We are aware that the migration process can be very long (a couple of days). Therefore, we encourage you to use cron jobs that migrate data incrementally. +The cron job requires access to Google Cloud buckets (or other storage): -These bundles are partial. Updated documentation with the new links and final data will be provided _after_ the Berkeley major upgrade is completed. +- A bucket to store migrated-so-far database dumps +- A bucket to store checkpoint files -The best practice is to collect precomputed blocks by yourself or by other third parties to preserve the benefits of decentralization. +We are tightly coupled with Google Cloud infrastructure due to the precomputed block upload mechanism. +This is why we are using also buckets for storing dumps and checkpoint. However, you do not have to use Google Cloud for other things than +precomputed blocks. With configuration, you can use any gsutil-compatible storage backend (for example, S3). ---- -url: /network-upgrades/berkeley/archive-migration/debian-example ---- +Before running the cron job, upload an initial database dump and an initial checkpoint file. -# Debian example +To create the files, run these steps locally: -You can follow these steps that can be copy-pasted directly into a fresh Debian 11. +1. Download a Devnet/Mainnet archive dump and load it into PostgreSQL. +2. Create an empty database using the new archive schema. +3. Run the **berkeley-migration** app against the Devnet/Mainnet and new databases. +4. Run the **replayer app in migration mode** with the `--checkpoint-interval` set to a suitable value (perhaps 100) and start with the original Devnet/Mainnet ledger in the input file. +5. Use pg_dump to dump the migrated database and upload it. +6. Upload the most recent checkpoint file. -This example uses an altered two-step version of the [full simplified workflow](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley#simplified-approach). +The cron job performs the same steps in an automated fashion: -```sh -apt update && apt install lsb-release sudo postgresql curl wget gpg # debian:11 is surprisingly light +1. Pulls the latest Devnet/Mainnet archive dump and loads it into PostgresQL. +2. Pulls the latest migrated database and loads it into PostgreSQL. +3. Pulls the latest checkpoint file. +4. Runs the **berkeley-migration** app against the two databases. +5. Runs the **replayer app in migration mode** using the downloaded checkpoint file; set the checkpoint interval to be smaller (perhaps 50) because there are typically only 200 or so blocks in a day. +7. Uploads the migrated database. +8. Uploads the most recent checkpoint file. -curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg -echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list -sudo apt-get update && sudo apt-get install google-cloud-sdk +Be sure to monitor the cron job for errors. -sudo rm /etc/apt/sources.list.d/mina*.list -sudo echo "deb [trusted=yes] http://packages.o1test.net $(lsb_release -cs) unstable" | sudo tee /etc/apt/sources.list.d/mina.list -sudo apt-get update && sudo apt-get install --allow-downgrades -y mina-archive-migration=3.0.0-rc1-4277e73 +Just before the Berkeley upgrade, migrate the last few blocks by running locally: -mkdir -p mina-migration-workdir -cd mina-migration-workdir +1. Download the Devnet/Mainnet archive data directly from the k8s PostgreSQL node (not from the archive dump), and load it into PostgreSQL. +2. Download the most recent migrated database and load it into PostgresQL. +3. Download the most recent checkpoint file. +4. Run the **berkeley-migration** app against the two databases. +5. Run the **replayer app in migration mode** using the most recent checkpoint file. -gsutil cp gs://mina_network_block_data/devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz . -tar -xf devnet-bundle-3NKRsRWBzmPR8Z8ZmJb4u8FLpnSkjRitUpKZzVkHp11QuwP5i839.tar.gz +It is worthwhile to perform these last steps as a dry run to make sure all goes well. You can run these steps as many times as needed. -wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql -wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql +## Known migration problems -# this next step is required only if you don't have an archive yet -createdb devnet_balances_migrated -createdb devnet_really_migrated +Please remember that rerunning after crash is always possible. +After solving any of below issues you can rerun process and migration +will continue form last position -psql -d devnet_really_migrated -f create_schema.sql +#### Async was unable to add a file descriptor to its table of open file descriptors +For example: -gsutil cp gs://mina-archive-dumps/devnet-archive-dump-2024-03-22_0000.sql.tar.gz . -tar -xf devnet-archive-dump-2024-03-22_0000.sql.tar.gz -# the next step ensures you don't accidentally merge mainnet and devnet data -sed -i -e s/archive_balances_migrated/devnet_balances_migrated/g devnet-archive-dump-2024-03-22_0000.sql -psql -d devnet_balances_migrated -f devnet-archive-dump-2024-03-22_0000.sql +``` + ("Async was unable to add a file descriptor to its table of open file descriptors" + (file_descr 18) + (error + "Attempt to register a file descriptor with Async that Async believes it is already managing.") + (backtrace + ...... +``` +A remedy is to lower `--block-batch-size` parameter to values up to 500. -mina-berkeley-migration-script initial \ - --genesis-ledger /var/lib/coda/devnet.json \ - --source-db postgres:///devnet_balances_migrated \ - --target-db postgres:///devnet_really_migrated \ - --blocks-batch-size 100 --blocks-bucket mina_network_block_data \ - --network devnet +#### Map.find_exn: not found +For example: +``` +(monitor.ml.Error + (Not_found_s + ("Map.find_exn: not found" +.... +``` -# now, do a final migration +Usually this error means that there is a gap in canonical chain. In order to fix it please +ensure that missing-block-auditor run is successful -gsutil cp gs://mina-archive-dumps/devnet-archive-dump-2024-03-22_2050.sql.tar.gz . -tar -xf devnet-archive-dump-2024-03-22_2050.sql.tar.gz -# the next step ensures you don't accidentally merge mainnet and devnet data -sed -i -e s/archive_balances_migrated/devnet_balances_migrated/g devnet-archive-dump-2024-03-22_2050.sql -psql -d devnet_balances_migrated -f devnet-archive-dump-2024-03-22_2050.sql +#### Yojson.Json_error .. Unexpected end of input -curl -O https://gist.githubusercontent.com/ghost-not-in-the-shell/cfe629a15702e7bae7b0c1415fe0d85e/raw/8d8bff2814c1d0c15deb70b388dea8a28a485184/genesis.json +For example: -mina-berkeley-migration-script final \ - --genesis-ledger /var/lib/coda/devnet.json \ - --source-db postgres:///devnet_balances_migrated \ - --target-db postgres:///devnet_really_migrated \ - --blocks-batch-size 100 --blocks-bucket mina_network_block_data \ - --network devnet \ - --replayer-checkpoint migration-checkpoint-437195.json \ - --fork-state-hash 3NKoUJX87VrfmNAoUdqoWUykVvt66ztm5rzruDQR7ihwYaWsdJKq \ - --fork-config genesis.json \ - --prefetch-blocks +``` +(monitor.ml.Error + ("Yojson.Json_error(\"Line 1, bytes 1003519-1003520:\\nUnexpected end of input\")") + ("Raised at Yojson.json_error in file \"common.ml\", line 5, characters 19-39" + "Called from Yojson.Safe.__ocaml_lex_read_json_rec in file \"lib/read.mll\", line 215, characters 28-52" +... ``` ---- -url: /network-upgrades/berkeley/archive-migration/docker-example ---- +This issue is caused by invalid precomputed block. Deleting the downloaded precomputed blocks should resolve this issue. -# Docker example +#### Error querying db, error: Request to ... failed: ERROR: column \"type\" does not exist -You can follow these steps that can be copy-pasted directly into a OS running Docker. +You provided the migrated schema as source one when invoking script or berkeley-migration app -This example performs a Mainnet initial migration following the [debian-example](/network-upgrades/berkeley/archive-migration/debian-example) +#### Poor performance of migration when accessing remote database -```sh +We conducted migration tests with both a local database and a distant database (RDS). +The migration using the local database appears to process significantly faster. We strongly suggest to use offline database installed locally -# Create a new directory for the migration data -mkdir $(pwd)/mainnet-migration && cd $(pwd)/mainnet-migration +#### ERROR: out of shared memory -# Create Network -docker network create mainnet +``` +(monitor.ml.Error (Failure "Error querying for user commands with id 1686617, error Request to postgresql://user:pwd@host:port/db failed: ERROR: out of shared memory +\nHINT: You might need to increase max_pred_locks_per_transaction +``` -# Launch Local Postgres Database -docker run --name postgres -d -p 5432:5432 --network mainnet -v $(pwd)/mainnet-migration/postgresql/data:/var/lib/postgresql/data -e POSTGRES_USER=mina -e POSTGRES_PASSWORD=minamina -d postgres:13-bullseye +Solution is either to increase `max_pred_locks_per_transaction` setting in postgres database. +Alternative is to isolate database from mainnet traffic (for example by exporting dump from live database and import it on isolated environment) -export PGHOST="localhost" -export PGPORT=5432 -export PGUSER="mina" -export PGPASSWORD="minamina" +#### Berkeley migration app is consuming all of my resources -# Drop DBs if they exist -psql -c "DROP DATABASE IF EXISTS mainnet_balances_migrated;" -psql -c "DROP DATABASE IF EXISTS mainnet_really_migrated;" +When running a full migration, you can stumble on memory leaks that prevent you from cleanly performing the migration in one pass. A machine with 64 GB of RAM can be frozen after ~40k migrated blocks. Each 200 blocks inserted into the database increases the memory leak by 4-10 MB. -# Create DBs -psql -c "CREATE DATABASE mainnet_balances_migrated;" -psql -c "CREATE DATABASE mainnet_really_migrated;" +A potential workaround is to split the migration into smaller parts using cron jobs or automation scripts. -# Retrieve Archive Node Backup -wget https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-2024-04-29_0000.sql.tar.gz -tar -xf mainnet-archive-dump-2024-04-29_0000.sql.tar.gz +## FAQ -# Replace the database name in the dump -sed -i -e s/archive_balances_migrated/mainnet_balances_migrated/g mainnet-archive-dump-2024-04-29_0000.sql -psql mainnet_balances_migrated -f mainnet-archive-dump-2024-04-29_0000.sql +### Migrated database is missing orphaned blocks -# Prepare target -wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/create_schema.sql -wget https://raw.githubusercontent.com/MinaProtocol/mina/berkeley/src/app/archive/zkapp_tables.sql -psql mainnet_really_migrated -f create_schema.sql +By design, Berkeley migration omits orphaned blocks and, by default, migrates only canonical (and pending, if setup correctly) blocks. -# Start migration -docker create --name mainnet-db-migration \ - -v $(pwd)/mainnet-migration:/data \ - --network mainnet minaprotocol/mina-archive-migration:3.0.1-e848ecb-bullseye -- bash -c ' - wget http://673156464838-mina-genesis-ledgers.s3-website-us-west-2.amazonaws.com/mainnet/genesis_ledger.json; mina-berkeley-migration-script initial \ - --genesis-ledger genesis_ledger.json \ - --source-db postgres://mina:minamina@postgres:5432/mainnet_balances_migrated \ - --target-db postgres://mina:minamina@postgres:5432/mainnet_really_migrated \ - --blocks-batch-size 5000 \ - --blocks-bucket mina_network_block_data \ - --checkpoint-output-path /data/checkpoints/. \ - --precomputed-blocks-local-path /data/precomputed_blocks/. \ - --network mainnet' +### Replayer in migration mode overrides my old checkpoints -docker start mainnet-db-migration +By default, the replayer dumps the checkpoint to the current folder. All checkpoint files have a similar format: -docker logs -f mainnet-db-migration +`replayer-checkpoint-{number}.json.` -``` +To prevent override of old checkpoints, use the `--checkpoint-output-folder` and `--checkpoint-file-prefix` parameters to modify the output folder and prefix. --- -url: /network-upgrades/berkeley/archive-migration +url: /network-upgrades/berkeley/archive-migration/understanding-archive-migration --- -# Archive Migration - -The Berkeley upgrade is a major upgrade that requires all nodes in a network to upgrade to a newer version. It is not backward compatible. +# Understanding the Archive Migration -A major upgrade occurs when there are major changes to the core protocol that require all nodes on the network to update to the latest software. +You can reduce risks and effort by reading all of the archive documentation in entirety. -## How to prepare for the Berkeley upgrade +## Archive node migration overview -The Berkeley upgrade requires upgrading all nodes, including archive nodes. One of the required steps is to migrate archive databases from the current Mainnet format to Berkeley. This migration requires actions and efforts from node operators and exchanges. +Archive node migration is a crucial part of the Berkeley upgrade. The current Devnet and Mainnet database format must be converted to the Berkeley format to preserve historical data and assure archive node chain continuity. -Learn about the archive data migration: +For this purpose, the o1Labs and Mina Foundation teams prepared a migration package. -- [Understanding the migration process](/network-upgrades/berkeley/archive-migration/understanding-archive-migration) -- [Prerequisites before migration](/network-upgrades/berkeley/archive-migration/archive-migration-prerequisites) -- [Suggested installation procedure](/network-upgrades/berkeley/archive-migration/archive-migration-installation) -- [How to perform archive migration](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley) +### Archive node Berkeley migration package -Finally, see the shell script example that is compatible with a stock Debian 11 container: +This package contains the required applications to migrate existing Devnet and Mainnet databases into the new Berkeley schema and a usability script: -- [Worked Devnet Debian example using March 22 data](/network-upgrades/berkeley/archive-migration/debian-example) -- [Worked Mainnet Docker example using April 29 data](/network-upgrades/berkeley/archive-migration/docker-example) +1. **berkeley-migration** -## What will happen with original Devnet/Mainnet data + Use the **berkeley-migration** app to migrate as much data as possible from the Devnet/Mainnet database and download precomputed blocks to get the window density data. + + This app runs against the Devnet/Mainnet database and the new Berkeley database. -After the migration, you will have two databases: +2. **replayer app in migration mode** -- The original Devnet/Mainnet database with small data adjustments (all pending blocks from last canoncial block until the fork block are converted to canoncial blocks) -- A new Berkeley database based on Devnet/Mainnet data, but: - - Without Devnet/Mainnet orphaned blocks - - Without pending blocks that are not in the canonical chain - - With all pending blocks on the canonical chain converted to canonical blocks - -There is no requirement to preserve the original Devnet/Mainnet database after migration. However, if for some reason you want to keep the Mainnet orphaned or non-canonical pending blocks, you can download the archive maintenance package for the Devnet/Mainnet database. + The existing replayer application is enhanced with a new migration mode. Use the **replayer app in migration mode** to analyze the transactions in the partially migrated database (resulting from running berkeley-migration app) and populate the `accounts_accessed` and `accounts_created` tables. This app also does the checks performed by the standard replayer, but does not check ledger hashes because the Berkeley ledger has greater depth that results in different hashes. + + This app runs only against the new archive database. -To learn about maintaining archive data, see [Devnet/Mainnet database maintenance](/network-upgrades/berkeley/archive-migration/mainnet-database-maintenance). +3. **berkeley-migration-verifier** ---- -url: /network-upgrades/berkeley/archive-migration/mainnet-database-maintenance ---- + Use the **berkeley-migration-verifier** verification software to determine if the migration (even incomplete) was successful. The app uses SQL validations on the migrated database. -# Devnet/Mainnet database maintenance +4. **end-to-end migration script** -After the Berkeley migration, the original Devnet/Mainnet database is not required unless you are interested in -preserving some aspect of the database that is lost during the migration process. + This shell script wraps all phases and stages of migration into a single script. It is provided purely for node operators usability and is equivalent of running the **berkely-migration** app, the **replayer app in migration mode**, and the **berkeley-migration-verifier** apps in the correct order. -Two databases exist after the successful migration: +### Incrementality -- The original Devnet/Mainnet database with small data adjustments: - - All pending blocks from last canoncial block until the fork block are converted to canonical blocks +Use the **berkeley-migration** and **replayer** apps incrementally so that you can migrate part of the Devnet/Mainnet database, and, as new blocks are added to the Devnet/Mainnet databases, the new data can be migrated. -- A new Berkeley database based on Devnet/Mainnet data with these differences: - - Without Devnet/Mainnet orphaned blocks - - Without pending blocks that are not in the canonical chain - - With all pending blocks on the canonical chain converted to canonical blocks +To obtain that incrementality, the **berkeley-migration** app looks at the migrated database and determines the most recent migrated block. It continues migration starting at the next block in the Devnet/Mainnet data. The **replayer app in migration mode** uses the checkpoint mechanism already in place for the replayer. A checkpoint file indicates the global slot since genesis for starting the replay and the ledger to use for that replay. New checkpoint files are written as it proceeds. -The o1Labs and Mina Foundation teams have consistently prioritized rigorous testing and the delivery of high-quality software products. +To take advantage of the incrementality, run a cron job that migrates a day's worth of data at a time (or some other interval). With the cron job in place, at the time of the actual Berkeley upgrade, you will need to migrate only a small amount of data. -However, being human entails the possibility of making mistakes. +--- +url: /network-upgrades/berkeley/flags-configs +--- -## Known issues +# Post-Upgrade Flags and Configurations for Mainnet -Recently, a few mistakes were identified while working on a version of Mina used on Mainnet. These issues were promptly addressed; however, within the decentralized environment, archive nodes can retain historical issues despite our best efforts. +Please refer to the Berkeley node release notes [here](https://github.com/MinaProtocol/mina/releases/tag/3.0.3). -Fixes are available for the following known issues: +### Network details -- **Missing or invalid nonces** - a historical issue skewed nonces in the `balances` table. Although the issue was resolved, you might still have nonces that are missing or invalid. -- **Incorrect ledger hashes** - a historical issue with the same root cause as 'Missing or invalid nonces'. However, the outcome is that a 'replayer run' operation of validating archive node against daemon ledger shows ledger mismatches and cannot pass problematic blocks. -- **Missing blocks** - This recurring missing blocks issue consistently poses challenges and is a source of concern for all archive node operators. This persistent challenge from disruptions in daemon node operations can potentially lead to incomplete block reception by archive nodes. This situation can compromise chain continuity within the archive database. +``` +Chain ID +a7351abc7ddf2ea92d1b38cc8e636c271c1dfd2c081c637f62ebc2af34eb7cc1 -To address these issues, install and use the special archive node maintenance package that includes fixes. +Git SHA-1 +ae112d3a96fe71b4ccccf3c54e7b7494db4898a4 -## Installing the archive node maintenance package +Seed List +https://bootnodes.minaprotocol.com/networks/mainnet.txt -The package provides support for codenames: +Node build +https://github.com/MinaProtocol/mina/releases/tag/3.0.3 +``` -- bullseye -- buster -- focal +### Block Producer's -The following steps describe only the bullseye package installation. Modify the steps as appropriate for your environment. +Start your node post-upgrade in Mainnet with the flags and environment variables listed below. -### Debian packages +``` +mina daemon +--block-producer-key +--config-directory +--file-log-rotations 500 +--generate-genesis-proof true +--libp2p-keypair +--log-json +--peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt -To get the Debian package: +ENVIRONMENT VARIABLES +RAYON_NUM_THREADS=6 +MINA_LIBP2P_PASS +MINA_PRIVKEY_PASS +``` -```sh -CODENAME=bullseye -CHANNEL=stable -VERSION=1.4.1 +### SNARK Coordinator +Configure your node post-upgrade in Mainnet with specific flags and environment variables as listed. -echo "deb [trusted=yes] http://packages.o1test.net $CODENAME $CHANNEL" | tee /etc/apt/sources.list.d/mina.list -apt-get update -apt-get install --allow-downgrades -y "mina-archive-maintenance=$VERSION" ``` +mina daemon +--config-directory +--enable-peer-exchange true +--file-log-rotations 500 +--libp2p-keypair +--log-json +--peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt +--run-snark-coordinator +--snark-worker-fee 0.001 +--work-selection [seq|rand|roffset] -### Docker image +ENVIRONMENT VARIABLES +MINA_LIBP2P_PASS +``` -To get the Docker image: +### SNARK Workers +Connect to a SNARK Coordinator node if required and run the following flags. +``` +mina internal snark-worker +--proof-level full +--shutdown-on-disconnect false +--daemon-address -```sh -docker pull minaprotocol/mina-archive-maintenance:1.4.1-060f0a5-bullseye +ENVIRONMENT VARIABLES +RAYON_NUM_THREADS:8 ``` -## Usage for missing or invalid nonces +### Archive Node +Running an Archive Node involves setting up a non-block-producing node and a PostgreSQL database configured with specific flags and environment variables. -The replayer application was developed to verify the Devnet/Mainnet archive data. You must run the replayer application against your existing Devnet/Mainnet database to verify the blockchain state. +For more information about running archive nodes, see [Archive Node](/node-operators/archive-node). -To run the replayer application: +The PostgreSQL database requires two schemas: +1. The PostgreSQL schema used by the Mina archive database: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) +2. The PostgreSQL schema extensions to support zkApp commands: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) -```sh -mina-replayer \ - --archive-uri {db_connection_string} \ - --input-file reference_replayer_input.json \ - --output-file replayer_input_file.json \ - --checkpoint-interval 10000 \ - --fix-nonces \ - --set-nonces \ - --dump-repair-script +The non-block-producing node must be configured with the following flags: ``` +mina daemon +--archive-address : +--config-directory +--enable-peer-exchange true +--file-log-rotations 500 +--generate-genesis-proof true +--libp2p-keypair +--log-json +--peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt -where: +ENVIRONMENT VARIABLES +MINA_LIBP2P_PASS +``` -- `archive-uri` - connection string to the archive database -- `input-file` - JSON file that holds the archive database -- `output-file` - JSON file that will hold the ledger with auxiliary information, like global slot and blockchain height, which will be dumped on the last block -- `checkpoint-interval` - frequency of checkpoints expressed in blocks count -- `replayer_input_file.json` - JSON file constructed from the Devnet/Mainnet genesis ledger: +This non-block-producing node connects to the archive node with the addresses and port specified in the `--archive-address` flag. - ```sh - jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json - ``` +The **archive node** command looks like this: -- `--fix-nonces` - adjust nonces values while replaying transactions -- `--set-nonces` - set missing nonces while replaying transactions -- `--dump-repair-script` - path to the output SQL script that will contain all updates to nonces made during the replayer run that can be directly applied to other database instances that contain the same data with invalid nonces +``` +mina-archive run +--metrics-port +--postgres-uri postgres://:@
:/ +--server-port 3086 +--log-json +--log-level DEBUG +``` -Running a replayer from scratch on a Devnet/Mainnet database can take up to a couple of days. The recommended best practice is to break the replayer into smaller parts by using the checkpoint capabilities of the replayer. -Additionally, running the replayer can exert significant demands on system resources that potentially affect the performance of the archive node. Because of the large resource requirements, we recommend that you execute the replayer in isolation from network connections, preferably within an isolated environment where the Devnet/Mainnet dumps can be imported. +### Rosetta API +Once you have the Archive Node stack up and running, start the Rosetta API Docker image with the following command: -## Bad ledger hashes +``` +docker run +--name rosetta --rm \ +-p 3088:3088 \ +--entrypoint '' \ +minaprotocol/mina-rosetta:3.1.0-ae112d3-bullseye-mainnet \ +/usr/local/bin/mina-rosetta \ +--archive-uri "${PG_CONNECTION_STRING}" \ +--graphql-uri "${GRAPHQL_URL}" \ +--log-json \ +--log-level ${LOG_LEVEL} \ +--port 3088 +``` -There is no ultimate fix for this issue because preserving historical ledger hashes is essential to the overall security of the Mina network. Even with this issue, you can validate archive data integrity. +--- +url: /network-upgrades/berkeley/requirements +--- -The replayer application has a built-in mechanism to skip errors when the `--continue-on-error` flag is enabled. -However, instead of skipping only blocks with bad ledger hashes, this mode skipped all of the problems with integrity. -With the new archive node maintenance package, you can run the replayer application without a special flag and to correctly handle the bad ledger hashes issue. +# Requirements -To run replayer: +## Hardware Requirements -```sh -mina-replayer --archive-uri {db_connection_string} --input-file reference_replayer_input.json --output-file reference_replayer_output.json --checkpoint-interval 10000 -``` +Please note the following are the hardware requirements for each node type after the upgrade: -where: +| Node Type | Memory | CPU | Storage | Network | +|--|--|--|--|--| +| Mina Daemon Node | 32 GB RAM | 8 core processor with BMI2 and AVX CPU instruction set are required | 64 GB | 1 Mbps Internet Connection | +| SNARK Coordinator | 32 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | +| SNARK Worker | 32 GB RAM | 4 core/8 threads per worker with BMI2 and AVX CPU instruction set are required | 64 GB | 1 Mbps Internet Connection | +| Archive Node | 32 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | +| Rosetta API standalone Docker image | 32 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | +| Mina Seed Node | 64 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | -- `archive-uri` - connection string to the archive database -- `input-file` - JSON file that holds the archive database -- `output-file` - JSON file that will hold the ledger with auxiliary information, like global slot and blockchain height, which will be dumped on the last block -- `checkpoint-interval` - frequency of checkpoints expressed in blocks count -- `replayer_input_file.json` - JSON file constructed from the Devnet/Mainnet genesis ledger: +## Mina Daemon Requirements - ``` - jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json - ``` +### Installation -:warning: Running a replayer from scratch on a Devnet/Mainnet database can take up to a couple of days. The recommended best practice is to break the replayer into smaller parts by using the checkpoint capabilities of the replayer. +:::caution -:warning: You must run the replayer using the Mainnet version. You can run it from the Docker image at `minaprotocol/mina-archive:3.1.0-ae112d3-bullseye`. +If you have `mina-generate-keypair` installed, you will need to first `sudo apt remove mina-generate-keypair` before installing `mina-mainnet=3.1.0-ae112d3`. +The `mina-generate-keypair` binary is now installed as part of the mina-mainnet package. -## Missing blocks +::: -The daemon node unavailability can cause the archive node to miss some of the blocks. This recurring missing blocks issue consistently poses challenges. To address this issue, you can reapply missing blocks. +### IP and Port configuration -If you uploaded the missing blocks to Google Cloud, the missing blocks can be reapplied from precomputed blocks to preserve chain continuity. +**IP:** -1. To automatically verify and patch missing blocks, use the [download-missing-blocks.sh](https://github.com/MinaProtocol/mina/blob/berkeley/scripts/archive/download-missing-blocks.sh) script. +By default, the Mina Daemon will attempt to retrieve its public IP address from the system. If you are running the node behind a NAT or firewall, you can set the `--external-ip` flag to specify the public IP address. - The `download-missing-blocks` script uses `localhost` as the database host so the script assumes that psql is running on localhost on port 5432. Modify `PG_CONN` in `download_missing_block.sh` for your environment. +**Port:** -1. Install the required `mina-archive-blocks` and `mina-missing-blocks-auditor` scripts that are packed in the `minaprotocol/mina-archive:3.1.0-ae112d3-bullseye` Docker image. +Nodes must expose a port publicly to communicate with other peers. +Mina uses by default the port `8302` which is the default libp2p port. -1. Export the `BLOCKS_BUCKET`: +You can use a different port by setting the `--external-port` flag. - ```sh - export BLOCKS_BUCKET="https://storage.googleapis.com/my_bucket_with_precomputed_blocks" - ``` +### Node Auto-restart -1. Run the `mina-missing-blocks-auditor` script from the database host: +Ensure your nodes are set to restart automatically after a crash. For guidance, refer to the [auto-restart instructions](/node-operators/validator-node/connecting-to-the-network#running-mina-node-as-a-service) - For Devnet: +## Seed Peer Requirements - ```sh - download-missing-blocks.sh devnet {db_user} {db_password} - ``` +### Generation of libp2p keypair - For Mainnet: +To ensure connectivity across the network, it is essential that all seed nodes start with the **same** `libp2p` keypair. +This consistency allows other nodes in the network to reliably connect. +Although the same libp2p keys can be reused from before the upgrade, if you need to manually generate new libp2p keys, use the following command: - ```sh - download-missing-blocks.sh mainnet {db_user} {db_password} - ``` -### Using precomputed blocks from O1labs bucket - -O1labs maintains a Google bucket containing precomputed blocks from Devnet and Mainnet, accessible at https://storage.googleapis.com/mina_network_block_data/. - -Note: It's important to highlight that precomputed blocks for **Devnet** between heights `2` and `1582` have missing fields or incorrect transaction data. Utilizing these blocks to patch your Devnet archive database will result in failure. For those who rely on precomputed blocks from this bucket, please follow the outlined steps: - -1. Download additional blocks from `gs://mina_network_block_data/devnet-extensional-bundle.tar.gz`. -2. Install the necessary `mina-archive-blocks` script contained within the `minaprotocol/mina-archive:3.1.0-ae112d3-bullseye` Docker image. -3. Execute mina-archive-blocks to import the extracted blocks from step 1 using the provided command: - - ```sh - mina-archive-blocks --archive-uri --extensional ./extensional/* - ``` -4. Proceed with patching your Devnet database with blocks having heights other than `2` to `1582` using the available precomputed blocks. - -## Next steps +``` +mina libp2p generate-keypair --privkey-path +``` -Now that you have completed the steps to properly maintain the correctness of the archive database, you are ready to perform the archive [migration process](/network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley). +Further information on [generating key pairs](/node-operators/validator-node/generating-a-keypair) on Mina Protocol. --- -url: /network-upgrades/berkeley/archive-migration/migrating-archive-database-to-berkeley +url: /network-upgrades/berkeley/upgrade-steps --- -# Migrating Devnet/Mainnet Archive to Berkeley Archive +# Upgrade Steps -Before you start the process to migrate your archive database from the current Mainnet or Devnet format to Berkeley, be sure that you: + Mainnet Upgrade steps -- [Understand the Archive Migration](/network-upgrades/berkeley/archive-migration/understanding-archive-migration) -- Meet the foundational requirements in [Archive migration prerequisites](/network-upgrades/berkeley/archive-migration/archive-migration-prerequisites) -- Have successfully installed the [archive migration package](/network-upgrades/berkeley/archive-migration/archive-migration-installation) +Below it's the description in detail of all the upgrade steps and what which node operator type should do to in each step. -## Migration process +## Pre-Upgrade -The Devnet/Mainnet migration can take up to a couple of days. -Therefore, you can achieve a successful migration by using three stages: +- During the Pre-Upgrade phase, node operators should prepare for the upcoming upgrade. The most important steps are: + - Review the [upgrade readiness checklist](https://docs.google.com/document/d/1rTmJvyaK33dWjJXMOSiUIGgf8z7turxolGHUpVHNxEU/edit#heading=h.2hqz0ixwjk3f) to confirm they have covered the required steps. + - Upgrade their nodes to the 1.4.1 stable version + - Ensure servers are provisioned to run Berkeley nodes, meeting the new hardware requirements + - Upgrade their nodes to the node version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3), with stop-slots, when this version becomes available + - Start the archive node initial migration if they run archive nodes and wish to perform the migration in a decentralized manner -- **Stage 1:** Initial migration +**Please note:** a simplified Node Status service will be part of the upgrade tooling and enabled by default in Pre-Upgrade release with the stop-slots ([3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3)). This feature will allow for a safe upgrade by monitoring the amount of upgraded active stake. Only non-sensitive data will be reported. If operators are not comfortable sharing their node version, they will have the option to disable the node version reports by using the appropriate node flag `--node-stats-type none` -- **Stage 2:** Incremental migration +### Block Producers and SNARK Workers +1. Review the [upgrade readiness checklist](https://docs.google.com/document/d/1rTmJvyaK33dWjJXMOSiUIGgf8z7turxolGHUpVHNxEU). +1. Provision servers that meet the minimum hardware requirements, including the new 32GB RAM requirement and support for _AVX_ and _BMI2_ CPU instructions. +1. Upgrade nodes to node version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) ([3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) has built-in stop slots). -- **Stage 3:** Remainder migration +### Archive Node Operators and Rosetta Operators +- Two migration processes will be available to archive node operators: _trustless_ and _trustful_. If the archive node operator wants to perform the _trustless_ migration, they should follow these steps; otherwise, proceed to the Upgrade phase. The _trustful_ migration will rely on o1Labs database exports and Docker images to migrate the archive node database and doesn’t require any actions at this stage. -Each stage has three migration phases: +1. Trustless migration: + - Perform the initial archive node migration. Since Mainnet is a long-lived network, the initial migration process can take up to 48 hours, depending on your server specification and infrastructure. + - If your Mina Daemon, archive node, or PostgreSQL database runs on different machines, the migration performance will be greatly impacted. + - For more information on the archive node migration process, please refer to the [Archive Migration](/network-upgrades/berkeley/archive-migration) section. +2. Upgrade all nodes to the latest stable version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3). +3. Provision servers that meet the minimum hardware requirements, primarily the new 32GB RAM requirement. +4. Upgrade their nodes to the version that includes built-in stop slots before the pre-defined _stop-transaction-slot_. -- **Phase 1:** Copying data and precomputed blocks from Devnet/Mainnet database using the **berkeley_migration** app. +### Exchanges +1. Make sure to test your system integration with Berkeley's new features. Pay special attention to: + - If you use the **o1labs/client-sdk** library to sign transactions, you should switch to **[mina-signer](https://www.npmjs.com/package/mina-signer)**. o1labs/client-sdk was **deprecated** some time ago and will be **unusable** once the network has been upgraded. Please review the migration instructions in [Appendix](/network-upgrades/berkeley/appendix). + - If you rely on the archive node SQL database tables, please review the schema changes in Appendix 1 of this document. +2. Upgrade all nodes to the latest stable version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3). +3. Provision servers that meet the minimum hardware requirements, particularly the new 32GB RAM requirement. +4. Upgrade your nodes to the version that includes built-in stop slots before the pre-defined _stop-transaction-slot_. -- **Phase 2:** Populating new Berkeley tables using the **replayer app in migration mode** +*** -- **Phase 3:** Additional validation for migrated database +## State Finalization +- Between the predefined _stop-transaction-slot_ and _stop-network-slot_, a stabilization period of 100 slots will occur. During this phase, the network consensus will not accept new blocks with transactions on them, including coinbase transactions. The state finalization period ensures all nodes reach a consensus on the latest network state before the upgrade. +- During the state finalization slots, it is crucial to maintain a high block density. Therefore, block producers and SNARK workers shall continue running their nodes to support the network's stability and security. +- Archive nodes should also continue to execute to ensure finalized blocks are in the database and can be migrated, preserving the integrity and accessibility of the network's history. -Review these phases and stages before you start the migration. +### Block Producers and SNARK Workers +1. It is crucial for the network's successful upgrade that all block producers and SNARK workers maintain their block-producing nodes up and running throughout the state finalization phase. +2. If you are running multiple daemons like is common with many operators, you can run one single node at this stage. +3. If you are a Delegation Program operator, remember that your uptime data will continue to be tracked during the state finalization phase and will be considered for the delegation grant in the following epoch. -## Simplified approach +### Archive Node Operators and Rosetta Operators +**If you plan to do the _trustful_ migration, you can skip this step.** +If you are doing the trustless migration, then: +1. Continue to execute the archive node to ensure finalized blocks are in the database and can be migrated. +2. Continue to run incremental archive node migrations until after the network stops at the stop-network slot. +3. For more information on the archive node migration process, please refer to the [Archive Migration](/network-upgrades/berkeley/archive-migration) section -For convenience, use the `mina-berkeley-migration-script` app if you do not need to delve into the details of migration or if your environment does not require a special approach to migration. +### Exchanges -### Stage 1: Initial migration +Exchanges shall disable MINA deposits and withdrawals during the state finalization period (the period between _stop-transaction-slot_ and _stop-network-slot_) since any transactions after the _stop-transaction-slot_ will not be part of the upgraded chain. -``` -mina-berkeley-migration-script \ - initial \ - --genesis-ledger ledger.json \ - --source-db postgres://postgres:postgres@localhost:5432/source \ - --target-db postgres://postgres:postgres@localhost:5432/migrated \ - --blocks-bucket mina_network_block_data \ - --blocks-batch-size 500 \ - --checkpoint-interval 10000 \ - --checkpoint-output-path . \ - --precomputed-blocks-local-path . \ - --network NETWORK -``` +Remember that although you might be able to submit transactions, the majority of the block producers will be running a node that discards any blocks with transactions. -where: +*** -`-g | --genesis-ledger`: path to the genesis ledger file +## Upgrade -`-s | --source-db`: connection string to the database to be migrated +- Starting at the _stop-network-slot_ the network will not produce nor accept new blocks, resulting in halting the network. During the upgrade period, o1Labs will use automated tooling to export the network state based on the block at the slot just before the _stop-transaction-slot_. The exported state will then be baked into the new Berkeley build, which will be used to initiate the upgraded network. It is during the upgrade windows that the Berkeley network infrastructure will be bootstrapped, and seed nodes will become available. o1Labs will also finalize the archive node migration and publish the PostgreSQL database dumps for import by the archive node operators who wish to bootstrap their archives in a trustful manner. +- There is a tool available to validate that the Berkeley node was built from the pre-upgrade network state. To validate, follow the instructions provided in this [location](https://github.com/MinaProtocol/mina/blob/berkeley/docs/upgrading-to-berkeley.md) -`-t | --target-db`: connection string to the database that will hold the migrated data +### Block Producers and SNARK Workers +1. During the upgrade phase (between _stop-network-slot_ and the publishing of the Berkeley release), block producers can shut down their nodes. +2. After the publication of the Berkeley node release, block producers and SNARK workers should upgrade their nodes and be prepared for block production at the genesis timestamp, which is the slot when the first Berkeley block will be produced. +3. It is possible to continue using the same libp2p key after the upgrade. Remember to adjust the new flag to pass the libp2p key to the node. -`-b | --blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +### Archive Node Operators and Rosetta Operators +1. Upon publishing the archive node Berkeley release, archive node operators and Rosetta operators should upgrade their systems. +There will be both Docker images and archive node releases available to choose from. +2. Depending on the chosen migration method: + - _Trustless_ + - Operators should direct their Berkeley archive process to the previously migrated database. + - _Trustful_ + - Operators shall import the SQL dump file provided by o1Labs to a freshly created database. + - Operators should direct their Berkeley archive process to the newly created database. -`-bs | --blocks-batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up the migration process. +**Please note:** both the _trustless_ and _trustful_ migration processes will discard all Mainnet blocks that are not canonical. If you wish to preserve the entire block history, i.e. including non-canonical blocks, you should maintain the Mainnet archive node database for posterior querying needs. -`-n | --network`: network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json`. +### Exchanges +1. Exchanges shall disable MINA deposits and withdrawals during the entirety of the upgrade downtime, since the _stop-transaction-slot_ until the Mainnet Berkeley network is operational. +2. After the Berkeley releases are published, exchanges should upgrade their nodes and prepare for the new network to start block production. -`-c | --checkpoint-output-path`: path to folder for replayer checkpoint files +*** -`-i | --checkpoint-interval`: frequency of dumping checkpoint expressed in blocks count +## Post-Upgrade +- At approximately 1 hour after the publishing of the Berkeley node release, at a predefined slot (Berkeley genesis timestamp), block production will start, and the network is successfully upgraded. +- Node operators can monitor their nodes and provide feedback to the technical team in case of any issues. Builders can start deploying zkApps. +- **Please note:** The Node Status service will not be enabled by default in the Berkeley release. If you wish to provide Node Status and Error metrics and reports to Mina Foundation, helping monitor the network in the initial phase, please use the following flags when running your nodes: + - `--node-stats-type [full|simple]` + - `--node-status-url https://nodestats.minaprotocol.com/submit/stats` + - `--node-error-url https://nodestats.minaprotocol.com/submit/stats` + - The error collection service tries to report any node crashes before the node process is terminated -`-l | --precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location +### Block Producers and SNARK Workers +1. Ensure that all systems have been upgraded and prepared for the start of block production. +2. Monitor nodes and network health, and provide feedback to the engineering team in case of any issues. -The command output is the `migration-replayer-XXX.json` file required for the next run. +### Archive Node Operators and Rosetta Operators +1. Ensure that all systems have been upgraded and prepared for the start of block production. +2. Monitor nodes and network health, and provide feedback to the engineering team in case of any issues. -### Stage 2: Incremental migration +### Exchange and Builders +1. After the predefined Berkeley genesis timestamp, block production will commence, and MINA deposits and withdrawals can be resumed. +2. Ensure that all systems have been upgraded and prepared for the start of block production. +3. Monitor nodes and network health, and provide feedback to the engineering team in case of any issues. -``` -mina-berkeley-migration-script \ - incremental \ - --genesis-ledger ledger.json \ - --source-db postgres://postgres:postgres@localhost:5432/source \ - --target-db postgres://postgres:postgres@localhost:5432/migrated \ - --blocks-bucket mina_network_block_data \ - --blocks-batch-size 500 \ - --network NETWORK \ - --checkpoint-output-path . \ - --checkpoint-interval 10000 \ - --precomputed-blocks-local-path . \ - --replayer-checkpoint migration-checkpoint-XXX.json -``` +--- +url: /network-upgrades +--- -where: +# Network Upgrades -`-g | --genesis-ledger`: path to the genesis ledger file +Mina protocol evolves through network upgrades (hard forks) that introduce new features and improvements. Each upgrade requires node operators to update their software to remain compatible with the network. -`-s | --source-db`: connection string to the database to be migrated +:::tip Mesa preflight hard fork completed -`-t | --target-db`: connection string to the database that will hold the migrated data +The Mesa preflight network hard-forked at **2026-04-27 13:00 UTC**. The post-fork chain is now running the Mesa release **`4.0.0-preflight-3f038cb`**, which raises the **transaction protocol version to 5.0.0**. -`-b | --blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +Operators must run **`4.0.0-preflight-3f038cb`** to remain on the preflight network. Earlier builds — including the stop-slot release `4.0.0-preflight-stop-2967b39` — are no longer compatible with the post-fork chain. -`-bs | --blocks-batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. +zkApp developers must update to **`o1js@3.0.0-mesa.698ca`**, which targets transaction protocol v5.0.0; earlier o1js releases will produce transactions that the post-fork network rejects. -`-n | --network`: network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json`. +See [Preflight Network](/network-upgrades/mesa/preflight-network) for the full upgrade path. -`-r | --replayer-checkpoint`: path to the latest checkpoint file `migration-checkpoint-XXX.json` +::: -`-c | --checkpoint-output-path`: path to folder for replayer checkpoint files +| Upgrade | Status | Date | Key Changes | +|---------|--------|------|-------------| +| [Berkeley](/network-upgrades/berkeley/requirements) | Completed | June 2024 | zkApp programmability, recursive proofs, new transaction model, archive database migration | +| [Mesa](/network-upgrades/mesa/preflight-network) | Preflight hard fork completed | 2026-04-27 13:00 UTC (preflight) | Transaction protocol v5.0.0, automode upgrades, simplified archive migration | -`-i | --checkpoint-interval`: frequency of dumping checkpoint expressed in blocks count +--- +url: /network-upgrades/mesa/archive-upgrade +--- -`-l | --precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location +# Archive Upgrade -### Stage 3: Remainder migration +This guide describes the general procedure for upgrading a Mina archive database from Berkeley to Mesa. The same steps apply to every Mesa deployment (preflight, devnet, mainnet) — only the archive package version differs. -``` -mina-berkeley-migration-script \ - final \ - --genesis-ledger ledger.json \ - --source-db postgres://postgres:postgres@localhost:5432/source \ - --target-db postgres://postgres:postgres@localhost:5432/migrated \ - --blocks-bucket mina_network_block_data \ - --blocks-batch-size 500 \ - --network NETWORK \ - --checkpoint-output-path . \ - --checkpoint-interval 10000 \ - --precomputed-blocks-local-path . \ - --replayer-checkpoint migration-checkpoint-XXX.json \ - -fc fork-genesis-config.json -``` +:::tip Mesa preflight hard fork completed -where: +The Mesa preflight network hard-forked at **2026-04-27 13:00 UTC**. The post-fork chain runs **`4.0.0-preflight-3f038cb`**, which raises the **transaction protocol version to 5.0.0**. Pre-fork builds — including the stop-slot release `4.0.0-preflight-stop-2967b39` — are no longer compatible. -`-g | --genesis-ledger`: path to the genesis ledger file +When upgrading a preflight archive database, install the matching `mina-archive-mesa=4.0.0-preflight-3f038cb` package so the archive node speaks the new transaction protocol. -`-s | --source-db`: connection string to the database to be migrated +::: -`-t | --target-db`: connection string to the database that will hold the migrated data +:::info Choosing the Mesa archive version -`-b | --blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +The commands below use `` as a placeholder for the archive build you want to install. Substitute the tag that matches your target network: -`-bs | --blocks-batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up the migration process. +- **Preflight network** — use `4.0.0-preflight-3f038cb` (post-hard-fork; transaction protocol v5.0.0). See [Preflight Network](./preflight-network) for the full release matrix. +- **Devnet / mainnet** — use the Mesa release tag published for that network when it is announced. -`-n | --network`: network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json`. +::: -`-r | --replayer-checkpoint`: path to the latest checkpoint file `migration-checkpoint-XXX.json` +To successfully upgrade the archive database to Mesa, you must ensure that your environment meets the foundational requirements. -`-c | --checkpoint-output-path`: path to folder for replayer checkpoint files +## Migration host -`-i | --checkpoint-interval`: frequency of dumping checkpoint expressed in blocks count +- PostgreSQL database for database server +- If you use Docker, then any of the supported OS by Mina (bullseye, focal, noble, bookworm or jammy) with at least 32 GB of RAM +- gsutil application from Google Cloud Suite in version 5 or later +- (Optional) Docker in version 23.0 or later -`-l | --precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location +## Archive database -`-fc | --fork-config`: fork genesis config file is the new genesis config that is distributed with the new daemon and is published after the fork block is announced +One of the most obvious prerequisites is a Mainnet database. If you don't have an existing database with Devnet/Mainnet archive data, +you can always download it from the O1Labs Google Cloud bucket. -## Advanced approach +## Upgrade process -If the simplified berkeley migration script is, for some reason, not suitable for you, it is possible to run the migration using the **berkeley_migration** and **replayer** apps without an interface the script provides. +### Upgrade script -### Stage 1: Initial migration +Assuming that you have a PostgreSQL database with Mainnet archive data, in order to upgrade it to Mesa version, you need to run SQL upgrade script. +We put all efforts to make the upgrade process as smooth as possible. Script can be run on archive node which is online or offline. +Script can be run multiple times, it will skip steps that were already completed. It also performs sanity checks before each step to ensure that the upgrade process is successful. +Finally it creates new table (version) in the database to keep track of the upgrade process. -This first stage requires only the initial Berkeley schema, which is the foundation for the next migration stage. This schema populates the migrated database and creates an initial checkpoint for further incremental migration. +#### Getting the script -- Inputs - - Unmigrated Devnet/Mainnet database - - Devnet/Mainnet genesis ledger - - Empty target Berkeley database with the schema created, but without any content +You can find the SQL upgrade script in the Mina repository on GitHub. Make sure to download the latest version of the script before proceeding. +You can download the script directly using the following command: -- Outputs - - Migrated Devnet/Mainnet database to the Berkeley format from genesis up to the last canonical block in the original database - - Replayer checkpoint that can be used for incremental migration +```bash +curl -O https://raw.githubusercontent.com/MinaProtocol/mina/refs/heads/mesa/src/app/archive/upgrade_to_mesa.sql +``` -#### Phase 1: Berkeley migration app run +We also ship the script in the Mina archive Docker image and Debian package. +```bash +docker run --rm gcr.io/o1labs-192920/mina-archive:-bookworm-mesa cat /etc/mina/archive/upgrade-to-mesa.sql > upgrade-to-mesa.sql ``` -mina-berkeley-migration \ - --batch-size 1000 \ - --config-file ledger.json \ - --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ - --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ - --blocks-bucket mina_network_block_data \ - --precomputed-blocks-local-path . \ - --keep-precomputed-blocks \ - --network NETWORK + +```bash +# Setup the Mina repository and install the archive package. +# Repository setup (apt sources, GPG key, channel) is network-specific — +# see Preflight Network for the preflight pin, or your network's release notes. + +apt-get install mina-archive-mesa= + +# View the upgrade and downgrade scripts +cat /etc/mina/archive/upgrade-to-mesa.sql +cat /etc/mina/archive/downgrade-to-berkeley.sql ``` -where: +#### Running the script -`--batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. +:::caution Database Backup -`--config-file`: path to the genesis ledger file +Before running the upgrade script, **backup your archive database**. The upgrade modifies the database schema. -`--mainnet-archive-uri`: connection string to the database to be migrated +```bash +pg_dump -U > berkeley-archive-backup.sql +``` -`--migrated-archive-uri`: connection string to the database that will hold the migrated data +::: -`--blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +To run the upgrade script, execute the following command: -`--precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location +```bash +psql -U -d -f upgrade-to-mesa.sql +``` -`--keep-precomputed-blocks`: keep the precomputed blocks on-disk after the migration is complete +Make sure to replace `` and `` with your actual PostgreSQL username and database name. -`--network`: the network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` -#### Phase 2: Replayer in migration mode run +#### Rollback -Replayer config must contain the Devnet/Mainnet ledger as the starting point. So first, you must prepare the replayer config file: +You can rollback the upgrade process by restoring the database from a backup taken before running the upgrade script. +Another is to run rollback script which is part of the upgrade script. It will drop all tables and other database objects created by the upgrade script. +It will also update the version table to reflect the rollback. -``` - jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json +##### Running the rollback script + +To run the rollback script, you need to execute the following command: + +```bash +psql -U -d -f /etc/mina/archive/downgrade-to-berkeley.sql ``` -where: +Make sure to replace `` and `` with your actual PostgreSQL username and database name. - `genesis_ledger.json` is the genesis file from a daemon bootstrap on a particular network -Then: -``` - mina-migration-replayer \ - --migration-mode \ - --archive-uri postgres://postgres:postgres@localhost:5432/migrated \ - --input-file replayer_input_config.json \ - --checkpoint-interval 10000 \ - --checkpoint-output-folder . -``` +### Post-upgrade steps -where: +After successfully running the upgrade script, you DO NOT need to restart your archive node or Rosetta API. +Changes in upgrade script are backward compatible and will be picked up by the archive node and Rosetta API automatically. -`--migration-mode`: flag for migration +### Verification +To verify that the upgrade was successful, you can check the version table in the PostgreSQL database. -`--archive-uri`: connection string to the database that will hold the migrated data +You can do this by running the following command: +```bash +psql -U -d -c "SELECT * FROM version;" +``` +Make sure to replace `` and `` with your actual PostgreSQL username and database name. -`--input-file`: path to the replayer input file, see below on how's created +If the upgrade was successful, you should see the new version number in the output. -`replayer_input_config.json`: is a file constructed out of network genesis ledger: - ``` - jq '.ledger.accounts' genesis_ledger.json | jq '{genesis_ledger: {accounts: .}}' > replayer_input_config.json - ``` +We put a lot of effort into making the upgrade process as smooth as possible. +However, if you encounter any issues or need assistance, please reach out to the Mina community on [Discord](https://discord.gg/minaprotocol) or [GitHub Discussions](https://github.com/MinaProtocol/mina/discussions). -`--checkpoint-interval`: frequency of checkpoints file expressed in blocks count +## Appendix: Database Schema Changes -`--checkpoint-output-folder`: path to folder for replayer checkpoint files +Below we present details of what was changed in the archive node database schema between Berkeley and Mesa versions. -#### Phase 3: Validations +### Zkapp_state_nullable Additional Columns -Use the **berkeley_migration_verifier** app to perform checks for both the fully migrated and partially migrated databases. +The `zkapp_state_nullable` table has been modified to include new columns `element8` through `element31` which are nullable and can store additional state information for zkApps. -``` - mina-berkeley-migration-verifier \ - pre-fork \ - --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ - --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated +```sql +, element8 int REFERENCES zkapp_field(id) +... +, element31 int REFERENCES zkapp_field(id) +); ``` -where: +This expansion allows zkApps to store up to 31 state elements instead of the previous 8, significantly increasing the state storage capacity for complex smart contracts. -`--mainnet-archive-uri`: connection string to the database to be migrated +### Version Table -`--migrated-archive-uri`: connection string to the database that will hold the migrated data +We also introduced a new table `version` to keep track of the database schema version. +The purpose of this table is to help with future database migrations. The table tracks which migration scripts were applied and when. +Ultimately it helps to determine the current version of the database schema and helps to avoid applying the same migration script multiple times. -### Stage 2: Incremental migration +This table is created if it does not exist already. Rollback and upgrade scripts will insert a new row with the version number and timestamp when the script was applied. -After the initial migration, the data is migrated data up to the last canonical block. However, Devnet/Mainnet data is progressing with new blocks that must also be migrated again and again until the fork block is announced. +```sql +CREATE TABLE IF NOT EXISTS version ( + version_num INT PRIMARY KEY, + applied_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP +); +``` -:::info -Incremental migration can, and probably must, be repeated a couple of times until the fork block is announced by Mina Foundation. -Run the incremental migration multiple times with the latest Devnet/Mainnet database and the latest replayer checkpoint file. -::: +The version table provides: +- **Migration tracking**: Records which migrations have been applied +- **Timestamp tracking**: Shows when each migration was executed +- **Idempotency**: Prevents duplicate migration runs +- **Version identification**: Easily identify the current database schema version -- Inputs - - Latest Devnet/Mainnet database - - Devnet/Mainnet genesis ledger - - Replayer checkpoint from last run - - Migrated berkeley database from initial migration +--- +url: /network-upgrades/mesa/preflight-network +--- -- Outputs - - Migrated Devnet/Mainnet database to the Berkeley format up to the last canonical block - - Replayer checkpoint which can be used for the next incremental migration +# Connect to Mesa Preflight Network -### Phase 1: Berkeley migration app run +The Mesa preflight network is a testing environment for validating the Mesa upgrade before deployment to devnet and mainnet. This network allows node operators and developers to test their infrastructure and applications with the new Mesa release. -``` -mina-berkeley-migration \ - --batch-size 1000 \ - --config-file ledger.json \ - --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ - --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ - --blocks-bucket mina_network_block_data \ - --precomputed-blocks-local-path . \ - --keep-precomputed-blocks \ - --network NETWORK -``` +:::tip Mesa preflight hard fork completed -where: +The Mesa preflight network hard-forked at **2026-04-27 13:00 UTC**. The post-fork chain is now running the Mesa release **`4.0.0-preflight-3f038cb`**, which bumps the **transaction protocol version to 5.0.0**. -`--batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. +The transaction protocol bump is a breaking wire-format change: any node still running a pre-fork build (including the stop-slot release `4.0.0-preflight-stop-2967b39` and the original preflight build `4.0.0-preflight1-b649c79`) will reject the new transaction format and is no longer compatible with the network. Upgrade to **`4.0.0-preflight-3f038cb`** to resume participation. -`--config-file`: path to the genesis ledger file +zkApp developers must rebuild and redeploy against **`o1js@3.0.0-mesa.698ca`** — the first o1js release that targets transaction protocol v5.0.0. Transactions produced by earlier o1js versions will be rejected by post-fork nodes. -`--mainnet-archive-uri`: connection string to the database to be migrated +A dedicated **Upgrade Steps** page covering the post-fork operator checklist will be linked from the Mesa Upgrade sidebar once it ships. -`--migrated-archive-uri`: connection string to the database that will hold the migrated data +::: -`--blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +:::caution Preflight Network Notice -`--precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location +The preflight network is intended for testing purposes only. This network may experience instability, breaking changes, and unexpected behavior. Data on this network should not be considered persistent or reliable. -`--keep-precomputed-blocks`: keep the precomputed blocks on-disk after the migration is complete +::: -`--network`: the network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +## Available Build Versions -#### Phase 2: Replayer in migration mode run +The Mesa preflight network uses the following build versions: -``` - mina-migration-replayer \ - --migration-mode \ - --archive-uri postgres://postgres:postgres@localhost:5432/migrated \ - --input-file replayer-checkpoint-XXX.json \ - --checkpoint-interval 10000 \ - --checkpoint-output-folder . -``` +**Current version (install this):** `4.0.0-preflight-3f038cb` — post-hard-fork release. Implements transaction protocol version **5.0.0** and is the only build compatible with the post-fork chain. -where: +**Compatible o1js release:** `o1js@3.0.0-mesa.698ca` — the matching SDK for transaction protocol v5.0.0. Install with `npm install o1js@3.0.0-mesa.698ca` (or the equivalent for your package manager) when developing or redeploying zkApps against the post-fork preflight network. -`--migration-mode`: flag for migration +**Previous versions (historical, do not install on new nodes):** -`--archive-uri`: connection string to the database that will hold the migrated data +- `4.0.0-preflight-stop-2967b39` — stop-slot release used to carry nodes up to the fork. Pre-fork transaction protocol; incompatible with the post-fork chain. +- `4.0.0-preflight1-b649c79` — original preflight build. Listed for reference only. -`--input-file`: path to the latest checkpoint file `replayer-checkpoint-XXX.json` +### Docker Images -`replayer-checkpoint-XXX.json`: the latest checkpoint generated from the previous migration +Docker images are available for both `amd64` and `arm64` architectures, on the `bookworm` and `noble` base distributions: -`--checkpoint-interval`: frequency of checkpoints file expressed in blocks count +- **Mina Daemon:** `gcr.io/o1labs-192920/mina-daemon:4.0.0-preflight-3f038cb-bookworm-mesa` +- **Archive Node:** `gcr.io/o1labs-192920/mina-archive:4.0.0-preflight-3f038cb-bookworm-mesa` +- **Rosetta:** `gcr.io/o1labs-192920/mina-rosetta:4.0.0-preflight-3f038cb-bookworm-mesa` -`--checkpoint-output-folder`: path to folder for replayer checkpoint files +For `noble`, swap `bookworm` for `noble` in the tag (e.g. `mina-daemon:4.0.0-preflight-3f038cb-noble-mesa`). -Incremental migration can be run continuously on top of the initial migration or last incremental until the fork block is announced. +### Debian Packages -#### Phase 3: Validations +Debian packages are available from the unstable repository: -Use the **berkeley_migration_verifier** app to perform checks for both the fully migrated and partially migrated database. +**Repository:** `unstable.apt.packages.minaprotocol.com` +**Channel:** `preflight` +**Codenames:** `bookworm`, `noble` -``` - mina-berkeley-migration-verifier \ - pre-fork \ - --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ - --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated -``` +Available packages (each at version `4.0.0-preflight-3f038cb`): -where: +- `mina-mesa` +- `mina-archive-mesa` +- `mina-rosetta-mesa` -`--mainnet-archive-uri`: connection string to the database to be migrated +For detailed information about the Debian repository structure and configuration, please refer to the [Debian Repository documentation](https://unstable.apt.packages.minaprotocol.com/). -`--migrated-archive-uri`: connection string to the database that will hold the migrated data +## Connecting to the Network -Note that: you can run incremental migration continuously on top of the initial migration or the last incremental until the fork block is announced. +To connect your node to the Mesa preflight network, you must use the preflight network seed peer list. -### Stage 3: Remainder migration +### Required Parameter -When the fork block is announced, you must tackle the remainder migration. This is the last migration run -you need to perform. In this stage, you close the migration cycle with the last migration of the remainder blocks between the current last canonical block and the fork block (which can be pending, so you don't need to wait 290 blocks until it would become canonical). -You must use `--fork-state-hash` as an additional parameter to the **berkeley-migration** app. - -- Inputs - - Latest Devnet/Mainnet database - - Devnet/Mainnet genesis ledger - - Replayer checkpoint from last run - - Migrated Berkeley database from last run - - Fork block state hash +Add the following parameter to your mina daemon command: -- Outputs - - Migrated devnet/mainnet database to berkeley up to fork point - - Replayer checkpoint which can be used for the next incremental migration +```bash +--peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt +``` -:::info -The migrated database output from this stage of the final migration is required to initialize your archive nodes on the upgraded network. -::: +### Example: Running with Docker -#### Phase 1: Berkeley migration app run +To start a Mina daemon node using Docker: -``` -mina-berkeley-migration \ - --batch-size 1000 \ - --config-file ledger.json \ - --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ - --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ - --blocks-bucket mina_network_block_data \ - --precomputed-blocks-local-path \ - --keep-precomputed-blocks \ - --network NETWORK \ - --fork-state-hash {fork-state-hash} +```bash +docker run --name mina-mesa-preflight -d \ + -p 8302:8302 \ + --restart=always \ + gcr.io/o1labs-192920/mina-daemon:4.0.0-preflight-3f038cb-bookworm-mesa \ + daemon \ + --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt ``` -where: +For production deployments, you'll want to mount configuration directories and add additional flags: -`--batch-size`: number of precomputed blocks to be fetched at one time from Google Cloud. A larger number, like 1000, can help speed up migration process. +```bash +docker run --name mina-mesa-preflight -d \ + -p 8302:8302 \ + --restart=always \ + -v $(pwd)/.mina-config:/root/.mina-config \ + gcr.io/o1labs-192920/mina-daemon:4.0.0-preflight-3f038cb-bookworm-mesa \ + daemon \ + --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt \ + --libp2p-keypair /root/.mina-config/keys/libp2p-key +``` -`--config-file`: path to the genesis ledger file +### Example: Running with Debian Package -`--mainnet-archive-uri`: connection string to the database to be migrated +First, install dependencies and configure the repository: -`--migrated-archive-uri`: connection string to the database that will hold the migrated data +```bash +# Step 1: Install dependencies +sudo apt-get install -y lsb-release ca-certificates wget gnupg -`--blocks-bucket`: name of the precomputed blocks bucket. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +# Step 2: Import the GPG key +wget -q https://unstable.apt.packages.minaprotocol.com/repo-signing-key.gpg \ + -O /etc/apt/trusted.gpg.d/minaprotocol.gpg -`--precomputed-blocks-local-path`: path to folder for on-disk precomputed blocks location +# Step 3: Add the unstable repository with preflight channel +echo "deb https://unstable.apt.packages.minaprotocol.com $(lsb_release -cs) preflight" | \ + sudo tee /etc/apt/sources.list.d/mina-preflight.list -`--keep-precomputed-blocks`: keep the precomputed blocks on-disk after the migration is complete +# Step 4: Update and install +sudo apt-get update +sudo apt-get install -y mina-mesa=4.0.0-preflight-3f038cb +``` -`--network`: the network name (`devnet` or `mainnet`) when determining precomputed blocks. Precomputed blocks are assumed to be named with format: `{network}-{height}-{state_hash}.json` +Then start the daemon: -`--fork-state-hash`: fork state hash +```bash +mina daemon \ + --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt \ + --libp2p-keypair ~/.mina-config/keys/libp2p-key +``` -:::info -When you run the **berkeley-migration** app with fork-state-hash, there is no requirement for the fork state block to be canonical. -The tool automatically converts all pending blocks in the subchain, including the fork block, to canonical blocks. -::: +### Running Archive Node -#### Phase 2: Replayer in migration mode run +:::info Upgrading from Berkeley to Mesa -``` -mina-migration-replayer \ - --migration-mode \ - --archive-uri postgres://postgres:postgres@localhost:5432/migrated \ - --input-file replayer-checkpoint-XXX.json \ - --checkpoint-interval 10000 \ - --checkpoint-output-folder . -``` +If you have an existing Berkeley archive database that you want to upgrade to Mesa, please refer to the comprehensive [Archive Upgrade](archive-upgrade) guide for detailed instructions on: +- Migration prerequisites and requirements +- Running the upgrade script +- Rollback procedures +- Database schema changes +- Verification steps -where: +::: -`--migration-mode`: flag for migration +#### Installing and Running Mesa Archive Node -`--archive-uri`: connection string to the database that will hold the migrated data +An archive node stores the full history of the blockchain and provides a GraphQL API for querying historical data. It requires a PostgreSQL database. -`--input-file`: path to the latest checkpoint file `replayer-checkpoint-XXX.json` from stage 1 +**Installation with Debian:** -`replayer-checkpoint-XXX.json`: the latest checkpoint generated from the previous migration +```bash +# Step 1: Install dependencies +sudo apt-get install -y lsb-release ca-certificates wget gnupg postgresql postgresql-contrib -`--checkpoint-interval`: frequency of checkpoints file expressed in blocks count +# Step 2: Import the GPG key +wget -q https://unstable.apt.packages.minaprotocol.com/repo-signing-key.gpg \ + -O /etc/apt/trusted.gpg.d/minaprotocol.gpg -`--checkpoint-output-folder`: path to folder for replayer checkpoint files +# Step 3: Add the repository +echo "deb https://unstable.apt.packages.minaprotocol.com $(lsb_release -cs) preflight" | \ + sudo tee /etc/apt/sources.list.d/mina-preflight.list -#### Phase 3: Validations +# Step 4: Install mina-archive-mesa +sudo apt-get update +sudo apt-get install -y mina-archive-mesa=4.0.0-preflight-3f038cb -Use the **berkeley_migration_verifier** app to perform checks for both the fully migrated and partially migrated databases. +# Step 5: Create PostgreSQL database +sudo -u postgres createdb archive +sudo -u postgres createuser archive_user +sudo -u postgres psql -c "ALTER USER archive_user WITH PASSWORD 'your-secure-password';" +sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE archive TO archive_user;" -``` - mina-berkeley-migration-verifier \ - post-fork \ - --mainnet-archive-uri postgres://postgres:postgres@localhost:5432/source \ - --migrated-archive-uri postgres://postgres:postgres@localhost:5432/migrated \ - --fork-config-file fork_genesis_config.json \ - --migrated-replayer-output replayer-checkpoint-XXXX.json +# Step 6: Start the archive node +mina-archive run \ + --postgres-uri postgresql://archive_user:your-secure-password@localhost:5432/archive \ + --server-port 3086 ``` -where: +**Using Docker:** -`--mainnet-archive-uri`: connection string to the database to be migrated +```bash +# Start the archive node (assumes you have a PostgreSQL server running) +docker run --name mina-archive-mesa -d \ + -p 3086:3086 \ + --restart=always \ + gcr.io/o1labs-192920/mina-archive:4.0.0-preflight-3f038cb-bookworm-mesa \ + mina-archive run \ + --postgres-uri postgresql://archive_user:your-secure-password@postgres-host:5432/archive \ + --server-port 3086 +``` -`--migrated-archive-uri`: connection string to the database that will hold the migrated data +**Connecting Mina Daemon to Archive:** -`--migrated-replayer-output`: path to the latest checkpoint file `replayer-checkpoint-XXX.json` +Configure your mina daemon to send blocks to the archive node: -`--fork-config`: fork genesis config file is the new genesis config that is distributed with the new daemon and is published after the fork block is announced +```bash +mina daemon \ + --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt \ + --archive-address localhost:3086 +``` -### Example migration steps using Mina Foundation data for Devnet using Debian +### Installing and Running Rosetta -See: [Worked example using March 22 data](/network-upgrades/berkeley/archive-migration/debian-example) +Rosetta is a standardized API for blockchain integration. The Rosetta API requires both a mina daemon and an archive node. -### Example migration steps using Mina Foundation data for Mainnet using Docker +**Installation with Debian:** -See: [Worked example using March 22 data](/network-upgrades/berkeley/archive-migration/docker-example) +```bash +# Step 1: Install dependencies (if not already done) +sudo apt-get install -y lsb-release ca-certificates wget gnupg -## How to verify a successful migration +# Step 2: Import the GPG key (if not already done) +wget -q https://unstable.apt.packages.minaprotocol.com/repo-signing-key.gpg \ + -O /etc/apt/trusted.gpg.d/minaprotocol.gpg -o1Labs and Mina Foundation make every effort to provide reliable tools of high quality. However, it is not possible to eliminate all errors and test all possible Mainnet archive variations. -All important checks are implemented in the `mina-berkeley-migration-verifier` application. -However, you can use the following checklist if you want to perform the checks manually: +# Step 3: Add the repository (if not already done) +echo "deb https://unstable.apt.packages.minaprotocol.com $(lsb_release -cs) preflight" | \ + sudo tee /etc/apt/sources.list.d/mina-preflight.list -1. All transaction (user command and internal command) hashes are left intact. +# Step 4: Install mina-rosetta-mesa +sudo apt-get update +sudo apt-get install -y mina-rosetta-mesa=4.0.0-preflight-3f038cb - Verify that the `user_command` and `internal_command` tables have the Devnet/Mainnet format of hashes. For example, `CkpZirFuoLVV...`. +# Step 5: Start Rosetta +# Rosetta connects to both the mina daemon and archive node +mina-rosetta \ + --port 3087 \ + --archive-uri http://localhost:3086/graphql \ + --graphql-uri http://localhost:3085/graphql +``` -2. Parent-child block relationship is preserved +**Using Docker:** - Verify that a given block in the migrated archive has the same parent in the Devnet/Mainnet archive (`state_hash` and `parent_hash` columns) that was used as input. +```bash +docker run --name mina-rosetta-mesa -d \ + -p 3087:3087 \ + --restart=always \ + gcr.io/o1labs-192920/mina-rosetta:4.0.0-preflight-3f038cb-bookworm-mesa \ + --port 3087 \ + --archive-uri http://archive-host:3086/graphql \ + --graphql-uri http://mina-daemon-host:3085/graphql +``` -3. Account balances remain the same +**Note:** Rosetta requires: +- A running mina daemon (GraphQL endpoint, typically port 3085) +- A running archive node (GraphQL endpoint, typically port 3086) +- Both must be fully synced for Rosetta to function properly - Verify the same balance exists for a given block in Mainnet and the migrated databases. +## Nightly Builds -## Tips and tricks +In addition to the preflight release builds, nightly builds are also available for testing the latest changes. -We are aware that the migration process can be very long (a couple of days). Therefore, we encourage you to use cron jobs that migrate data incrementally. -The cron job requires access to Google Cloud buckets (or other storage): +**Repository:** `nightly.apt.packages.minaprotocol.com` -- A bucket to store migrated-so-far database dumps -- A bucket to store checkpoint files +Nightly builds are available for both Debian packages and Docker images. These builds represent the latest development state and may be even more unstable than preflight releases. -We are tightly coupled with Google Cloud infrastructure due to the precomputed block upload mechanism. -This is why we are using also buckets for storing dumps and checkpoint. However, you do not have to use Google Cloud for other things than -precomputed blocks. With configuration, you can use any gsutil-compatible storage backend (for example, S3). +For more information about nightly builds and repository configuration, see the [Debian Repository documentation](https://nightly.apt.packages.minaprotocol.com/). -Before running the cron job, upload an initial database dump and an initial checkpoint file. +## Verification -To create the files, run these steps locally: +After starting your node, verify connectivity to the preflight network: -1. Download a Devnet/Mainnet archive dump and load it into PostgreSQL. -2. Create an empty database using the new archive schema. -3. Run the **berkeley-migration** app against the Devnet/Mainnet and new databases. -4. Run the **replayer app in migration mode** with the `--checkpoint-interval` set to a suitable value (perhaps 100) and start with the original Devnet/Mainnet ledger in the input file. -5. Use pg_dump to dump the migrated database and upload it. -6. Upload the most recent checkpoint file. +### Check Node Status -The cron job performs the same steps in an automated fashion: +```bash +# For Docker +docker exec -it mina-mesa-preflight mina client status -1. Pulls the latest Devnet/Mainnet archive dump and loads it into PostgresQL. -2. Pulls the latest migrated database and loads it into PostgreSQL. -3. Pulls the latest checkpoint file. -4. Runs the **berkeley-migration** app against the two databases. -5. Runs the **replayer app in migration mode** using the downloaded checkpoint file; set the checkpoint interval to be smaller (perhaps 50) because there are typically only 200 or so blocks in a day. -7. Uploads the migrated database. -8. Uploads the most recent checkpoint file. +# For Debian installation +mina client status +``` -Be sure to monitor the cron job for errors. +### Monitor Logs -Just before the Berkeley upgrade, migrate the last few blocks by running locally: +```bash +# For Docker +docker logs -f mina-mesa-preflight -1. Download the Devnet/Mainnet archive data directly from the k8s PostgreSQL node (not from the archive dump), and load it into PostgreSQL. -2. Download the most recent migrated database and load it into PostgresQL. -3. Download the most recent checkpoint file. -4. Run the **berkeley-migration** app against the two databases. -5. Run the **replayer app in migration mode** using the most recent checkpoint file. +# For systemd service (Debian) +journalctl -u mina -f +``` -It is worthwhile to perform these last steps as a dry run to make sure all goes well. You can run these steps as many times as needed. +## Support and Feedback -## Known migration problems +If you encounter issues or have feedback about the Mesa preflight network: -Please remember that rerunning after crash is always possible. -After solving any of below issues you can rerun process and migration -will continue form last position +1. Check the [Mina Protocol Discord](https://discord.gg/minaprotocol) for community support +2. Report issues on the [Mina GitHub repository](https://github.com/MinaProtocol/mina/issues) +3. Join the Mesa upgrade discussions in the community channels -#### Async was unable to add a file descriptor to its table of open file descriptors -For example: +## Next Steps -``` - ("Async was unable to add a file descriptor to its table of open file descriptors" - (file_descr 18) - (error - "Attempt to register a file descriptor with Async that Async believes it is already managing.") - (backtrace - ...... -``` -A remedy is to lower `--block-batch-size` parameter to values up to 500. +- Explore additional Mesa upgrade documentation +- Test your applications and infrastructure against the preflight network +- Provide feedback to help improve the Mesa upgrade process -#### Map.find_exn: not found -For example: -``` -(monitor.ml.Error - (Not_found_s - ("Map.find_exn: not found" -.... -``` +--- +url: /node-developers/bip44 +--- -Usually this error means that there is a gap in canonical chain. In order to fix it please -ensure that missing-block-auditor run is successful +# BIP44 Information -#### Yojson.Json_error .. Unexpected end of input +| index | hexa | symbol | coin | +|:------|:-----------|:-------|:----------------------------------| +| 12586 | 0x8000312a | MINA | [Mina](https://minaprotocol.com/) | -For example: +Mina uses the 5 level [BIP32](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) path format specified in [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) ``` -(monitor.ml.Error - ("Yojson.Json_error(\"Line 1, bytes 1003519-1003520:\\nUnexpected end of input\")") - ("Raised at Yojson.json_error in file \"common.ml\", line 5, characters 19-39" - "Called from Yojson.Safe.__ocaml_lex_read_json_rec in file \"lib/read.mll\", line 215, characters 28-52" -... +m / purpose' / coin_type' / account' / change / address_index ``` -This issue is caused by invalid precomputed block. Deleting the downloaded precomputed blocks should resolve this issue. - -#### Error querying db, error: Request to ... failed: ERROR: column \"type\" does not exist - -You provided the migrated schema as source one when invoking script or berkeley-migration app - -#### Poor performance of migration when accessing remote database - -We conducted migration tests with both a local database and a distant database (RDS). -The migration using the local database appears to process significantly faster. We strongly suggest to use offline database installed locally +Keypairs are derived by varying only the ``account`` while keeping the ``change`` and ``address_index`` zero. -#### ERROR: out of shared memory +# BIP32 path format ``` -(monitor.ml.Error (Failure "Error querying for user commands with id 1686617, error Request to postgresql://user:pwd@host:port/db failed: ERROR: out of shared memory -\nHINT: You might need to increase max_pred_locks_per_transaction +m / 44' / 12586' / account' / 0 / 0 ``` -Solution is either to increase `max_pred_locks_per_transaction` setting in postgres database. -Alternative is to isolate database from mainnet traffic (for example by exporting dump from live database and import it on isolated environment) - -#### Berkeley migration app is consuming all of my resources - -When running a full migration, you can stumble on memory leaks that prevent you from cleanly performing the migration in one pass. A machine with 64 GB of RAM can be frozen after ~40k migrated blocks. Each 200 blocks inserted into the database increases the memory leak by 4-10 MB. - -A potential workaround is to split the migration into smaller parts using cron jobs or automation scripts. +# Examples -## FAQ +| account | path | +|:--------|:------------------------| +| 0 | m/44'/12586'/0'/0/0 | +| 271 | m/44'/12586'/271'/0/0 | -### Migrated database is missing orphaned blocks +--- +url: /node-developers/code-review-guidelines +--- -By design, Berkeley migration omits orphaned blocks and, by default, migrates only canonical (and pending, if setup correctly) blocks. +# Code Review Guidelines -### Replayer in migration mode overrides my old checkpoints +A good pull request: -By default, the replayer dumps the checkpoint to the current folder. All checkpoint files have a similar format: +- Does about one thing (new feature, bug fix, etc) +- Adds tests and documentation for any new functionality +- When fixing a bug, adds or fixes test that would have caught said bug -`replayer-checkpoint-{number}.json.` +## OCaml things -To prevent override of old checkpoints, use the `--checkpoint-output-folder` and `--checkpoint-file-prefix` parameters to modify the output folder and prefix. +- Are the [style guidelines](./style-guide) being followed? +- Do the signatures make sense? Are they minimal and reusable? +- Does anything need to be functored over? +- Are there any error cases that aren't handled correctly? +- Are calls to `_exn` functions justified? Are their preconditions for not + throwing an exception met? Is the exception it throws useful? +- There shouldn't be commented out code. +- No stray debug code lying around. +- Any logging is appropriate. All `Logger.trace` logs should be inessential, + because they won't be shown to anyone by default. +- Should this code live in its library? Should it live in a different library? +- Does the code confuse you? Maybe there should be a comment, or it should be + structured differently. +- Does a behavior change break assumptions other code makes? --- -url: /network-upgrades/berkeley/archive-migration/understanding-archive-migration +url: /node-developers/codebase-overview --- -# Understanding the Archive Migration +# Codebase Overview -You can reduce risks and effort by reading all of the archive documentation in entirety. +The Mina Protocol is written in OCaml, a statically typed, functional programming language. -## Archive node migration overview +For OCaml beginners, it may help to skim through [Real World OCaml](https://realworldocaml.org/) for a good introduction to the language, and some deep dives into specific topics if you're interested. +Assuming basic familiarity with OCaml, here's some more info on how it is used in the Mina Protocol. -Archive node migration is a crucial part of the Berkeley upgrade. The current Devnet and Mainnet database format must be converted to the Berkeley format to preserve historical data and assure archive node chain continuity. +## Code Structure -For this purpose, the o1Labs and Mina Foundation teams prepared a migration package. +See the [Repository Structure page](/node-developers/repository-structure). -### Archive node Berkeley migration package +## Compilation -This package contains the required applications to migrate existing Devnet and Mainnet databases into the new Berkeley schema and a usability script: +The OCaml compiler can target bytecode and native compilation. The code statically links with some libraries so it can't compile to bytecode. The code doesn't play well with the REPL. Dune, the build system, has a concept of folders that represent modules and files that a module. If the folder has a file with the same name, it's essentially equivalent to `index.js` in Node. -1. **berkeley-migration** +Interface files in OCaml with the `.mli` extension contain type signatures and structures for a module. The corresponding implementation must have the same file name with the `.ml` extension. Only the things defined in the interface are available from other modules. If an interface file does not exist for a module, everything is exposed by default. The same convention and rules apply to files with the `.rei` and `.re` extensions. - Use the **berkeley-migration** app to migrate as much data as possible from the Devnet/Mainnet database and download precomputed blocks to get the window density data. - - This app runs against the Devnet/Mainnet database and the new Berkeley database. +For the linking step, `dune` uses `ldd` under the hood. You can also use things like `-O3` for optimization. For debugging, you can use `gdb`. -2. **replayer app in migration mode** +## Open-source Library Documentation - The existing replayer application is enhanced with a new migration mode. Use the **replayer app in migration mode** to analyze the transactions in the partially migrated database (resulting from running berkeley-migration app) and populate the `accounts_accessed` and `accounts_created` tables. This app also does the checks performed by the standard replayer, but does not check ledger hashes because the Berkeley ledger has greater depth that results in different hashes. - - This app runs only against the new archive database. +There are multiple libraries for OCaml. One challenge with learning OCaml is locating and reading documentation for the various libraries. For example, the Jane Street `Core` library has the following structure: -3. **berkeley-migration-verifier** +``` + Base + | + Core_kernel -> Async_kernel + | | + Unix <- Core -> Async +``` - Use the **berkeley-migration-verifier** verification software to determine if the migration (even incomplete) was successful. The app uses SQL validations on the migrated database. +In general, the source code of an installed library is not available, so follow these tips to find the documentation. -4. **end-to-end migration script** +To review the docs for Core, a standard library overlay. Core is a popular alternative to the OCaml standard library. First, go to the [Core](https://github.com/janestreet/core) codebase on GitHub and then locate the correct documentation. If you don't see the module you're looking for, go to next to `Core_kernel`. If that fails, then look for `Base`. To use the find capability in GitHub, expand the sections. - This shell script wraps all phases and stages of migration into a single script. It is provided purely for node operators usability and is equivalent of running the **berkely-migration** app, the **replayer app in migration mode**, and the **berkeley-migration-verifier** apps in the correct order. +Most documentation is published in HTML. However, you can use the an IDE to find and review code and documentation, like the Merlin editor service that provides modern IDE features for OCaml. Merlin provides type hints and code navigation, like Go To Definition. Note that Merlin works only if your code compiles. -### Incrementality +OPAM, the source-based package manager for OCaml, usually ships documentation with libraries that you can access using Merlin. -Use the **berkeley-migration** and **replayer** apps incrementally so that you can migrate part of the Devnet/Mainnet database, and, as new blocks are added to the Devnet/Mainnet databases, the new data can be migrated. +## Extensions -To obtain that incrementality, the **berkeley-migration** app looks at the migrated database and determines the most recent migrated block. It continues migration starting at the next block in the Devnet/Mainnet data. The **replayer app in migration mode** uses the checkpoint mechanism already in place for the replayer. A checkpoint file indicates the global slot since genesis for starting the replay and the ledger to use for that replay. New checkpoint files are written as it proceeds. +OCaml uses the ppx meta-programming system that generates code at compile time. -To take advantage of the incrementality, run a cron job that migrates a day's worth of data at a time (or some other interval). With the cron job in place, at the time of the actual Berkeley upgrade, you will need to migrate only a small amount of data. +For example, consider this ppx extension on a type signature: ---- -url: /network-upgrades/berkeley/flags-configs ---- +``` + type t = + | A + | B [@ to_yojson f] -# Post-Upgrade Flags and Configurations for Mainnet + [@@ deriving yojson] +``` -Please refer to the Berkeley node release notes [here](https://github.com/MinaProtocol/mina/releases/tag/3.0.3). +The single `@` scopes an extension to a single expression. The `@@` denotes the extension is expanded in the scope of the calling context. -### Network details +For an extension on a structure or a value, use the following syntax. -``` -Chain ID -a7351abc7ddf2ea92d1b38cc8e636c271c1dfd2c081c637f62ebc2af34eb7cc1 +`%` returns a value/expression. -Git SHA-1 -ae112d3a96fe71b4ccccf3c54e7b7494db4898a4 +`%%` injects a statement -Seed List -https://bootnodes.minaprotocol.com/networks/mainnet.txt +``` + let x = [% ...] + [%% ...] + let y = + let%... z = ... in + match%... ... with + | ... + | ... in -Node build -https://github.com/MinaProtocol/mina/releases/tag/3.0.3 + [%% if x] + let x = y + [%% else] + let x = z + [%% endif] ``` -### Block Producer's +**TL;DR** Anytime you see `[@ ...]` `[@@ ...]` `[% ...]` `[%% ...]` it's an OCaml language extension. -Start your node post-upgrade in Mainnet with the flags and environment variables listed below. +## Monads -``` -mina daemon ---block-producer-key ---config-directory ---file-log-rotations 500 ---generate-genesis-proof true ---libp2p-keypair ---log-json ---peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt +Functional design patterns that allows you to write computation in a very generic way, that removes the need for boilerplate code. Monads allow us to abstract up to a higher level of computation, while taking care of all the glue for us. In other words, monads are programmable semicolons. + +For example consider the following imperative example: -ENVIRONMENT VARIABLES -RAYON_NUM_THREADS=6 -MINA_LIBP2P_PASS -MINA_PRIVKEY_PASS +``` + function example(x) { + if ( x == null ) return null; + x = x + 1; + if ( !isEven(x) ) return null; + return x; + } ``` -### SNARK Coordinator -Configure your node post-upgrade in Mainnet with specific flags and environment variables as listed. +This can expressed similarly in functional programming using a monad, using option: ``` -mina daemon ---config-directory ---enable-peer-exchange true ---file-log-rotations 500 ---libp2p-keypair ---log-json ---peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt ---run-snark-coordinator ---snark-worker-fee 0.001 ---work-selection [seq|rand|roffset] + type a' option = + | None + | Some of 'a -ENVIRONMENT VARIABLES -MINA_LIBP2P_PASS -``` + let return x = Some x -### SNARK Workers -Connect to a SNARK Coordinator node if required and run the following flags. -``` -mina internal snark-worker ---proof-level full ---shutdown-on-disconnect false ---daemon-address + (* Bind infix operation, applies f to m *) + let (>>=) m f = + match m with + | Some x -> f x + | None -> None -ENVIRONMENT VARIABLES -RAYON_NUM_THREADS:8 + (** + Map infix operation + Essentially the same as bind, but the inner function unwraps the value. + **) + let (>>|) m f = m >>= (fun x -> return (f x)) ``` -### Archive Node -Running an Archive Node involves setting up a non-block-producing node and a PostgreSQL database configured with specific flags and environment variables. - -For more information about running archive nodes, see [Archive Node](/node-operators/archive-node). - -The PostgreSQL database requires two schemas: -1. The PostgreSQL schema used by the Mina archive database: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) -2. The PostgreSQL schema extensions to support zkApp commands: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) +Now we an use these primitives to reimplement the imperative example above as follows. -The non-block-producing node must be configured with the following flags: ``` -mina daemon ---archive-address : ---config-directory ---enable-peer-exchange true ---file-log-rotations 500 ---generate-genesis-proof true ---libp2p-keypair ---log-json ---peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt + let add_one = ((+) 1) -ENVIRONMENT VARIABLES -MINA_LIBP2P_PASS -``` + let bind_even : int -> int option = + fun x -> if x mod 2 = 0 then Some x else None -This non-block-producing node connects to the archive node with the addresses and port specified in the `--archive-address` flag. + let example x = x >>| add_one >>= bind_even; +``` -The **archive node** command looks like this: +OCaml has a `ppx` that makes writing monads much easier to follow, using the let syntax. ``` -mina-archive run ---metrics-port ---postgres-uri postgres://:@
:/ ---server-port 3086 ---log-json ---log-level DEBUG + let%bind x = y in + f x + + (* This compiles down to the following *) + y >>= (fun x -> f x) ``` -### Rosetta API -Once you have the Archive Node stack up and running, start the Rosetta API Docker image with the following command: +Essentially, this syntax takes the value from the let statement and places it on the left of the bind infix call, and puts the assignment into a lambda. -``` -docker run ---name rosetta --rm \ --p 3088:3088 \ ---entrypoint '' \ -minaprotocol/mina-rosetta:3.1.0-ae112d3-bullseye-mainnet \ -/usr/local/bin/mina-rosetta \ ---archive-uri "${PG_CONNECTION_STRING}" \ ---graphql-uri "${GRAPHQL_URL}" \ ---log-json \ ---log-level ${LOG_LEVEL} \ ---port 3088 -``` +## Async ---- -url: /network-upgrades/berkeley/requirements ---- +Under the hood, async uses Monads. However, ivars are the low-level async primitive. `'a Ivar.t` is essentially a mutex that can only be filled once. After the value from a computation returns, it then fills the ivar. The rest of the syntactic sugar takes the ivar and passes them through `Deferred` monads. -# Requirements +A `yield` function exists, but avoid using it since it has some weird behavior. Instead, operate on the wrapped values that happen in between `Deferred` bindings. -## Hardware Requirements +## Custom Helpers -Please note the following are the hardware requirements for each node type after the upgrade: +Use these custom helpers: -| Node Type | Memory | CPU | Storage | Network | -|--|--|--|--|--| -| Mina Daemon Node | 32 GB RAM | 8 core processor with BMI2 and AVX CPU instruction set are required | 64 GB | 1 Mbps Internet Connection | -| SNARK Coordinator | 32 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | -| SNARK Worker | 32 GB RAM | 4 core/8 threads per worker with BMI2 and AVX CPU instruction set are required | 64 GB | 1 Mbps Internet Connection | -| Archive Node | 32 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | -| Rosetta API standalone Docker image | 32 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | -| Mina Seed Node | 64 GB RAM | 8 core processor | 64 GB | 1 Mbps Internet Connection | +- `Strict_pipe` - wraps pipe and gives certain guarantees around how it can be used. +- `Broadcast_pipe` - allows a single pipe to be hooked up to multiple downstream pipes. -## Mina Daemon Requirements +Do not use: -### Installation +- `Async.Pipe` operates essentially like a buffer and is unsafe since it has unlimited buffering by default (memory overflow) and some funky behavior around which end of the pipe should do what. +- `Linear_pipe` - deprecated in favor of `Strict_pipe` and `Broadcast_pipe`. -:::caution +--- +url: /node-developers/contributing +--- -If you have `mina-generate-keypair` installed, you will need to first `sudo apt remove mina-generate-keypair` before installing `mina-mainnet=3.1.0-ae112d3`. -The `mina-generate-keypair` binary is now installed as part of the mina-mainnet package. +# Contributing to Mina -::: +Mina is an open-source project with a mission to build an inclusive and sustainable community-driven protocol. As such, Mina welcomes contributions. -### IP and Port configuration +The protocol is in development and is always improving. You can make contributions in many ways, including writing code, user testing, documentation, and community support. For specific instructions for contributing in each of these domains, see the codebase repositories or ask in [Mina Protocol Discord](https://discord.gg/minaprotocol). -**IP:** +For general questions on getting involved, reach out to the Mina community on the [Mina Protocol Discord](https://discord.gg/minaprotocol) server. -By default, the Mina Daemon will attempt to retrieve its public IP address from the system. If you are running the node behind a NAT or firewall, you can set the `--external-ip` flag to specify the public IP address. +## Developers -**Port:** +Mina is entirely open source, and the code is distributed under the terms of the [Apache License 2.0](https://github.com/MinaProtocol/mina/blob/master/LICENSE). -Nodes must expose a port publicly to communicate with other peers. -Mina uses by default the port `8302` which is the default libp2p port. +## Docs -You can use a different port by setting the `--external-port` flag. +Mina Docs are also open source. We love our community. Help us make the docs better, contributions are welcome and appreciated. -### Node Auto-restart +- For a quick fix, click **EDIT THIS PAGE**. +- For a more substantial contribution, see the [Docs Contributing Guidelines](https://github.com/o1-labs/docs2/blob/main/CONTRIBUTING.md). -Ensure your nodes are set to restart automatically after a crash. For guidance, refer to the [auto-restart instructions](/node-operators/validator-node/connecting-to-the-network#running-mina-node-as-a-service) +## Mina Grants -## Seed Peer Requirements +Grants are rewarded for certain projects related to the development of Mina. See [Mina Grants](https://minaprotocol.com/grants) for details. -### Generation of libp2p keypair +The projects are mostly programming focused, but the areas of design and community development are included. Reach out in the [#grants](https://discord.com/channels/484437221055922177/727960609832042607) channel on Mina Protocol Discord for questions about the grant program. -To ensure connectivity across the network, it is essential that all seed nodes start with the **same** `libp2p` keypair. -This consistency allows other nodes in the network to reliably connect. -Although the same libp2p keys can be reused from before the upgrade, if you need to manually generate new libp2p keys, use the following command: +## Reporting Issues -``` -mina libp2p generate-keypair --privkey-path -``` +If you notice Code of Conduct violations, please follow the Reporting Guidelines in [Code of Conduct](https://github.com/MinaProtocol/mina/blob/develop/CODE_OF_CONDUCT.md) to file a report and alert the community to ensure a safe space for everyone. -Further information on [generating key pairs](/node-operators/validator-node/generating-a-keypair) on Mina Protocol. +If you encounter critical bugs or vulnerabilities in the protocol, report them to security@minaprotocol.com. For minor bugs and issues, create an issue on GitHub. --- -url: /network-upgrades/berkeley/upgrade-steps +url: /node-developers/graphql-api --- -# Upgrade Steps +# GraphQL API - Mainnet Upgrade steps +:::caution -Below it's the description in detail of all the upgrade steps and what which node operator type should do to in each step. +- Mina APIs are still under construction, so these endpoints may change. +- By default, the GraphQL port is bound to localhost. Exposing the GraphQL API to the internet allows anyone to send Mina from the accounts known to the daemon. -## Pre-Upgrade +::: -- During the Pre-Upgrade phase, node operators should prepare for the upcoming upgrade. The most important steps are: - - Review the [upgrade readiness checklist](https://docs.google.com/document/d/1rTmJvyaK33dWjJXMOSiUIGgf8z7turxolGHUpVHNxEU/edit#heading=h.2hqz0ixwjk3f) to confirm they have covered the required steps. - - Upgrade their nodes to the 1.4.1 stable version - - Ensure servers are provisioned to run Berkeley nodes, meeting the new hardware requirements - - Upgrade their nodes to the node version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3), with stop-slots, when this version becomes available - - Start the archive node initial migration if they run archive nodes and wish to perform the migration in a decentralized manner +The Mina daemon exposes a [GraphQL API](https://graphql.org/) used to request information from and submit commands to a running node. -**Please note:** a simplified Node Status service will be part of the upgrade tooling and enabled by default in Pre-Upgrade release with the stop-slots ([3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3)). This feature will allow for a safe upgrade by monitoring the amount of upgraded active stake. Only non-sensitive data will be reported. If operators are not comfortable sharing their node version, they will have the option to disable the node version reports by using the appropriate node flag `--node-stats-type none` +To use the GraphQL API, connect your GraphQL client to `http://localhost:3085/graphql` or open in your browser to use the [GraphiQL IDE](https://github.com/graphql/graphiql). -### Block Producers and SNARK Workers -1. Review the [upgrade readiness checklist](https://docs.google.com/document/d/1rTmJvyaK33dWjJXMOSiUIGgf8z7turxolGHUpVHNxEU). -1. Provision servers that meet the minimum hardware requirements, including the new 32GB RAM requirement and support for _AVX_ and _BMI2_ CPU instructions. -1. Upgrade nodes to node version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) ([3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3) has built-in stop slots). +- By default, an HTTP server runs on port `3085`. You can configure a different port, use the `-rest-port` flag with the daemon startup command. -### Archive Node Operators and Rosetta Operators -- Two migration processes will be available to archive node operators: _trustless_ and _trustful_. If the archive node operator wants to perform the _trustless_ migration, they should follow these steps; otherwise, proceed to the Upgrade phase. The _trustful_ migration will rely on o1Labs database exports and Docker images to migrate the archive node database and doesn’t require any actions at this stage. +- The default security permits only connections from `localhost`. To listen on all interfaces, add the `-insecure-rest-server` flag to the daemon startup command. -1. Trustless migration: - - Perform the initial archive node migration. Since Mainnet is a long-lived network, the initial migration process can take up to 48 hours, depending on your server specification and infrastructure. - - If your Mina Daemon, archive node, or PostgreSQL database runs on different machines, the migration performance will be greatly impacted. - - For more information on the archive node migration process, please refer to the [Archive Migration](/network-upgrades/berkeley/archive-migration) section. -2. Upgrade all nodes to the latest stable version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3). -3. Provision servers that meet the minimum hardware requirements, primarily the new 32GB RAM requirement. -4. Upgrade their nodes to the version that includes built-in stop slots before the pre-defined _stop-transaction-slot_. +In addition to information about the running node, the GraphQL API can return data about the network's latest blocks. However, as the blockchain's historical state is not persisted in Mina, only blocks in the node's transition frontier are returned, i.e., the last `k` blocks. For other historical data, use the [Archive Node](/node-operators/archive-node/getting-started) that is designed to retain and retrieve historical data. -### Exchanges -1. Make sure to test your system integration with Berkeley's new features. Pay special attention to: - - If you use the **o1labs/client-sdk** library to sign transactions, you should switch to **[mina-signer](https://www.npmjs.com/package/mina-signer)**. o1labs/client-sdk was **deprecated** some time ago and will be **unusable** once the network has been upgraded. Please review the migration instructions in [Appendix](/network-upgrades/berkeley/appendix). - - If you rely on the archive node SQL database tables, please review the schema changes in Appendix 1 of this document. -2. Upgrade all nodes to the latest stable version [3.0.3](https://github.com/MinaProtocol/mina/releases/tag/3.0.3). -3. Provision servers that meet the minimum hardware requirements, particularly the new 32GB RAM requirement. -4. Upgrade your nodes to the version that includes built-in stop slots before the pre-defined _stop-transaction-slot_. +The full Mina GraphQL schema is available [https://github.com/MinaProtocol/mina/blob/develop/graphql_schema.json](https://github.com/MinaProtocol/mina/blob/develop/graphql_schema.json). -*** +### Queries -## State Finalization -- Between the predefined _stop-transaction-slot_ and _stop-network-slot_, a stabilization period of 100 slots will occur. During this phase, the network consensus will not accept new blocks with transactions on them, including coinbase transactions. The state finalization period ensures all nodes reach a consensus on the latest network state before the upgrade. -- During the state finalization slots, it is crucial to maintain a high block density. Therefore, block producers and SNARK workers shall continue running their nodes to support the network's stability and security. -- Archive nodes should also continue to execute to ensure finalized blocks are in the database and can be migrated, preserving the integrity and accessibility of the network's history. +The Mina GraphQL API has a number of [queries](/node-operators/validator-node/querying-data) to extract data from a running node. [GraphQL queries](https://graphql.org/learn/queries/) allow specifying the data to be returned in the response. For example, to get the latest block and creator information known to the daemon: -### Block Producers and SNARK Workers -1. It is crucial for the network's successful upgrade that all block producers and SNARK workers maintain their block-producing nodes up and running throughout the state finalization phase. -2. If you are running multiple daemons like is common with many operators, you can run one single node at this stage. -3. If you are a Delegation Program operator, remember that your uptime data will continue to be tracked during the state finalization phase and will be considered for the delegation grant in the following epoch. +``` +query { + bestChain(maxLength: 1) { + creator + stateHash + protocolState { + consensusState { + blockHeight + } + previousStateHash + } + transactions { + coinbase + } + } +} +``` -### Archive Node Operators and Rosetta Operators -**If you plan to do the _trustful_ migration, you can skip this step.** -If you are doing the trustless migration, then: -1. Continue to execute the archive node to ensure finalized blocks are in the database and can be migrated. -2. Continue to run incremental archive node migrations until after the network stops at the stop-network slot. -3. For more information on the archive node migration process, please refer to the [Archive Migration](/network-upgrades/berkeley/archive-migration) section +The following query requests all pending transactions in the transaction pool together with their fees. This query can be used to generate an estimate of a suggested fee for a transaction: -### Exchanges +``` +query { + pooledUserCommands { + id, + fee + } +} +``` -Exchanges shall disable MINA deposits and withdrawals during the state finalization period (the period between _stop-transaction-slot_ and _stop-network-slot_) since any transactions after the _stop-transaction-slot_ will not be part of the upgraded chain. +:::tip -Remember that although you might be able to submit transactions, the majority of the block producers will be running a node that discards any blocks with transactions. +The memo field returned for a transaction is [Base58Check encoded](https://en.bitcoin.it/wiki/Base58Check_encoding). -*** +::: -## Upgrade +``` +query { + account(publicKey: "") { + balance { + total + } + delegate + nonce + } +} +``` -- Starting at the _stop-network-slot_ the network will not produce nor accept new blocks, resulting in halting the network. During the upgrade period, o1Labs will use automated tooling to export the network state based on the block at the slot just before the _stop-transaction-slot_. The exported state will then be baked into the new Berkeley build, which will be used to initiate the upgraded network. It is during the upgrade windows that the Berkeley network infrastructure will be bootstrapped, and seed nodes will become available. o1Labs will also finalize the archive node migration and publish the PostgreSQL database dumps for import by the archive node operators who wish to bootstrap their archives in a trustful manner. -- There is a tool available to validate that the Berkeley node was built from the pre-upgrade network state. To validate, follow the instructions provided in this [location](https://github.com/MinaProtocol/mina/blob/berkeley/docs/upgrading-to-berkeley.md) - -### Block Producers and SNARK Workers -1. During the upgrade phase (between _stop-network-slot_ and the publishing of the Berkeley release), block producers can shut down their nodes. -2. After the publication of the Berkeley node release, block producers and SNARK workers should upgrade their nodes and be prepared for block production at the genesis timestamp, which is the slot when the first Berkeley block will be produced. -3. It is possible to continue using the same libp2p key after the upgrade. Remember to adjust the new flag to pass the libp2p key to the node. +You can submit GraphQL requests from the command line of a node. For example, to use cURL to get the last ten block creators known to the node: -### Archive Node Operators and Rosetta Operators -1. Upon publishing the archive node Berkeley release, archive node operators and Rosetta operators should upgrade their systems. -There will be both Docker images and archive node releases available to choose from. -2. Depending on the chosen migration method: - - _Trustless_ - - Operators should direct their Berkeley archive process to the previously migrated database. - - _Trustful_ - - Operators shall import the SQL dump file provided by o1Labs to a freshly created database. - - Operators should direct their Berkeley archive process to the newly created database. +``` +curl -d '{"query": "{ + bestChain(maxLength: 10) { + creator + } +}"}' -H 'Content-Type: application/json' http://localhost:3085/graphql +``` -**Please note:** both the _trustless_ and _trustful_ migration processes will discard all Mainnet blocks that are not canonical. If you wish to preserve the entire block history, i.e. including non-canonical blocks, you should maintain the Mainnet archive node database for posterior querying needs. +### Mutations -### Exchanges -1. Exchanges shall disable MINA deposits and withdrawals during the entirety of the upgrade downtime, since the _stop-transaction-slot_ until the Mainnet Berkeley network is operational. -2. After the Berkeley releases are published, exchanges should upgrade their nodes and prepare for the new network to start block production. +GraphQL mutations modify the running node in some way. For example, mutations may be used to send a payment, create a new account, or to add additional peers. -*** +Consult the GraphQL schema for all available mutations. -## Post-Upgrade -- At approximately 1 hour after the publishing of the Berkeley node release, at a predefined slot (Berkeley genesis timestamp), block production will start, and the network is successfully upgraded. -- Node operators can monitor their nodes and provide feedback to the technical team in case of any issues. Builders can start deploying zkApps. -- **Please note:** The Node Status service will not be enabled by default in the Berkeley release. If you wish to provide Node Status and Error metrics and reports to Mina Foundation, helping monitor the network in the initial phase, please use the following flags when running your nodes: - - `--node-stats-type [full|simple]` - - `--node-status-url https://nodestats.minaprotocol.com/submit/stats` - - `--node-error-url https://nodestats.minaprotocol.com/submit/stats` - - The error collection service tries to report any node crashes before the node process is terminated +Adding a new peer: -### Block Producers and SNARK Workers -1. Ensure that all systems have been upgraded and prepared for the start of block production. -2. Monitor nodes and network health, and provide feedback to the engineering team in case of any issues. +``` +mutation { + addPeers(peers:{ + libp2p_port:10511, + host:"34.73.68.198", + peer_id:"12D3KooWSJB2gZWi3ruVmtTF9JBCEBpCrJfuWCWzzRr8mMQWFQ9U" + }) +} +``` -### Archive Node Operators and Rosetta Operators -1. Ensure that all systems have been upgraded and prepared for the start of block production. -2. Monitor nodes and network health, and provide feedback to the engineering team in case of any issues. +Update a SNARK worker to use a fee of 0.1 MINA: -### Exchange and Builders -1. After the predefined Berkeley genesis timestamp, block production will commence, and MINA deposits and withdrawals can be resumed. -2. Ensure that all systems have been upgraded and prepared for the start of block production. -3. Monitor nodes and network health, and provide feedback to the engineering team in case of any issues. +``` +mutation { + setSnarkWorkFee(input: {fee: "100000000"}) +} +``` ---- -url: /network-upgrades ---- +### Subscriptions -# Network Upgrades +A GraphQL subscription allows a GraphQL client to have data pushed to it when an event occurs. In Mina, there are subscriptions for: -Mina protocol evolves through network upgrades (hard forks) that introduce new features and improvements. Each upgrade requires node operators to update their software to remain compatible with the network. +- _newSyncUpdate_ - occurs when the sync status of the node changes. +- _newBlock_ - occurs when a new block is received. +- chainReorganisation - occurs when the best tip of the node changes in a non-trivial way. -:::tip Mesa preflight hard fork completed +For example, to subscribe to all new blocks produced: -The Mesa preflight network hard-forked at **2026-04-27 13:00 UTC**. The post-fork chain is now running the Mesa release **`4.0.0-preflight-3f038cb`**, which raises the **transaction protocol version to 5.0.0**. +``` +subscription { + newBlock { + creator + stateHash + protocolState { + consensusState { + blockHeight + } + previousStateHash + } + } +} +``` -Operators must run **`4.0.0-preflight-3f038cb`** to remain on the preflight network. Earlier builds — including the stop-slot release `4.0.0-preflight-stop-2967b39` — are no longer compatible with the post-fork chain. +The new block subscription can also be limited to return only new blocks created by a defined public key with the `publicKey` argument. -zkApp developers must update to **`o1js@3.0.0-mesa.698ca`**, which targets transaction protocol v5.0.0; earlier o1js releases will produce transactions that the post-fork network rejects. +### GraphQL API and public Internet -See [Preflight Network](/network-upgrades/mesa/preflight-network) for the full upgrade path. +Exposing the GraphQL endpoint with full API support to the public Internet **is not** recommended. +However, you can start the Mina Daemon node with the `--open-limited-graphql-port` and `--limited-graphql-port ` CLI arguments to expose the GraphQL endpoint with limited API support. -::: +### Resources -| Upgrade | Status | Date | Key Changes | -|---------|--------|------|-------------| -| [Berkeley](/network-upgrades/berkeley/requirements) | Completed | June 2024 | zkApp programmability, recursive proofs, new transaction model, archive database migration | -| [Mesa](/network-upgrades/mesa/preflight-network) | Preflight hard fork completed | 2026-04-27 13:00 UTC (preflight) | Transaction protocol v5.0.0, automode upgrades, simplified archive migration | +- [5-minute introduction video on Mina GraphQL API](http://bit.ly/GraphQLAPIin5) Coda +- [First steps with the Mina GraphQL API](https://garethtdavies.com/crypto/first-steps-with-coda-graphql-api.html) +- [Introduction to GraphQL](https://graphql.org/learn/) --- -url: /network-upgrades/mesa/archive-upgrade +url: /node-developers --- -# Archive Upgrade +# Node Developers -This guide describes the general procedure for upgrading a Mina archive database from Berkeley to Mesa. The same steps apply to every Mesa deployment (preflight, devnet, mainnet) — only the archive package version differs. +Explore the codebase on [GitHub](https://github.com/MinaProtocol/mina). -:::tip Mesa preflight hard fork completed +To start contributing code to Mina, see the [Contributing Guide](/node-developers/contributing). The [protocol](https://github.com/MinaProtocol/mina/tree/master/src) and [CLI](https://github.com/MinaProtocol/mina/tree/master/src/app/cli) are written in OCaml. All levels of experience in any or all of these tools is welcome. -The Mesa preflight network hard-forked at **2026-04-27 13:00 UTC**. The post-fork chain runs **`4.0.0-preflight-3f038cb`**, which raises the **transaction protocol version to 5.0.0**. Pre-fork builds — including the stop-slot release `4.0.0-preflight-stop-2967b39` — are no longer compatible. +Other documents relevant to contributing code include: -When upgrading a preflight archive database, install the matching `mina-archive-mesa=4.0.0-preflight-3f038cb` package so the archive node speaks the new transaction protocol. +- [Style Guide](/node-developers/style-guide) +- [Code Review Guidelines](/node-developers/code-review-guidelines) +- [Repository Structure](/node-developers/repository-structure) +- [BIP44 Information](/node-developers/bip44) -::: +Mina is entirely open source. The code is distributed under the terms of the [Apache License 2.0](https://github.com/MinaProtocol/mina/blob/master/LICENSE). -:::info Choosing the Mesa archive version +--- +url: /node-developers/repository-structure +--- -The commands below use `` as a placeholder for the archive build you want to install. Substitute the tag that matches your target network: +# Repository Structure -- **Preflight network** — use `4.0.0-preflight-3f038cb` (post-hard-fork; transaction protocol v5.0.0). See [Preflight Network](./preflight-network) for the full release matrix. -- **Devnet / mainnet** — use the Mesa release tag published for that network when it is announced. +The file structure of the [Mina repository](https://github.com/minaprotocol/mina)the roles various files play: -::: +- `dockerfiles/` + Contains Docker-related scripts -To successfully upgrade the archive database to Mesa, you must ensure that your environment meets the foundational requirements. +- `docs/` + Documentation for the code and processes for contributing are here. The documentation website with the walkthrough docs lives in `frontend/website/docs`. -## Migration host +- `frontend/` + All code related to Mina frontend UIs and products -- PostgreSQL database for database server -- If you use Docker, then any of the supported OS by Mina (bullseye, focal, noble, bookworm or jammy) with at least 32 GB of RAM -- gsutil application from Google Cloud Suite in version 5 or later -- (Optional) Docker in version 23.0 or later + - `wallet/` + Source code for the Mina wallet -## Archive database + - `website/` + Code for https://minaprotocol.com -One of the most obvious prerequisites is a Mainnet database. If you don't have an existing database with Devnet/Mainnet archive data, -you can always download it from the O1Labs Google Cloud bucket. + - `posts/` + Markdown docs for blog posts -## Upgrade process + - `src/` + Source code for the website -### Upgrade script + - `static/` + Static files like images, etc. -Assuming that you have a PostgreSQL database with Mainnet archive data, in order to upgrade it to Mesa version, you need to run SQL upgrade script. -We put all efforts to make the upgrade process as smooth as possible. Script can be run on archive node which is online or offline. -Script can be run multiple times, it will skip steps that were already completed. It also performs sanity checks before each step to ensure that the upgrade process is successful. -Finally it creates new table (version) in the database to keep track of the upgrade process. +- `rfcs/` + This directory contains all accepted RFCs (or "requests for comments") made according to the [RFC process](https://github.com/MinaProtocol/mina/blob/master/CONTRIBUTING.md#rfcs). -#### Getting the script +- `scripts/` -You can find the SQL upgrade script in the Mina repository on GitHub. Make sure to download the latest version of the script before proceeding. -You can download the script directly using the following command: +- `src/` + All protocol source code, both application and library code, is in this directory. -```bash -curl -O https://raw.githubusercontent.com/MinaProtocol/mina/refs/heads/mesa/src/app/archive/upgrade_to_mesa.sql -``` + - `*.opam` + These files are needed for our `dune` build system. There must be one for each + library in `lib`. When you create a library `lib/foo_lib` with a `dune` file giving + the library's name as `foo_lib`, you must create a `foo_lib.opam` file. -We also ship the script in the Mina archive Docker image and Debian package. + - `config/` + Build time config - these .mlh files define compile time constants and their values. -```bash -docker run --rm gcr.io/o1labs-192920/mina-archive:-bookworm-mesa cat /etc/mina/archive/upgrade-to-mesa.sql > upgrade-to-mesa.sql -``` + - `app/` + Applications live here. -```bash -# Setup the Mina repository and install the archive package. -# Repository setup (apt sources, GPG key, channel) is network-specific — -# see Preflight Network for the preflight pin, or your network's release notes. + - `cli/` + This is the mina client/daemon. It is what you use to run a staker, a snarker, or a simple client for sending and receiving transactions. -apt-get install mina-archive-mesa= + - `website/` + Soon to be deprecated directory for the website - most of the code has migrated over to `frontend/website/` -# View the upgrade and downgrade scripts -cat /etc/mina/archive/upgrade-to-mesa.sql -cat /etc/mina/archive/downgrade-to-berkeley.sql -``` + - `reformat/` + This program runs `ocamlformat` on most of the files in the source tree, with a few exceptions. -#### Running the script + - `logproc/` + This utility reads from `stdin` and can filter and pretty print the log messages emitted by the mina daemon. -:::caution Database Backup + - `libp2p_helper/` + This program uses go-libp2p to implement the peer-to-peer plumbing that Mina daemons need. -Before running the upgrade script, **backup your archive database**. The upgrade modifies the database schema. + - `external/` + Local copies of external libraries which we've had to make some tweaks to. -```bash -pg_dump -U > berkeley-archive-backup.sql -``` + - `lib/` + Libraries powering mina. + The libraries here basically fall into two categories. + 1. General purpose data-types and functionality. This includes `snarky`, `fold_lib`, `vrf_lib`, `sgn`, and others. + 2. Application specific functionality, structured as a library. This includes `syncable_ledger`, `staged_ledger`, `transaction_snark`, and others. -::: +--- +url: /node-developers/sandbox-node +--- -To run the upgrade script, execute the following command: +# Sandbox Node -```bash -psql -U -d -f upgrade-to-mesa.sql -``` +The Mina Sandbox Node enables you to test and get familiar with core features of the protocol and build tooling in a stable environment -- it's a single-node private network that uses the same configuration as the live testnet. This sandbox supports multiple accounts, sending transactions between them, and also supports performing SNARK work, delegating, and staking. In fact since it's a single node network, you earn all the block rewards! -Make sure to replace `` and `` with your actual PostgreSQL username and database name. +:::info +The sandbox does **NOT** connect you to a live network. -#### Rollback +::: -You can rollback the upgrade process by restoring the database from a backup taken before running the upgrade script. -Another is to run rollback script which is part of the upgrade script. It will drop all tables and other database objects created by the upgrade script. -It will also update the version table to reflect the rollback. +## Installation -##### Running the rollback script +[Docker](https://www.docker.com) is a tool for portably running applications. The Mina Sandbox is packaged with Docker, and now built-in to our daemon containers. It’s easy to install--we suggest the [Docker Desktop](https://www.docker.com/products/docker-desktop). -To run the rollback script, you need to execute the following command: +After you have Docker installed run the following command to spin up the Mina Sandbox. -```bash -psql -U -d -f /etc/mina/archive/downgrade-to-berkeley.sql ``` +docker run \ + --publish 3085:3085 \ + -d \ + --name mina \ + -e RUN_DEMO=true \ + -e MINA_PRIVKEY_PASS='' \ + minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet -Make sure to replace `` and `` with your actual PostgreSQL username and database name. +``` +This command starts a daemon inside the docker container and exposes the GraphQL port (3085) to your computer. This port is used for communication with the client. This daemon automatical runs in the background with a block producer and SNARK worker. -### Post-upgrade steps - -After successfully running the upgrade script, you DO NOT need to restart your archive node or Rosetta API. -Changes in upgrade script are backward compatible and will be picked up by the archive node and Rosetta API automatically. - -### Verification -To verify that the upgrade was successful, you can check the version table in the PostgreSQL database. +You can view logs by executing. -You can do this by running the following command: -```bash -psql -U -d -c "SELECT * FROM version;" ``` -Make sure to replace `` and `` with your actual PostgreSQL username and database name. +docker logs --follow mina +``` -If the upgrade was successful, you should see the new version number in the output. +And stop mina by running. -We put a lot of effort into making the upgrade process as smooth as possible. -However, if you encounter any issues or need assistance, please reach out to the Mina community on [Discord](https://discord.gg/minaprotocol) or [GitHub Discussions](https://github.com/MinaProtocol/mina/discussions). +``` +docker stop mina +``` -## Appendix: Database Schema Changes +You can use the Mina CLI to interact with the sandbox node. The following command opens a shell inside the docker container from where you can issue any of the available [cli commands](/node-operators/reference/mina-cli-reference). -Below we present details of what was changed in the archive node database schema between Berkeley and Mesa versions. +``` +docker exec -it mina bash +``` -### Zkapp_state_nullable Additional Columns +### Account details -The `zkapp_state_nullable` table has been modified to include new columns `element8` through `element31` which are nullable and can store additional state information for zkApps. +The container has one account with this public key: -```sql -, element8 int REFERENCES zkapp_field(id) -... -, element31 int REFERENCES zkapp_field(id) -); +``` +B62qiZfzW27eavtPrnF6DeDSAKEjXuGFdkouC3T5STRa6rrYLiDUP2p ``` -This expansion allows zkApps to store up to 31 state elements instead of the previous 8, significantly increasing the state storage capacity for complex smart contracts. +The password for this account is the empty string (there's no password -- you can leave the password field blank). -### Version Table +## How to use the sandbox -We also introduced a new table `version` to keep track of the database schema version. -The purpose of this table is to help with future database migrations. The table tracks which migration scripts were applied and when. -Ultimately it helps to determine the current version of the database schema and helps to avoid applying the same migration script multiple times. +There are a few things you can do with your sandbox now that you have it running: -This table is created if it does not exist already. Rollback and upgrade scripts will insert a new row with the version number and timestamp when the script was applied. +- [Install Mina](/node-operators/validator-node/installing-on-ubuntu-and-debian) as usual and use many of the client commands. Since the daemon is already running in the container, you don't need to run `mina daemon`! -```sql -CREATE TABLE IF NOT EXISTS version ( - version_num INT PRIMARY KEY, - applied_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP -); -``` +- Install the GUI Wallet app to use a graphical interface to your node. Enter '127.0.0.1' as the host of your node during setup. -The version table provides: -- **Migration tracking**: Records which migrations have been applied -- **Timestamp tracking**: Shows when each migration was executed -- **Idempotency**: Prevents duplicate migration runs -- **Version identification**: Easily identify the current database schema version +- Head over to [http://localhost:3085/graphql](http://localhost:3085/graphql) to play with the GraphQL API directly. --- -url: /network-upgrades/mesa/preflight-network +url: /node-developers/style-guide --- -# Connect to Mesa Preflight Network - -The Mesa preflight network is a testing environment for validating the Mesa upgrade before deployment to devnet and mainnet. This network allows node operators and developers to test their infrastructure and applications with the new Mesa release. +# Style Guide -:::tip Mesa preflight hard fork completed +## Ocaml -The Mesa preflight network hard-forked at **2026-04-27 13:00 UTC**. The post-fork chain is now running the Mesa release **`4.0.0-preflight-3f038cb`**, which bumps the **transaction protocol version to 5.0.0**. +[ocaml]: #ocaml -The transaction protocol bump is a breaking wire-format change: any node still running a pre-fork build (including the stop-slot release `4.0.0-preflight-stop-2967b39` and the original preflight build `4.0.0-preflight1-b649c79`) will reject the new transaction format and is no longer compatible with the network. Upgrade to **`4.0.0-preflight-3f038cb`** to resume participation. +### General -zkApp developers must rebuild and redeploy against **`o1js@3.0.0-mesa.698ca`** — the first o1js release that targets transaction protocol v5.0.0. Transactions produced by earlier o1js versions will be rejected by post-fork nodes. +[ocaml-general]: #ocaml-general -A dedicated **Upgrade Steps** page covering the post-fork operator checklist will be linked from the Mesa Upgrade sidebar once it ships. +Our style guidelines are an extension of a couple of existing style guidelines. The first is ocamlformat, and it acts as the source of truth for most of our coding style. In fact, ocamlformat is a blocker on CI, so your code must be formatted by it's guidelines in order to be merged into master. Ocamlformat does not handle all important cases of style, however, as it is only defining and enforcing how code should be spaced out and indented. For anything which ocamlformat does not cover, the [Jane Street styleguide](https://opensource.janestreet.com/standards/) should be referenced. This styleguide we define here is intended to be an extension of the janestreet styleguide, with more attention to detail in concern to a few specific constructs we use regularly throughout our codebase. -::: +### Mli Files -:::caution Preflight Network Notice +[ocaml-mli]: #ocaml-mli -The preflight network is intended for testing purposes only. This network may experience instability, breaking changes, and unexpected behavior. Data on this network should not be considered persistent or reliable. +A `*.mli` file should not be included for a `*.ml` file if the `*.ml` file's automatically derived interface is different. Many `*.ml` files in our codebase consist of only signatures and a functor. In the case of those files, there is not purpose to redefining the `*.mli` file because there is no new or restricted information in that file. If a `*.ml` file contains implementations in the root structure, then a `*.mli` file should most likely be created. -::: +### Modules -## Available Build Versions +[ocaml-modules]: #ocaml-modules -The Mesa preflight network uses the following build versions: +#### Prefer Standardized Shortnames -**Current version (install this):** `4.0.0-preflight-3f038cb` — post-hard-fork release. Implements transaction protocol version **5.0.0** and is the only build compatible with the post-fork chain. +The names `t`, `T`, and `S` are common shortnames used in modules to signify specific things. -**Compatible o1js release:** `o1js@3.0.0-mesa.698ca` — the matching SDK for transaction protocol v5.0.0. Install with `npm install o1js@3.0.0-mesa.698ca` (or the equivalent for your package manager) when developing or redeploying zkApps against the post-fork preflight network. +The name `t` is used to represent the root type of a module. For instance, if there is a module `Account` which contains types and values related to accounts, then `Account.t` is the type of an account. The name `t` can also be used as a value iff there is only intended to be one value of the root type of the module. As an example, if you wanted to have a single global logger in a `Logger` module, the type `Logger.t` could be the type of a logger, and the value `Logger.t` could be the global logger value of type `Logger.t`. -**Previous versions (historical, do not install on new nodes):** +The module name `T` is used to encapsulate the root type and basic definitions regarding a root type of a module. It is a common practice used when you want to instantiate some functors for a module's root type and have the instantiations appear in the module itself. As an example, it is common to call the `Comparable.Make` functor in order to derive various helper values/modules from a comparable type. In this case, if we had a module `Account` again, and we wanted to derive the `Comparable.S` signature, then we would define a module `T` in `Account` which defines a root type `t` and the required functions for the `Comparable.Make` functor argument (in this case, `compare`). With this `T` module, we can then `include T` and `include Comparable.Make (T)` in the `Account` module to bring in all related values/modules for the `Account.t` type. Here is a full example of that: -- `4.0.0-preflight-stop-2967b39` — stop-slot release used to carry nodes up to the fork. Pre-fork transaction protocol; incompatible with the post-fork chain. -- `4.0.0-preflight1-b649c79` — original preflight build. Listed for reference only. +``` +module Account = struct + module T = struct + type t = ... [@@deriving compare] + end -### Docker Images + include T + include Comparable.Make (T) +end +``` -Docker images are available for both `amd64` and `arm64` architectures, on the `bookworm` and `noble` base distributions: +The module type name `S` is used for defining the root signature of a module. This is most commonly used when you have a module which contains a functor. In this case, we typically call the functor `Make` and declare the functor returns the type `S`, putting both of these values in the same module. Looking back at our previous example, `Core_kernel`'s `Comparable` module follows this pattern: `Comparable.Make` is a functor which returns a `Comparable.S`. -- **Mina Daemon:** `gcr.io/o1labs-192920/mina-daemon:4.0.0-preflight-3f038cb-bookworm-mesa` -- **Archive Node:** `gcr.io/o1labs-192920/mina-archive:4.0.0-preflight-3f038cb-bookworm-mesa` -- **Rosetta:** `gcr.io/o1labs-192920/mina-rosetta:4.0.0-preflight-3f038cb-bookworm-mesa` +#### Prefer One Type Per Module -For `noble`, swap `bookworm` for `noble` in the tag (e.g. `mina-daemon:4.0.0-preflight-3f038cb-noble-mesa`). +[ocaml-modules-singleton-types]: #ocaml-modules-singleton-types -### Debian Packages +As a general rule of thumb, each module should be scoped to a single type. This pattern helps isolate concerns and, in turn, allows value names to be shorter, as they are located by context. Take, for example, a `Merkle_tree` module. This module will need a type `Merkle_tree.t` which represents the entire merkle tree (or a node of it). A `Merkle_tree` will also want to have a `path` type. It is preferable to place this `path` type into it's own nested module (`Merkle_tree.Path.t` instead of `Merkle_tree.path`). To help understand why this is preferable, imagine we did put path in `Merkle_tree.path`. Now, `Merkle_tree` contains values (functions) that relate not only to the merkle tree type itself, but also the a path of a merkle tree. For clarity, it would be natural to prepend all of the value names related to a path with `path_` (`path_map`, `path_length`, etc...). By isolating `Path` to it's own module, we can shorten these names while keep the context of values clear. Additionally, if we choose to in the future, we may encapsulate the implementation details of `Path` by applying a restrictive signature to it, which would make the separation of concerns more clear via compiler enforcement. -Debian packages are available from the unstable repository: +#### No Monkeypatching -**Repository:** `unstable.apt.packages.minaprotocol.com` -**Channel:** `preflight` -**Codenames:** `bookworm`, `noble` +[ocaml-modules-monkeypatching]: #ocaml-modules-monkeypatching -Available packages (each at version `4.0.0-preflight-3f038cb`): +Monkeypatching of modules is explicitly disallowed in our codebase. Monkeypatching is defined as the act of taking an existing module and redefining it with extended or modified values. More simply, it's anything of the form. -- `mina-mesa` -- `mina-archive-mesa` -- `mina-rosetta-mesa` +``` +module A = struct + module M = struct + let x = ... + end +end -For detailed information about the Debian repository structure and configuration, please refer to the [Debian Repository documentation](https://unstable.apt.packages.minaprotocol.com/). +module M = struct + include A.M + let y = ... + (* or `let x = ...` *) +end +``` -## Connecting to the Network +Monkeypatching may be the easiest path to getting code to compile sometimes, but in general, it creates confusion and/or technical debt in the codebase. If you need to monkeypatch a module, you should have a good reason as to why. -To connect your node to the Mesa preflight network, you must use the preflight network seed peer list. +#### Functor Signature Equalities -### Required Parameter +[ocaml-modules-functor-patterns]: #ocaml-modules-functor-patterns -Add the following parameter to your mina daemon command: +Signature `with` statements for signatures of modules generated by functors should be limited to the form `S with module M1 = M2` whenever possible. Replacement equalities `:=` should be limited to `include` statements where portions of the signature need to be limited (for example, when a nested module in the signature is already defined at the current structure scope). The form `S with type t = ...` is also not preferred as it scales poorly as the number of common dependencies between signatures involved with a functor increases. Note that this places increased importance on the janestreet styleguide rule "Prefer standard signature includes to hand-written interfaces". -```bash ---peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt -``` +#### Functor Arity -### Example: Running with Docker +[ocaml-modules-functor-arity]: #ocaml-modules-functor-arity -To start a Mina daemon node using Docker: +Functor can have a maximum arity of 3 (arity is the number of arguments; in this case, the number of nested functors - functors returning functors). If a functor requires more than 3 modules as arguments, then the required modules should all be nested into one module. The standard pattern for this is to define a signature `Inputs_intf` for your functor, which will, in turn, define the module arguments to the functor. See below for a simple example. -```bash -docker run --name mina-mesa-preflight -d \ - -p 8302:8302 \ - --restart=always \ - gcr.io/o1labs-192920/mina-daemon:4.0.0-preflight-3f038cb-bookworm-mesa \ - daemon \ - --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt ``` +module type Inputs_intf = sig + module A : A.S + module B : B.S + module C : C.S + module D : D.S +end -For production deployments, you'll want to mount configuration directories and add additional flags: - -```bash -docker run --name mina-mesa-preflight -d \ - -p 8302:8302 \ - --restart=always \ - -v $(pwd)/.mina-config:/root/.mina-config \ - gcr.io/o1labs-192920/mina-daemon:4.0.0-preflight-3f038cb-bookworm-mesa \ - daemon \ - --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt \ - --libp2p-keypair /root/.mina-config/keys/libp2p-key -``` +module type S = sig + include Inputs_intf -### Example: Running with Debian Package + (* ... *) +end -First, install dependencies and configure the repository: +module Make (Inputs : Inputs_intf) + : S + with module A = Inputs.A + and module B = Inputs.B + and module C = Inputs.C + and module D = Inputs.D = +struct + open Inputs -```bash -# Step 1: Install dependencies -sudo apt-get install -y lsb-release ca-certificates wget gnupg + (* ... *) +end +``` -# Step 2: Import the GPG key -wget -q https://unstable.apt.packages.minaprotocol.com/repo-signing-key.gpg \ - -O /etc/apt/trusted.gpg.d/minaprotocol.gpg +# Code Idiosyncrasies -# Step 3: Add the unstable repository with preflight channel -echo "deb https://unstable.apt.packages.minaprotocol.com $(lsb_release -cs) preflight" | \ - sudo tee /etc/apt/sources.list.d/mina-preflight.list +We use a particular style of OCaml. Here's some of the important things. -# Step 4: Update and install -sudo apt-get update -sudo apt-get install -y mina-mesa=4.0.0-preflight-3f038cb -``` +## Parameterized records -Then start the daemon: +```ocaml +type ('payload, 'pk, 'signature) t_ = + {payload: 'payload; sender: 'pk; signature: 'signature} +[@@deriving eq, sexp, hash] -```bash -mina daemon \ - --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt \ - --libp2p-keypair ~/.mina-config/keys/libp2p-key -``` +type t = (Payload.t, Public_key.t, Signature.t) t_ +[@@deriving eq, sexp, hash] -### Running Archive Node +(* ... *) -:::info Upgrading from Berkeley to Mesa +type var = (Payload.var, Public_key.var, Signature.var) t_ +``` -If you have an existing Berkeley archive database that you want to upgrade to Mesa, please refer to the comprehensive [Archive Upgrade](archive-upgrade) guide for detailed instructions on: -- Migration prerequisites and requirements -- Running the upgrade script -- Rollback procedures -- Database schema changes -- Verification steps +We're defining a base type `t_` with type variables for all types of record fields. Then we define the record using these type variables. Finally, we instantiate the record with `type t`, this is the OCaml type. And also `type var` this is the type of this value in a SNARK circuit. We'll cover this more later. Whenever we want something to be programmable from within a SNARK circuit we define it in this manner so we can reuse the record definition across both types. -::: +There is some talk of moving to OCaml object types to do this sort of thing so we don't need to deal with positional arguments. Perhaps I (@bkase) will write up an RFC for that at some point. -#### Installing and Running Mesa Archive Node +### Ppx_deriving -An archive node stores the full history of the blockchain and provides a GraphQL API for querying historical data. It requires a PostgreSQL database. +```ocaml +type t = int [@@deriving sexp, eq] +``` -**Installation with Debian:** +This is the first time we've seen a macro. Here we use `sexp` from [ppx_jane](https://github.com/janestreet/ppx_jane) and `eq` from [ppx_deriving](https://github.com/ocaml-ppx/ppx_deriving). -```bash -# Step 1: Install dependencies -sudo apt-get install -y lsb-release ca-certificates wget gnupg postgresql postgresql-contrib +### Stable.V1 -# Step 2: Import the GPG key -wget -q https://unstable.apt.packages.minaprotocol.com/repo-signing-key.gpg \ - -O /etc/apt/trusted.gpg.d/minaprotocol.gpg +```ocaml +module Stable : sig + module V1 : sig + type t = (* ... *) + [@@deriving bin_io, (*...*)] + end +end +``` -# Step 3: Add the repository -echo "deb https://unstable.apt.packages.minaprotocol.com $(lsb_release -cs) preflight" | \ - sudo tee /etc/apt/sources.list.d/mina-preflight.list +Whenever a type is serializable, it's important for us to maintain backwards compatibility once we have a stable release. Ideally, we wouldn't define `bin_io` on any types outside of `Stable.V1`. When we change the structure of the datatype we would create a `V2` under `Stable`. -# Step 4: Install mina-archive-mesa -sudo apt-get update -sudo apt-get install -y mina-archive-mesa=4.0.0-preflight-3f038cb +### Property based tests -# Step 5: Create PostgreSQL database -sudo -u postgres createdb archive -sudo -u postgres createuser archive_user -sudo -u postgres psql -c "ALTER USER archive_user WITH PASSWORD 'your-secure-password';" -sudo -u postgres psql -c "GRANT ALL PRIVILEGES ON DATABASE archive TO archive_user;" +[Core](https://opensource.janestreet.com/core/) has an implementation of [QuickCheck](https://blog.janestreet.com/quickcheck-for-core/) that we use whenever we can in unit tests. Here is an example signature for a `Quickcheck.Generator.t` of payments. -# Step 6: Start the archive node -mina-archive run \ - --postgres-uri postgresql://archive_user:your-secure-password@localhost:5432/archive \ - --server-port 3086 +```ocaml +(* Generate a single payment between + * $a, b \in keys$ + * for fee $\in [0,max_fee]$ + * and an amount $\in [1,max_amount]$ + *) + +val gen : + keys:Signature_keypair.t array + -> max_amount:int + -> max_fee:int + -> t Quickcheck.Generator.t ``` -**Using Docker:** + -```bash -# Start the archive node (assumes you have a PostgreSQL server running) -docker run --name mina-archive-mesa -d \ - -p 3086:3086 \ - --restart=always \ - gcr.io/o1labs-192920/mina-archive:4.0.0-preflight-3f038cb-bookworm-mesa \ - mina-archive run \ - --postgres-uri postgresql://archive_user:your-secure-password@postgres-host:5432/archive \ - --server-port 3086 -``` +### Typesafe invariants (help with naming this section) -**Connecting Mina Daemon to Archive:** +In Mina, very important checks are frequently performed on certain pieces of data. +For example, we need to confirm that the signature is valid on a user-command we receive over the network. +Such checks can be expensive, so we only want to do them once, but we want to remember that we've done them. -Configure your mina daemon to send blocks to the archive node: +```ocaml +(* inside user_command.mli *) -```bash -mina daemon \ - --peer-list-url https://storage.googleapis.com/o1labs-gitops-infrastructure/mina-mesa-network/mina-mesa-network-seeds.txt \ - --archive-address localhost:3086 +module With_valid_signature : sig + type nonrec t = private t [@@deriving sexp, eq] + + (*...*) +end + +val check : t -> With_valid_signature.t option ``` -### Installing and Running Rosetta +Here we define `With_valid_signature` (usage will be `User_command.With_valid_signature.t`) using `type nonrec t = private t` to allow upcasting to a `User_command.t`, but prevent downcasting. The _only_ way to turn a `User_command.t` into a `User_command.With_valid_signature.t` is to `check` it. Now the compiler will catch our mistakes. -Rosetta is a standardized API for blockchain integration. The Rosetta API requires both a mina daemon and an archive node. +### Unit Tests -**Installation with Debian:** +We use [ppx_inline_test](https://github.com/janestreet/ppx_inline_test) for unit testing. Of course whenever we can, we combine that with `QuickCheck`. -```bash -# Step 1: Install dependencies (if not already done) -sudo apt-get install -y lsb-release ca-certificates wget gnupg +```ocaml +let%test_unit = + Quickcheck.test ~sexp:[%sexp_of: Int.t] Int.quickcheck_generator + ~f:(fun x -> assert (Int.equal (f_inv (f x)) x)) +``` -# Step 2: Import the GPG key (if not already done) -wget -q https://unstable.apt.packages.minaprotocol.com/repo-signing-key.gpg \ - -O /etc/apt/trusted.gpg.d/minaprotocol.gpg +### Functors -# Step 3: Add the repository (if not already done) -echo "deb https://unstable.apt.packages.minaprotocol.com $(lsb_release -cs) preflight" | \ - sudo tee /etc/apt/sources.list.d/mina-preflight.list +We are in the process of migrating to using module signature equalities -- see [the above section](#functor-signature-equalities) and [the rfc for rationale](https://github.com/MinaProtocol/mina/blob/master/rfcs/0004-style-guidelines.md), but we still have a lot of code using type substitutions (`with type foo := bar`). -# Step 4: Install mina-rosetta-mesa -sudo apt-get update -sudo apt-get install -y mina-rosetta-mesa=4.0.0-preflight-3f038cb +First we define the resulting module type of the functor, keeping all types we'll be functoring in abstract. -# Step 5: Start Rosetta -# Rosetta connects to both the mina daemon and archive node -mina-rosetta \ - --port 3087 \ - --archive-uri http://localhost:3086/graphql \ - --graphql-uri http://localhost:3085/graphql +```ocaml +module type S = sig + type boolean_var + type curve + type curve_var + (*...*) +end ``` -**Using Docker:** +Then we define the functor: -```bash -docker run --name mina-rosetta-mesa -d \ - -p 3087:3087 \ - --restart=always \ - gcr.io/o1labs-192920/mina-rosetta:4.0.0-preflight-3f038cb-bookworm-mesa \ - --port 3087 \ - --archive-uri http://archive-host:3086/graphql \ - --graphql-uri http://mina-daemon-host:3085/graphql +```ocaml +module Schnorr + (Impl : Snark_intf.S) + (Curve : sig (*...*) end) + (Message : Message_intf + with type boolean_var := Impl.Boolean.var + (*...*)) +: S with type boolean_var := Impl.Boolean.var + and type curve := Curve.t + and type curve_var := Curve.var + (*...*) += struct + (* here we implement the signature described in S *) +end ``` -**Note:** Rosetta requires: -- A running mina daemon (GraphQL endpoint, typically port 3085) -- A running archive node (GraphQL endpoint, typically port 3086) -- Both must be fully synced for Rosetta to function properly +### Custom SNARK circuit logic -## Nightly Builds +This is also the first time we see custom SNARK circuit logic. A pattern we've been using is to scope all operations that you'd want to run inside a SNARK under a submodule `module Checked`. -In addition to the preflight release builds, nightly builds are also available for testing the latest changes. +For example, inside [sgn.mli](https://github.com/MinaProtocol/mina/blob/master/src/lib/sgn/sgn.mli) we see: -**Repository:** `nightly.apt.packages.minaprotocol.com` +```ocaml +(* ... *) +val negate : t -> t -Nightly builds are available for both Debian packages and Docker images. These builds represent the latest development state and may be even more unstable than preflight releases. +module Checked : sig + val negate : var -> var +end +``` -For more information about nightly builds and repository configuration, see the [Debian Repository documentation](https://nightly.apt.packages.minaprotocol.com/). +`negate` is the version of the function that runs in OCaml, and `Checked.negate` is the one that runs inside of a SNARK circuit. -## Verification +--- +url: /node-operators/archive-node/archive-redundancy +--- -After starting your node, verify connectivity to the preflight network: +# Archive Redundancy -### Check Node Status +The [archive node](/node-operators/archive-node/getting-started) stores its data in a PostgreSQL database that node operators host on a provider of their choice, including self-hosting. For redundancy, archive node data can also be stored to an object storage like [Google Cloud Storage](#upload-block-data-to-google-cloud-storage); soon S3 and others) or to a [`mina.log`](#save-block-data-from-logs) file that reside on your computer or be streamed to any typical logging service, for example, LogDNA. -```bash -# For Docker -docker exec -it mina-mesa-preflight mina client status +Archive data is critical for applications that require historical lookup. -# For Debian installation -mina client status -``` +On the protocol side, archive data is important for disaster recovery to reconstruct a certain state. A single [archive node](/node-operators/archive-node/getting-started) set up might not be sufficient. -### Monitor Logs +If the daemon that sends blocks to the archive process or if the archive process itself fails for some reason, there can be missing blocks in the database. To minimize the risk of archive data loss, employ redundancy techniques. -```bash -# For Docker -docker logs -f mina-mesa-preflight +A single archive node setup has a daemon sending blocks to an archive process that writes them to the database. -# For systemd service (Debian) -journalctl -u mina -f +To connect multiple daemons to the archive process, specify the address of an archive process in multiple daemons to reduce the dependency on a single daemon to provide blocks to the archive process. + +For example, the server port of an archive process is 3086. The daemons can connect to that port using the flag `archive-address` + +``` +mina daemon \ + ..... + --archive-address :3086\ ``` -## Support and Feedback +Similarly, it is possible to have multiple archive processes write to the same database. In this case, the `postgres-uri` passed to the archive process is the same across multiple archive processes. -If you encounter issues or have feedback about the Mesa preflight network: +However, multiple archive processes concurrently writing to a database could cause data inconsistencies (explained in https://github.com/MinaProtocol/mina/issues/7567). To avoid this, set the transaction isolation level of the archive database to `Serializable` with the following query: -1. Check the [Mina Protocol Discord](https://discord.gg/minaprotocol) for community support -2. Report issues on the [Mina GitHub repository](https://github.com/MinaProtocol/mina/issues) -3. Join the Mesa upgrade discussions in the community channels + ALTER DATABASE `` SET DEFAULT_TRANSACTION_ISOLATION TO SERIALIZABLE ; -## Next Steps +Set the transaction level after you create the [database](/node-operators/archive-node/getting-started) and before you connect an archive process to it. -- Explore additional Mesa upgrade documentation -- Test your applications and infrastructure against the preflight network -- Provide feedback to help improve the Mesa upgrade process +## Back up block data ---- -url: /node-developers/bip44 ---- +To ensure that archive data can be restored, use the following features to back up and restore block data. -# BIP44 Information +A mechanism for logging a high-fidelity machine-readable representation of blocks using JSON includes some opaque information deep within. -| index | hexa | symbol | coin | -|:------|:-----------|:-------|:----------------------------------| -| 12586 | 0x8000312a | MINA | [Mina](https://minaprotocol.com/) | +These logs are used internally to quickly replay blocks to get to certain chain states for debugging. This information suffices to recreate exact states of the network. -Mina uses the 5 level [BIP32](https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki) path format specified in [BIP44](https://github.com/bitcoin/bips/blob/master/bip-0044.mediawiki) +Some of the internal data look like this: -``` -m / purpose' / coin_type' / account' / change / address_index +```json +{"data":["Signed_command",{"payload":{"common":{"fee":"100","fee_token":"1","fee_payer_pk":"B62qixbmBBmCmv1xH5SeF1hw6EqwSNVPi9B28epa3phqVMSyuZk9EoH","nonce":"340","valid_until":"4294967295","memo":"E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"},"body":["Payment",{"source_pk":"B62qixbmBBmCmv1xH5SeF1hw6EqwSNVPi9B28epa3phqVMSyuZk9EoH","receiver_pk":"B62qm2GCuGCEK79mEjeyaeiFoukThuZLJCHGe9HAzuAnfbtS5FHtPnP","token_id":"1","amount":"100000000"}]},"signer":"B62qixbmBBmCmv1xH5SeF1hw6EqwSNVPi9B28epa3phqVMSyuZk9EoH","signature":"7mXGz8Df1gu92HVWGue24wcrGxDWkgQrDK59xQGXc627PKFQvVAPSzZn7JMkHtfdBUXavDHcgLBZy4iR4UmA5seRCPMkFDci"}],"status":["Applied",{"fee_payer_account_creation_fee_paid":null,"receiver_account_creation_fee_paid":null,"created_token":null},{"fee_payer_balance":"31866000100000","source_balance":"31866000100000","receiver_balance":"34099000000"}]}],"coinbase":["One",null],"internal_command_balances":[["Coinbase",{"coinbase_receiver_balance":"75477804514901","fee_transfer_receiver_balance":null}],["Fee_transfer",{"receiver1_balance":"65266376010003","receiver2_balance":"78601129170700"}],["Fee_transfer",{"receiver1_balance":"66001820000000","receiver2_balance":"76870784414900"}],["Fee_transfer",{"receiver1_balance":"71158365898775","receiver2_balance":"59264207944722"}],["Fee_transfer",{"receiver1_balance":"68546088449962","receiver2_balance":"66721919100000"}],["Fee_transfer",{"receiver1_balance":"67700798001000","receiver2_balance":"66372760000000"}],["Fee_transfer",{"receiver1_balance":"85383891400000","receiver2_balance":"107174952265469"}],["Fee_transfer",{"receiver1_balance":"65879310000000","receiver2_balance":"66282230000000"}]]}]},"delta_transition_chain_proof":["jxLZWooV57gKCmanzCHHt1CDbHfUpMu6MkynUdqN9ZkBUJi7B1W",[]]} ``` -Keypairs are derived by varying only the ``account`` while keeping the ``change`` and ``address_index`` zero. +This JSON evolves as the format of the block and transaction payloads evolve in the network. -# BIP32 path format +### Upload block data to Google Cloud Storage -``` -m / 44' / 12586' / account' / 0 / 0 -``` +The daemon generates a file for each block with the name `-.json` . These files are called precomputed blocks and have all the fields of a block. -# Examples +To specify a daemon to upload block data to Google Cloud Storage, pass the flag `--upload-blocks-to-gcloud`. -| account | path | -|:--------|:------------------------| -| 0 | m/44'/12586'/0'/0/0 | -| 271 | m/44'/12586'/271'/0/0 | +Set the following environment variables: ---- -url: /node-developers/code-review-guidelines ---- +- `GCLOUD_KEYFILE`: Key file for authentication -# Code Review Guidelines +- `NETWORK_NAME`: Network name to use in the filename to easily distinguish between blocks in different networks (Mainnet and Testnets) -A good pull request: +- `GCLOUD_BLOCK_UPLOAD_BUCKET`: Google Cloud Storage bucket where the files are uploaded -- Does about one thing (new feature, bug fix, etc) -- Adds tests and documentation for any new functionality -- When fixing a bug, adds or fixes test that would have caught said bug +### Save block data from logs -## OCaml things +The daemon logs the block data if the flag `-log-precomputed-blocks` is passed. -- Are the [style guidelines](./style-guide) being followed? -- Do the signatures make sense? Are they minimal and reusable? -- Does anything need to be functored over? -- Are there any error cases that aren't handled correctly? -- Are calls to `_exn` functions justified? Are their preconditions for not - throwing an exception met? Is the exception it throws useful? -- There shouldn't be commented out code. -- No stray debug code lying around. -- Any logging is appropriate. All `Logger.trace` logs should be inessential, - because they won't be shown to anyone by default. -- Should this code live in its library? Should it live in a different library? -- Does the code confuse you? Maybe there should be a comment, or it should be - structured differently. -- Does a behavior change break assumptions other code makes? +The log to look for is `Saw block with state hash $state_hash` that contains `precomputed_block` in the metadata and has the block information. These precomputed blocks contain the same information that gets uploaded to Google Cloud Storage. ---- -url: /node-developers/codebase-overview ---- +### Generate block data from another archive database -# Codebase Overview +From a fully synced archive database, you can generate block data for each block using the `mina-extract-blocks` tool. -The Mina Protocol is written in OCaml, a statically typed, functional programming language. +The `mina-extract-blocks` tool generates a file for each block with name `.json`. -For OCaml beginners, it may help to skim through [Real World OCaml](https://realworldocaml.org/) for a good introduction to the language, and some deep dives into specific topics if you're interested. -Assuming basic familiarity with OCaml, here's some more info on how it is used in the Mina Protocol. +The tool takes an `--archive-uri`, an `--end-state-hash`, and an optional `--start-state-hash`, and writes all the blocks in the chain starting from start-state-hash and ending at end-state-hash (including start and end). -## Code Structure +If only the end hash is provided, then the tool generates blocks starting with the unparented block closest to the end block. This would be the genesis block if there are no missing blocks in between. The block data in these files are called extensional blocks. Since these blocks are generated from the database, they have only the data stored in the archive database and do not contain any other information pertaining to a block (for example, blockchain SNARK) like the precomputed blocks and can only be used to restore blocks in the archive database. -See the [Repository Structure page](/node-developers/repository-structure). +Provide the flag `--all-blocks` to write out all blocks contained in the database. -## Compilation +## Identify missing blocks -The OCaml compiler can target bytecode and native compilation. The code statically links with some libraries so it can't compile to bytecode. The code doesn't play well with the REPL. Dune, the build system, has a concept of folders that represent modules and files that a module. If the folder has a file with the same name, it's essentially equivalent to `index.js` in Node. +To determine any missing blocks in an archive database, use the `mina-missing-block-auditor` tool. -Interface files in OCaml with the `.mli` extension contain type signatures and structures for a module. The corresponding implementation must have the same file name with the `.ml` extension. Only the things defined in the interface are available from other modules. If an interface file does not exist for a module, everything is exposed by default. The same convention and rules apply to files with the `.rei` and `.re` extensions. +The tool outputs a list of state hashes of all the blocks in the database that are missing a parent. You can use this list to monitor the archive database for any missing blocks. To specify the URI of the PostgreSQL database, use the `--archive-uri` flag. -For the linking step, `dune` uses `ldd` under the hood. You can also use things like `-O3` for optimization. For debugging, you can use `gdb`. +## Restore blocks -## Open-source Library Documentation +When you have block data (precomputed or extensional) available from [Back up block data](/node-operators/archive-node/archive-redundancy#back-up-block-data), you can restore missing blocks in an archive database using the tool `mina-archive-blocks`. -There are multiple libraries for OCaml. One challenge with learning OCaml is locating and reading documentation for the various libraries. For example, the Jane Street `Core` library has the following structure: +1. Restore precomputed blocks from: + + - [Upload block data to Google Cloud Storage](/node-operators/archive-node/archive-redundancy#upload-block-data-to-google-cloud-storage) + + - [Save block data from logs](/node-operators/archive-node/archive-redundancy#save-block-data-from-logs) ``` - Base - | - Core_kernel -> Async_kernel - | | - Unix <- Core -> Async + mina-archive-blocks --precomputed --archive-uri FILES ``` -In general, the source code of an installed library is not available, so follow these tips to find the documentation. - -To review the docs for Core, a standard library overlay. Core is a popular alternative to the OCaml standard library. First, go to the [Core](https://github.com/janestreet/core) codebase on GitHub and then locate the correct documentation. If you don't see the module you're looking for, go to next to `Core_kernel`. If that fails, then look for `Base`. To use the find capability in GitHub, expand the sections. +2. For extensional blocks: (Generated from option [3](#generate-block-data-from-another-archive-database)) -Most documentation is published in HTML. However, you can use the an IDE to find and review code and documentation, like the Merlin editor service that provides modern IDE features for OCaml. Merlin provides type hints and code navigation, like Go To Definition. Note that Merlin works only if your code compiles. +``` + mina-archive-blocks --extensional --archive-uri FILES +``` -OPAM, the source-based package manager for OCaml, usually ships documentation with libraries that you can access using Merlin. +## Staking ledgers -## Extensions +Staking ledgers are used to determine slot winners for each epoch. Mina daemon stores staking ledger for the current and the next epoch after it is finalized. When transitioning to a new epoch, the "next" staking ledger from the previous epoch is used to determine slot winners of the new epoch and a new "next" staking ledger is chosen. Since staking ledgers for older epochs are no longer accessible, you can still keep them around for reporting or other purposes. -OCaml uses the ppx meta-programming system that generates code at compile time. +Export these ledgers using the mina cli command: -For example, consider this ppx extension on a type signature: + mina ledger export [current-staged-ledger|staking-epoch-ledger|next-epoch-ledger] -``` - type t = - | A - | B [@ to_yojson f] +Epoch ledger transition happens once every 14 days (given slot-time = 3mins and slots-per-epoch = 7140). - [@@ deriving yojson] -``` +The window to backup a staking ledger is ~27 days considering "next" staking ledger is finalized after k (currently 290) blocks in the current epoch and therefore is available for the rest of the current epoch and the entire next epoch. -The single `@` scopes an extension to a single expression. The `@@` denotes the extension is expanded in the scope of the calling context. +--- +url: /node-operators/archive-node/docker-compose +--- -For an extension on a structure or a value, use the following syntax. +# Docker Compose Archive -`%` returns a value/expression. +This example demonstrates how to run a Mina archive node using Docker Compose for the Mainnet network. This Docker Compose setup includes a Postgres database, a bootstrap database with the latest SQL Dump available, an archive node, a Mina node and a Missing Blocks Guardian script to monitor and populate the gaps in the archive database -`%%` injects a statement +Copy and paste the provided configuration into a `docker-compose.yml` file. Then run `docker compose up -d` to start the services, and use `docker compose logs -f` to monitor the logs. +```yaml +services: + postgres: + image: postgres:17 + restart: always + environment: + POSTGRES_PASSWORD: postgres + POSTGRES_DB: archive + healthcheck: + test: ["CMD-SHELL", "psql -U postgres -d archive -tAc \"SELECT COUNT(*) FROM pg_database WHERE datname='archive';\" | grep -q '^1$'"] + interval: 5s + timeout: 10s + retries: 10 + volumes: + - './archive/postgresql/data:/var/lib/postgresql/data' + ports: + - '5432:5432' + bootstrap_db: + image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' + # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet + command: > + bash -c ' + curl -O https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-$(date +%F_0000).sql.tar.gz; + tar -zxvf mainnet-archive-dump-$(date +%F_0000).sql.tar.gz; + psql postgres://postgres:postgres@postgres:5432/archive -c " + ALTER SYSTEM SET max_connections = 500; + ALTER SYSTEM SET max_locks_per_transaction = 100; + ALTER SYSTEM SET max_pred_locks_per_relation = 100; + ALTER SYSTEM SET max_pred_locks_per_transaction = 5000; + " + psql postgres://postgres:postgres@postgres:5432/archive -f mainnet-archive-dump-$(date +%F_0000).sql; + ' + # For Devnet Network, replace "mainnet" references with "devnet" in the block above + depends_on: + postgres: + condition: service_healthy + missing_blocks_guardian: + image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' + # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet + command: > + bash -c ' + curl -O https://raw.githubusercontent.com/MinaFoundation/helm-charts/main/mina-archive/scripts/missing-blocks-guardian-command.sh; + export GUARDIAN_PRECOMPUTED_BLOCKS_URL=https://673156464838-mina-precomputed-blocks.s3.us-west-2.amazonaws.com/mainnet; + export MINA_NETWORK=mainnet; + export PG_CONN=postgres://postgres:postgres@postgres:5432/archive; + while true; do bash missing-blocks-guardian-command.sh; sleep 600; done + ' + # For Devnet Network, replace "mainnet" references with "devnet" in the block above + depends_on: + bootstrap_db: + condition: service_completed_successfully + mina_archive: + image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' + restart: always + command: + - mina-archive + - run + - --postgres-uri + - postgres://postgres:postgres@postgres:5432/archive + - --server-port + - "3086" + volumes: + - './archive/data:/data' + depends_on: + bootstrap_db: + condition: service_completed_successfully + mina_node: + image: 'minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet' + # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet + restart: always + entrypoint: [] + command: > + bash -c ' + mina daemon --archive-address mina_archive:3086 \ + --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ + --insecure-rest-server \ + --rest-port 3085 + ' + # use --peer-list-url https://bootnodes.minaprotocol.com/networks/devnet.txt for Devnet + ports: + - '3085:3085' + - '8302:8302' + depends_on: + bootstrap_db: + condition: service_completed_successfully ``` - let x = [% ...] - [%% ...] - let y = - let%... z = ... in - match%... ... with - | ... - | ... in - [%% if x] - let x = y - [%% else] - let x = z - [%% endif] -``` +Once the services are running, you can access the Mina node graphql endpoint at `http://localhost:3085/graphql` and the postgres database using `psql postgres://postgres:postgres@localhost:5432/archive`. -**TL;DR** Anytime you see `[@ ...]` `[@@ ...]` `[% ...]` `[%% ...]` it's an OCaml language extension. +To retrieve the status of the Mina Node, run `docker compose exec mina_node mina client status` -## Monads +--- +url: /node-operators/archive-node/getting-started +--- -Functional design patterns that allows you to write computation in a very generic way, that removes the need for boilerplate code. Monads allow us to abstract up to a higher level of computation, while taking care of all the glue for us. In other words, monads are programmable semicolons. +# Archive Nodes Getting Started -For example consider the following imperative example: +Mina nodes are succinct by default, so they don't need to maintain historical information about the network, block, or transactions. -``` - function example(x) { - if ( x == null ) return null; - x = x + 1; - if ( !isEven(x) ) return null; - return x; - } -``` +For some use cases, it is useful to maintain this historical data on an archive node. -This can expressed similarly in functional programming using a monad, using option: +:::tip -``` - type a' option = - | None - | Some of 'a +A zkApp can retrieve events and actions from one or more Mina archive nodes. If your smart contract needs to fetch events and actions from an archive node, see [How to Fetch Events and Actions](/zkapps/writing-a-zkapp/feature-overview/fetch-events-and-actions). - let return x = Some x +::: - (* Bind infix operation, applies f to m *) - let (>>=) m f = - match m with - | Some x -> f x - | None -> None +An archive node is a regular mina daemon that is connected to a running `mina-archive` process using the `--archive-address` flag. - (** - Map infix operation - Essentially the same as bind, but the inner function unwraps the value. - **) - let (>>|) m f = m >>= (fun x -> return (f x)) -``` +The daemon regularly sends blockchain data to the archive process that stores it in a [PostgreSQL](https://www.postgresql.org/) database. -Now we an use these primitives to reimplement the imperative example above as follows. +## Archive Node Requirements -``` - let add_one = ((+) 1) +**Software**: Supported environments include macOS, Linux (Debian 10, 11 and Ubuntu 20.04 LTS), and any host machine with Docker. - let bind_even : int -> int option = - fun x -> if x mod 2 = 0 then Some x else None +**Processor**: Only x86-64 CPU architecture is supported. - let example x = x >>| add_one >>= bind_even; -``` +**Hardware**: Running an archive node does not require any special hardware. -OCaml has a `ppx` that makes writing monads much easier to follow, using the let syntax. +In addition to the [PostgreSQL](https://www.postgresql.org/) database requirements, running an archive node on the Mina network requires at least: -``` - let%bind x = y in - f x +- 8-core processor +- 32 GB of RAM +- 64 GB of free storage - (* This compiles down to the following *) - y >>= (fun x -> f x) -``` +Running an archive node requires some knowledge of managing a PostgreSQL database instance. You must set up a database, run the archive node, connect it to a daemon, and run queries on the data. -Essentially, this syntax takes the value from the let statement and places it on the left of the bind infix call, and puts the assignment into a lambda. +## Install Mina, PostgreSQL, and the archive node package -## Async +1. Install the latest version of Mina. -Under the hood, async uses Monads. However, ivars are the low-level async primitive. `'a Ivar.t` is essentially a mutex that can only be filled once. After the value from a computation returns, it then fills the ivar. The rest of the syntactic sugar takes the ivar and passes them through `Deferred` monads. + You must upgrade to the latest version of the daemon. Follow the steps in [Getting Started](../block-producer-node/getting-started). -A `yield` function exists, but avoid using it since it has some weird behavior. Instead, operate on the wrapped values that happen in between `Deferred` bindings. +1. Download and install [PostgreSQL](https://www.postgresql.org/download/). -## Custom Helpers +1. Install the archive node package. -Use these custom helpers: + - Ubuntu/Debian: -- `Strict_pipe` - wraps pipe and gives certain guarantees around how it can be used. -- `Broadcast_pipe` - allows a single pipe to be hooked up to multiple downstream pipes. + ``` + sudo apt-get install mina-archive=3.3.0-8c0c2e6 + ``` -Do not use: + - Docker: -- `Async.Pipe` operates essentially like a buffer and is unsafe since it has unlimited buffering by default (memory overflow) and some funky behavior around which end of the pipe should do what. -- `Linear_pipe` - deprecated in favor of `Strict_pipe` and `Broadcast_pipe`. + ``` + minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet + ``` ---- -url: /node-developers/contributing ---- +## Set up the archive node -# Contributing to Mina +These steps might be different for your operating system, if you're connecting to a cloud instance of PostgreSQL, if your deployment uses Docker, or if you want to run these processes on different machines. -Mina is an open-source project with a mission to build an inclusive and sustainable community-driven protocol. As such, Mina welcomes contributions. +:::caution -The protocol is in development and is always improving. You can make contributions in many ways, including writing code, user testing, documentation, and community support. For specific instructions for contributing in each of these domains, see the codebase repositories or ask in [Mina Protocol Discord](https://discord.gg/minaprotocol). +Using the `--config` parameter ensures genesis accounts are inserted into the database, which is important to avoid gaps in account balances since the archive node stores only incremental changes. -For general questions on getting involved, reach out to the Mina community on the [Mina Protocol Discord](https://discord.gg/minaprotocol) server. +However, inserting genesis accounts can take significant time and resources. You can skip `--config` if you're connecting to devnet or mainnet and starting from an existing archive database dump rather than an empty archive. -## Developers +*Never* use it on long lived network such as mainnet or devnet -Mina is entirely open source, and the code is distributed under the terms of the [Apache License 2.0](https://github.com/MinaProtocol/mina/blob/master/LICENSE). +::: -## Docs + + -Mina Docs are also open source. We love our community. Help us make the docs better, contributions are welcome and appreciated. +For production, run the archive database in the background, use your operating system service manager (like systemd) to run it for you, or use a postgres service hosted by a cloud provider. -- For a quick fix, click **EDIT THIS PAGE**. -- For a more substantial contribution, see the [Docs Contributing Guidelines](https://github.com/o1-labs/docs2/blob/main/CONTRIBUTING.md). +To run a local archive node in the foreground for testing: -## Mina Grants +1. Start a local postgres server and connect to port 5432: -Grants are rewarded for certain projects related to the development of Mina. See [Mina Grants](https://minaprotocol.com/grants) for details. + ```sh + postgres -p 5432 -D /usr/local/var/postgres + ``` -The projects are mostly programming focused, but the areas of design and community development are included. Reach out in the [#grants](https://discord.com/channels/484437221055922177/727960609832042607) channel on Mina Protocol Discord for questions about the grant program. + For macOS: -## Reporting Issues + ```sh + brew services start postgres + ``` -If you notice Code of Conduct violations, please follow the Reporting Guidelines in [Code of Conduct](https://github.com/MinaProtocol/mina/blob/develop/CODE_OF_CONDUCT.md) to file a report and alert the community to ensure a safe space for everyone. +1. Create a local postgres database called `archive`: -If you encounter critical bugs or vulnerabilities in the protocol, report them to security@minaprotocol.com. For minor bugs and issues, create an issue on GitHub. + ```sh + psql -p 5432 --h localhost -c "create database archive" + ``` ---- -url: /node-developers/graphql-api ---- +1. Load the mina archive schema into the archive database, (create_schema.sql and zkapp_tables.sql): -# GraphQL API + ```sh + psql -h localhost -p 5432 -d archive -f <(curl -Ls https://raw.githubusercontent.com/MinaProtocol/mina/release/3.0.2/src/app/archive/create_schema.sql) + ``` -:::caution +1. Start the archive process on port 3086 and connect to the postgres database that runs on port 5432: -- Mina APIs are still under construction, so these endpoints may change. -- By default, the GraphQL port is bound to localhost. Exposing the GraphQL API to the internet allows anyone to send Mina from the accounts known to the daemon. + ```sh + mina-archive run \ + --postgres-uri postgres://localhost:5432/archive \ + --server-port 3086 + ``` -::: +1. Start the mina daemon and connect it to the archive process that you started on port 3086: -The Mina daemon exposes a [GraphQL API](https://graphql.org/) used to request information from and submit commands to a running node. + ``` + mina daemon \ + ..... + --archive-address 3086 + ``` -To use the GraphQL API, connect your GraphQL client to `http://localhost:3085/graphql` or open in your browser to use the [GraphiQL IDE](https://github.com/graphql/graphiql). + To connect to an archive process on another machine, specify a hostname with `` i.e. `154.97.53.97:3086`. -- By default, an HTTP server runs on port `3085`. You can configure a different port, use the `-rest-port` flag with the daemon startup command. + + -- The default security permits only connections from `localhost`. To listen on all interfaces, add the `-insecure-rest-server` flag to the daemon startup command. +1. Install Docker on your machine. For more information, see [Docker](https://docs.docker.com/get-docker/). -In addition to information about the running node, the GraphQL API can return data about the network's latest blocks. However, as the blockchain's historical state is not persisted in Mina, only blocks in the node's transition frontier are returned, i.e., the last `k` blocks. For other historical data, use the [Archive Node](/node-operators/archive-node/getting-started) that is designed to retain and retrieve historical data. +2. Pull the archive node image from Docker Hub. -The full Mina GraphQL schema is available [https://github.com/MinaProtocol/mina/blob/develop/graphql_schema.json](https://github.com/MinaProtocol/mina/blob/develop/graphql_schema.json). + ```sh + docker pull minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet + ``` -### Queries +3. Pull and install the postgres image from Docker Hub. -The Mina GraphQL API has a number of [queries](/node-operators/validator-node/querying-data) to extract data from a running node. [GraphQL queries](https://graphql.org/learn/queries/) allow specifying the data to be returned in the response. For example, to get the latest block and creator information known to the daemon: + ```sh + docker pull postgres + ``` -``` -query { - bestChain(maxLength: 1) { - creator - stateHash - protocolState { - consensusState { - blockHeight - } - previousStateHash - } - transactions { - coinbase - } - } -} -``` +4. Start the postgres container and expose its networking to other containers. -The following query requests all pending transactions in the transaction pool together with their fees. This query can be used to generate an estimate of a suggested fee for a transaction: + ```sh + docker run --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=postgres -d postgres + ``` -``` -query { - pooledUserCommands { - id, - fee - } -} -``` +5. Create a local postgres database called `archive`. -:::tip + ```sh + docker exec -it postgres createdb -U postgres archive + ``` -The memo field returned for a transaction is [Base58Check encoded](https://en.bitcoin.it/wiki/Base58Check_encoding). +6. Load the mina archive schemas into the archive database, (create_schema.sql and zkapp_tables.sql.) -::: + ```sh + curl -Ls https://raw.githubusercontent.com/MinaProtocol/mina/1551e2faaa246c01636908aabe5f7981715a10f4/src/app/archive/create_schema.sql | docker exec -i postgres psql -U postgres -d archive -``` -query { - account(publicKey: "") { - balance { - total - } - delegate - nonce - } -} -``` + curl -Ls https://raw.githubusercontent.com/MinaProtocol/mina/1551e2faaa246c01636908aabe5f7981715a10f4/src/app/archive/zkapp_tables.sql | docker exec -i postgres psql -U postgres -d archive + ``` -You can submit GraphQL requests from the command line of a node. For example, to use cURL to get the last ten block creators known to the node: +7. Create a local directory to store the archive node data. -``` -curl -d '{"query": "{ - bestChain(maxLength: 10) { - creator - } -}"}' -H 'Content-Type: application/json' http://localhost:3085/graphql -``` + ```sh + mkdir -p /tmp/archive + ``` -### Mutations +8. Start the archive node. -GraphQL mutations modify the running node in some way. For example, mutations may be used to send a payment, create a new account, or to add additional peers. + ```sh + docker run \ + --name archive \ + -p 3086:3086 \ + -v /tmp/archive:/data \ + minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet \ + mina-archive run \ + --postgres-uri postgres://postgres:postgres@postgres:5432/archive \ + --server-port 3086 + ``` -Consult the GraphQL schema for all available mutations. +9. Start the mina daemon and connect it to the archive process that you started on port 3086: -Adding a new peer: + ``` + mina daemon \ + ..... + --archive-address 3086 + ``` -``` -mutation { - addPeers(peers:{ - libp2p_port:10511, - host:"34.73.68.198", - peer_id:"12D3KooWSJB2gZWi3ruVmtTF9JBCEBpCrJfuWCWzzRr8mMQWFQ9U" - }) -} -``` + To connect to an archive process on another machine, specify a hostname with `` i.e. `154.97.53.97:3086`. -Update a SNARK worker to use a fee of 0.1 MINA: + + -``` -mutation { - setSnarkWorkFee(input: {fee: "100000000"}) -} -``` +Docker Compose is a tool for defining and running multi-container Docker applications. +With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration. -### Subscriptions +1. Install Docker and Docker Compose on your machine. For more information, see [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/). -A GraphQL subscription allows a GraphQL client to have data pushed to it when an event occurs. In Mina, there are subscriptions for: +2. Pull the required images: -- _newSyncUpdate_ - occurs when the sync status of the node changes. -- _newBlock_ - occurs when a new block is received. -- chainReorganisation - occurs when the best tip of the node changes in a non-trivial way. + ```sh + docker pull minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet + docker pull postgres + ``` -For example, to subscribe to all new blocks produced: +3. Create a local directory to store the archive node data. -``` -subscription { - newBlock { - creator - stateHash - protocolState { - consensusState { - blockHeight - } - previousStateHash - } - } -} -``` + ```sh + mkdir -p /tmp/archive + ``` -The new block subscription can also be limited to return only new blocks created by a defined public key with the `publicKey` argument. +4. Create a `docker-compose.yml` file with the following contents: -### GraphQL API and public Internet + ```yml + services: + postgres: + image: postgres:17 + environment: + POSTGRES_PASSWORD: postgres + volumes: + - './postgres-data:/var/lib/postgresql/data' + ports: + - '5432:5432' + archive: + image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' + command: >- + mina-archive run --postgres-uri + postgres://postgres:postgres@postgres:5432/archive --server-port 3086 + volumes: + - '/tmp/archive:/data' + ports: + - '3086:3086' + depends_on: + - postgres + ``` -Exposing the GraphQL endpoint with full API support to the public Internet **is not** recommended. -However, you can start the Mina Daemon node with the `--open-limited-graphql-port` and `--limited-graphql-port ` CLI arguments to expose the GraphQL endpoint with limited API support. +5. Start the archive node. -### Resources + ```sh + docker compose up + ``` -- [5-minute introduction video on Mina GraphQL API](http://bit.ly/GraphQLAPIin5) Coda -- [First steps with the Mina GraphQL API](https://garethtdavies.com/crypto/first-steps-with-coda-graphql-api.html) -- [Introduction to GraphQL](https://graphql.org/learn/) +6. Start the mina daemon and connect it to the archive process that you started on port 3086: ---- -url: /node-developers ---- + ``` + mina daemon \ + ..... + --archive-address 3086 + ``` -# Node Developers + To connect to an archive process on another machine, specify a hostname with `` i.e. `154.97.53.97:3086`. -Explore the codebase on [GitHub](https://github.com/MinaProtocol/mina). + + -To start contributing code to Mina, see the [Contributing Guide](/node-developers/contributing). The [protocol](https://github.com/MinaProtocol/mina/tree/master/src) and [CLI](https://github.com/MinaProtocol/mina/tree/master/src/app/cli) are written in OCaml. All levels of experience in any or all of these tools is welcome. -Other documents relevant to contributing code include: +## Using the Archive Node -- [Style Guide](/node-developers/style-guide) -- [Code Review Guidelines](/node-developers/code-review-guidelines) -- [Repository Structure](/node-developers/repository-structure) -- [BIP44 Information](/node-developers/bip44) +Take a look at the tables in the database. -Mina is entirely open source. The code is distributed under the terms of the [Apache License 2.0](https://github.com/MinaProtocol/mina/blob/master/LICENSE). +To list the tables, run the `\dt` command in psql. The output will look like this: ---- -url: /node-developers/repository-structure ---- +``` + List of relations + Schema | Name | Type | Owner +--------+-------------------------------+-------+------- + public | account_identifiers | table | mina + public | accounts_accessed | table | mina + public | accounts_created | table | mina + public | blocks | table | mina + public | blocks_internal_commands | table | mina + public | blocks_user_commands | table | mina + public | blocks_zkapp_commands | table | mina + public | epoch_data | table | mina + public | internal_commands | table | mina + public | protocol_versions | table | mina + public | public_keys | table | mina + public | snarked_ledger_hashes | table | mina + public | timing_info | table | mina + public | token_symbols | table | mina + public | tokens | table | mina + public | user_commands | table | mina + public | voting_for | table | mina + public | zkapp_account_precondition | table | mina + public | zkapp_account_update | table | mina + public | zkapp_account_update_body | table | mina + public | zkapp_account_update_failures | table | mina + public | zkapp_accounts | table | mina + public | zkapp_action_states | table | mina + public | zkapp_amount_bounds | table | mina + public | zkapp_balance_bounds | table | mina + public | zkapp_commands | table | mina + public | zkapp_epoch_data | table | mina + public | zkapp_epoch_ledger | table | mina + public | zkapp_events | table | mina + public | zkapp_fee_payer_body | table | mina + public | zkapp_field | table | mina + public | zkapp_field_array | table | mina + public | zkapp_global_slot_bounds | table | mina + public | zkapp_length_bounds | table | mina + public | zkapp_network_precondition | table | mina + public | zkapp_nonce_bounds | table | mina + public | zkapp_permissions | table | mina + public | zkapp_states | table | mina + public | zkapp_states_nullable | table | mina + public | zkapp_timing_info | table | mina + public | zkapp_token_id_bounds | table | mina + public | zkapp_updates | table | mina + public | zkapp_uris | table | mina + public | zkapp_verification_key_hashes | table | mina + public | zkapp_verification_keys | table | mina +(45 rows) -# Repository Structure +``` -The file structure of the [Mina repository](https://github.com/minaprotocol/mina)the roles various files play: +Use the `\d table_name` to look at the structure of a table in the database. -- `dockerfiles/` - Contains Docker-related scripts +For example to see the structure of the user_commands table, run the `\d user_commands` command in psql. The output will look like this: -- `docs/` - Documentation for the code and processes for contributing are here. The documentation website with the walkthrough docs lives in `frontend/website/docs`. +``` + Table "public.user_commands" + Column | Type | Collation | Nullable | Default +--------------+-------------------+-----------+----------+------------------------------------------- + id | integer | | not null | nextval('user_commands_id_seq'::regclass) + command_type | user_command_type | | not null | + fee_payer_id | integer | | not null | + source_id | integer | | not null | + receiver_id | integer | | not null | + nonce | bigint | | not null | + amount | text | | | + fee | text | | not null | + valid_until | bigint | | | + memo | text | | not null | + hash | text | | not null | +Indexes: + "user_commands_pkey" PRIMARY KEY, btree (id) + "user_commands_hash_key" UNIQUE CONSTRAINT, btree (hash) +Foreign-key constraints: + "user_commands_fee_payer_id_fkey" FOREIGN KEY (fee_payer_id) REFERENCES public_keys(id) + "user_commands_receiver_id_fkey" FOREIGN KEY (receiver_id) REFERENCES public_keys(id) + "user_commands_source_id_fkey" FOREIGN KEY (source_id) REFERENCES public_keys(id) +Referenced by: + TABLE "blocks_user_commands" CONSTRAINT "blocks_user_commands_user_command_id_fkey" FOREIGN KEY (user_command_id) REFERENCES user_commands(id) ON DELETE CASCADE -- `frontend/` - All code related to Mina frontend UIs and products +``` - - `wallet/` - Source code for the Mina wallet +Review the full schema at [/archive/create_schema.sql](https://github.com/minaProtocol/mina/blob/master/src/app/archive/create_schema.sql) and [/archive/zkapp_tables.sql](https://github.com/MinaProtocol/mina/blob/berkeley/src/app/archive/zkapp_tables.sql) - - `website/` - Code for https://minaprotocol.com +## Query the database - - `posts/` - Markdown docs for blog posts +Now that you know the structure of the data, try some queries. - - `src/` - Source code for the website +**Example 1:** Find all blocks that were created by your public key: - - `static/` - Static files like images, etc. +``` +SELECT * +FROM blocks AS b +INNER JOIN public_keys AS pk1 ON b.creator_id = pk1.id +WHERE value = 'MY_PK' +``` -- `rfcs/` - This directory contains all accepted RFCs (or "requests for comments") made according to the [RFC process](https://github.com/MinaProtocol/mina/blob/master/CONTRIBUTING.md#rfcs). +**Example 2:** Find all payments received by your public key: -- `scripts/` +``` +SELECT * +FROM user_commands AS uc +JOIN blocks_user_commands AS buc ON uc.id = buc.user_command_id +JOIN public_keys AS pk ON uc.receiver_id = pk.id +WHERE value = 'MY_PK' +AND type = 'payment' +``` -- `src/` - All protocol source code, both application and library code, is in this directory. +**Example 3:** Find the block at height 12 on the canonical chain: - - `*.opam` - These files are needed for our `dune` build system. There must be one for each - library in `lib`. When you create a library `lib/foo_lib` with a `dune` file giving - the library's name as `foo_lib`, you must create a `foo_lib.opam` file. +``` +WITH RECURSIVE chain AS ( + (SELECT ... FROM blocks b WHERE height = (select MAX(height) from blocks) + ORDER BY timestamp ASC + LIMIT 1) - - `config/` - Build time config - these .mlh files define compile time constants and their values. + UNION ALL - - `app/` - Applications live here. + SELECT ... FROM blocks b + INNER JOIN chain + ON b.id = chain.parent_id AND chain.id <> chain.parent_id +) SELECT ..., pk.value as creator FROM chain c + INNER JOIN public_keys pk + ON pk.id = c.creator_id + WHERE c.height = 12 +``` - - `cli/` - This is the mina client/daemon. It is what you use to run a staker, a snarker, or a simple client for sending and receiving transactions. +**Example 3:** List the counts of blocks created by each public key and sort them in descending order" - - `website/` - Soon to be deprecated directory for the website - most of the code has migrated over to `frontend/website/` +``` +SELECT p.value, COUNT(*) FROM blocks +INNER JOIN public_keys AS p ON creator_id = ip.id +GROUP BY p.value +ORDER BY count DESC; +``` - - `reformat/` - This program runs `ocamlformat` on most of the files in the source tree, with a few exceptions. +**Example 4:** List the counts of applied payments created by each public key and sort them in descending order: - - `logproc/` - This utility reads from `stdin` and can filter and pretty print the log messages emitted by the mina daemon. +``` +SELECT p.value, COUNT(*) FROM user_commands +INNER JOIN public_keys AS p ON source_id = p.id +WHERE status = 'applied' +AND type = 'payment' +GROUP BY p.value ORDER BY count DESC; +``` +**Example 5** Get the latest block: - - `libp2p_helper/` - This program uses go-libp2p to implement the peer-to-peer plumbing that Mina daemons need. +``` +SELECT +height as blockheight, +global_slot_since_genesis as globalslotsincegenesis, +global_slot_since_hard_fork as globalslot, +state_hash as statehash, +parent_hash as parenthash, +ledger_hash as ledgerhash, +to_char(to_timestamp(cast ("timestamp" as bigint) / 1000) AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS') || '.' || + LPAD(((cast("timestamp" as bigint) % 1000)::text), 3, '0') || 'Z' as datetime +FROM blocks +WHERE id in (SELECT MAX(id) FROM blocks); - - `external/` - Local copies of external libraries which we've had to make some tweaks to. +``` - - `lib/` - Libraries powering mina. - The libraries here basically fall into two categories. - 1. General purpose data-types and functionality. This includes `snarky`, `fold_lib`, `vrf_lib`, `sgn`, and others. - 2. Application specific functionality, structured as a library. This includes `syncable_ledger`, `staged_ledger`, `transaction_snark`, and others. +**Example 6** Identify blocks with missing parents, between blockheight 500 and blockheight 5000 +``` +SELECT height +FROM blocks +WHERE parent_id is null AND height >= 500 AND height <= 5000 and height > 1; + +``` --- -url: /node-developers/sandbox-node +url: /node-operators/archive-node --- -# Sandbox Node +# About Archive Nodes -The Mina Sandbox Node enables you to test and get familiar with core features of the protocol and build tooling in a stable environment -- it's a single-node private network that uses the same configuration as the live testnet. This sandbox supports multiple accounts, sending transactions between them, and also supports performing SNARK work, delegating, and staking. In fact since it's a single node network, you earn all the block rewards! +Mina nodes are succinct by default, so they don't need to maintain historical information about the network, block, or transactions. An archive node is a Mina node that stores the historical chain data to a persistent data source, PostgreSQL, so it can later be retrieved. For some use cases, it is useful to maintain this historical data on an archive node. -:::info +:::tip -The sandbox does **NOT** connect you to a live network. +A zkApp can retrieve events and actions from one or more Mina archive nodes. If your smart contract needs to fetch events and actions from an archive node, see [How to Fetch Events and Actions](../../zkapps/writing-a-zkapp/feature-overview/fetch-events-and-actions). ::: -## Installation +An archive node is a regular mina daemon that is connected to a running `mina-archive` process. -[Docker](https://www.docker.com) is a tool for portably running applications. The Mina Sandbox is packaged with Docker, and now built-in to our daemon containers. It’s easy to install--we suggest the [Docker Desktop](https://www.docker.com/products/docker-desktop). +The mina daemon regularly sends blockchain data to the `mina-archive` process that then stores it in a [PostgreSQL](https://www.postgresql.org/) database. -After you have Docker installed run the following command to spin up the Mina Sandbox. +Running an archive node requires some knowledge of managing a PostgreSQL database instance. You must set up a database, run the archive node, connect it to a daemon, and run queries on the data. -``` -docker run \ - --publish 3085:3085 \ - -d \ - --name mina \ - -e RUN_DEMO=true \ - -e MINA_PRIVKEY_PASS='' \ - minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet +# Archive Node -``` +This section describes how to set up and run an Archive node within the Mina protocol. -This command starts a daemon inside the docker container and exposes the GraphQL port (3085) to your computer. This port is used for communication with the client. This daemon automatical runs in the background with a block producer and SNARK worker. +- [Getting Started with Archive Nodes](archive-node/getting-started) - A beginner's guide to launching an Archive Node. +- [Ensuring Archive Redundancy](archive-node/archive-redundancy) - Strategies to mitigate the risk of losing archived data. -You can view logs by executing. +--- +url: /node-operators/block-producer-node/docker-compose +--- -``` -docker logs --follow mina -``` +# Docker Compose Block Producer -And stop mina by running. +This example demonstrates how to run a Mina Block Producer node using Docker Compose for the Mainnet network. The Docker Compose setup includes a Mina Block Producer node, and another script to generate a wallet key. -``` -docker stop mina +Copy and paste the provided configuration into a `docker-compose.yml` file. Then run `docker compose up -d` to start the services, and use `docker compose logs -f` to monitor the logs. + +```yaml +services: + generate_wallet_key: + image: 'minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet' + # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet + environment: + MINA_PRIVKEY_PASS: PssW0rD + entrypoint: [] + command: > + bash -c ' + mina advanced generate-keypair --privkey-path /root/.mina-config/keys/wallet-key + chmod -R 0700 /root/.mina-config/keys + chmod -R 0600 /root/.mina-config/keys/wallet-key + ' + volumes: + - './node/mina-config:/root/.mina-config' + mina_block_producer: + image: 'minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet' + # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet + restart: always + environment: + MINA_PRIVKEY_PASS: PssW0rD + entrypoint: [] + command: > + bash -c ' + mina daemon \ + --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ + --block-producer-key /root/.mina-config/keys/wallet-key + ' + # use --peer-list-url https://bootnodes.minaprotocol.com/networks/devnet.txt for Devnet + volumes: + - './node/mina-config:/root/.mina-config' + ports: + - '8302:8302' + depends_on: + generate_wallet_key: + condition: service_completed_successfully ``` -You can use the Mina CLI to interact with the sandbox node. The following command opens a shell inside the docker container from where you can issue any of the available [cli commands](/node-operators/reference/mina-cli-reference). +--- +url: /node-operators/block-producer-node/getting-started +--- -``` -docker exec -it mina bash -``` +# Block Producer Getting Started -### Account details +:::note -The container has one account with this public key: +Before following this guide, complete the Validator Node setup — from [Requirements](/node-operators/validator-node/requirements) through [Connect to Mainnet or Devnet](/node-operators/validator-node/connecting-to-the-network). You should have a synced node and a [key pair](/node-operators/validator-node/generating-a-keypair) before proceeding. -``` -B62qiZfzW27eavtPrnF6DeDSAKEjXuGFdkouC3T5STRa6rrYLiDUP2p -``` +::: -The password for this account is the empty string (there's no password -- you can leave the password field blank). +## Start the daemon -## How to use the sandbox +Run the daemon with the `--block-producer-key` flag pointing to your wallet key: -There are a few things you can do with your sandbox now that you have it running: +```sh +mina daemon --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ + --block-producer-key ~/keys/my-wallet +``` -- [Install Mina](/node-operators/validator-node/installing-on-ubuntu-and-debian) as usual and use many of the client commands. Since the daemon is already running in the container, you don't need to run `mina daemon`! +To send block rewards to a different account (e.g. a cold wallet), add the `--coinbase-receiver` flag: -- Install the GUI Wallet app to use a graphical interface to your node. Enter '127.0.0.1' as the host of your node during setup. +```sh +mina daemon --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ + --block-producer-key ~/keys/my-wallet \ + --coinbase-receiver $RECEIVER_PUBLICKEY +``` -- Head over to [http://localhost:3085/graphql](http://localhost:3085/graphql) to play with the GraphQL API directly. +## Verify block production + +Run `mina client status` and check the `Block producers running` field: + +```text +Block producers running: 1 (B62q...) +Coinbase receiver: Block producer +Next block will be produced in: in 7.077h for slot: ... +``` --- -url: /node-developers/style-guide +url: /node-operators/block-producer-node/hot-cold-block-production --- -# Style Guide - -## Ocaml +# Hot and Cold Block Production -[ocaml]: #ocaml +Block production requires a node connected to the internet, which means the block producer's private key is exposed on an online machine. To mitigate this risk, Mina supports a hot/cold wallet pattern: -### General +- A **[hot wallet](/glossary#hot-wallet)** has its private key on an internet-connected machine. It is used to run the block producer node but should hold minimal funds. +- A **[cold wallet](/glossary#cold-wallet)** has its private key stored offline (e.g. generated on an air-gapped laptop or a hardware wallet like [Ledger](https://shop.ledger.com/)). It holds the majority of your stake. -[ocaml-general]: #ocaml-general +By delegating from your cold wallet to your hot wallet, you can produce blocks — and earn all associated rewards — while keeping the bulk of your funds in cold storage. -Our style guidelines are an extension of a couple of existing style guidelines. The first is ocamlformat, and it acts as the source of truth for most of our coding style. In fact, ocamlformat is a blocker on CI, so your code must be formatted by it's guidelines in order to be merged into master. Ocamlformat does not handle all important cases of style, however, as it is only defining and enforcing how code should be spaced out and indented. For anything which ocamlformat does not cover, the [Jane Street styleguide](https://opensource.janestreet.com/standards/) should be referenced. This styleguide we define here is intended to be an extension of the janestreet styleguide, with more attention to detail in concern to a few specific constructs we use regularly throughout our codebase. +## Setup -### Mli Files +You need at least two accounts: one cold wallet and one hot wallet. See [Generating a Key Pair](/node-operators/validator-node/generating-a-keypair) for key generation instructions. -[ocaml-mli]: #ocaml-mli +### 1. Create a hot wallet -A `*.mli` file should not be included for a `*.ml` file if the `*.ml` file's automatically derived interface is different. Many `*.ml` files in our codebase consist of only signatures and a functor. In the case of those files, there is not purpose to redefining the `*.mli` file because there is no new or restricted information in that file. If a `*.ml` file contains implementations in the root structure, then a `*.mli` file should most likely be created. +Generate a key pair on your block producer machine. Do **not** use a Ledger or other hardware security module (HSM) for this key — the private key must be accessible to the mina daemon. -### Modules +Take note of your hot wallet public key. -[ocaml-modules]: #ocaml-modules +### 2. Fund the hot wallet -#### Prefer Standardized Shortnames +Your hot wallet must be present in the consensus ledger before it can be used for staking. -The names `t`, `T`, and `S` are common shortnames used in modules to signify specific things. +- If your hot wallet address is present in the genesis ledger, no further action is needed. +- Otherwise, send at least enough MINA to cover the account creation fee to your hot wallet address. -The name `t` is used to represent the root type of a module. For instance, if there is a module `Account` which contains types and values related to accounts, then `Account.t` is the type of an account. The name `t` can also be used as a value iff there is only intended to be one value of the root type of the module. As an example, if you wanted to have a single global logger in a `Logger` module, the type `Logger.t` could be the type of a logger, and the value `Logger.t` could be the global logger value of type `Logger.t`. +### 3. Create a cold wallet -The module name `T` is used to encapsulate the root type and basic definitions regarding a root type of a module. It is a common practice used when you want to instantiate some functors for a module's root type and have the instantiations appear in the module itself. As an example, it is common to call the `Comparable.Make` functor in order to derive various helper values/modules from a comparable type. In this case, if we had a module `Account` again, and we wanted to derive the `Comparable.S` signature, then we would define a module `T` in `Account` which defines a root type `t` and the required functions for the `Comparable.Make` functor argument (in this case, `compare`). With this `T` module, we can then `include T` and `include Comparable.Make (T)` in the `Account` module to bring in all related values/modules for the `Account.t` type. Here is a full example of that: +Generate a key pair using the most secure method available — preferably on a machine disconnected from the internet, or on a hardware wallet. -``` -module Account = struct - module T = struct - type t = ... [@@deriving compare] - end +### 4. Fund the cold wallet - include T - include Comparable.Make (T) -end -``` +Your cold wallet must be present in the consensus ledger before its stake can be counted when delegated to your hot wallet. -The module type name `S` is used for defining the root signature of a module. This is most commonly used when you have a module which contains a functor. In this case, we typically call the functor `Make` and declare the functor returns the type `S`, putting both of these values in the same module. Looking back at our previous example, `Core_kernel`'s `Comparable` module follows this pattern: `Comparable.Make` is a functor which returns a `Comparable.S`. +- If your cold wallet address is present in the genesis ledger, no further action is needed. +- Otherwise, send enough MINA to meaningfully participate in consensus to your cold wallet address. -#### Prefer One Type Per Module +### 5. Delegate from cold to hot -[ocaml-modules-singleton-types]: #ocaml-modules-singleton-types +Delegate your cold wallet's stake to your hot wallet: -As a general rule of thumb, each module should be scoped to a single type. This pattern helps isolate concerns and, in turn, allows value names to be shorter, as they are located by context. Take, for example, a `Merkle_tree` module. This module will need a type `Merkle_tree.t` which represents the entire merkle tree (or a node of it). A `Merkle_tree` will also want to have a `path` type. It is preferable to place this `path` type into it's own nested module (`Merkle_tree.Path.t` instead of `Merkle_tree.path`). To help understand why this is preferable, imagine we did put path in `Merkle_tree.path`. Now, `Merkle_tree` contains values (functions) that relate not only to the merkle tree type itself, but also the a path of a merkle tree. For clarity, it would be natural to prepend all of the value names related to a path with `path_` (`path_map`, `path_length`, etc...). By isolating `Path` to it's own module, we can shorten these names while keep the context of values clear. Additionally, if we choose to in the future, we may encapsulate the implementation details of `Path` by applying a restrictive signature to it, which would make the separation of concerns more clear via compiler enforcement. +```sh +mina client delegate-stake \ + --sender $COLD_PUBLIC_KEY \ + --receiver $HOT_PUBLIC_KEY +``` -#### No Monkeypatching +If your cold wallet is on a Ledger, follow the delegate instructions in the [Mina Ledger app README](https://github.com/jspada/ledger-app-mina/tree/v1.0.0-beta.2). -[ocaml-modules-monkeypatching]: #ocaml-modules-monkeypatching +### 6. Start producing blocks -Monkeypatching of modules is explicitly disallowed in our codebase. Monkeypatching is defined as the act of taking an existing module and redefining it with extended or modified values. More simply, it's anything of the form. +Follow the [Getting Started](/node-operators/block-producer-node/getting-started) guide using your hot wallet key as the `--block-producer-key`. -``` -module A = struct - module M = struct - let x = ... - end -end +## Why can't I use an HSM directly? -module M = struct - include A.M - let y = ... - (* or `let x = ...` *) -end -``` +You may wonder why Mina can't let you produce blocks directly from a secure enclave or HSM. Two components of block production require the private key in ways that make this impractical: -Monkeypatching may be the easiest path to getting code to compile sometimes, but in general, it creates confusion and/or technical debt in the codebase. If you need to monkeypatch a module, you should have a good reason as to why. +### Finding eligible slots -#### Functor Signature Equalities +A block producer determines slot eligibility by evaluating a VRF (verifiable random function) with their private key. The VRF must be evaluated for your account and every account that delegates to you, for all slots within an epoch. -[ocaml-modules-functor-patterns]: #ocaml-modules-functor-patterns +### Creating the blockchain SNARK -Signature `with` statements for signatures of modules generated by functors should be limited to the form `S with module M1 = M2` whenever possible. Replacement equalities `:=` should be limited to `include` statements where portions of the signature need to be limited (for example, when a nested module in the signature is already defined at the current structure scope). The form `S with type t = ...` is also not preferred as it scales poorly as the number of common dependencies between signatures involved with a functor increases. Note that this places increased importance on the janestreet styleguide rule "Prefer standard signature includes to hand-written interfaces". +When a block producer wins a slot, they must create a SNARK proof that the new block is a valid extension of the existing chain. This proof embeds VRF information using the private key — replacing the simple signature that other protocols use. Creating this proof is computationally expensive and relies on advanced cryptography that is extremely difficult, and likely impossible, to perform quickly enough inside today's secure hardware. -#### Functor Arity +--- +url: /node-operators/block-producer-node +--- -[ocaml-modules-functor-arity]: #ocaml-modules-functor-arity +# About Block Producers -Functor can have a maximum arity of 3 (arity is the number of arguments; in this case, the number of nested functors - functors returning functors). If a functor requires more than 3 modules as arguments, then the required modules should all be nested into one module. The standard pattern for this is to define a signature `Inputs_intf` for your functor, which will, in turn, define the module arguments to the functor. See below for a simple example. +The role of a block producer in Mina is to achieve [consensus](https://minaprotocol.com/blog/what-is-ouroboros-samasika) and provide security to the blockchain. The block producer is responsible for creating new blocks that include recent transactions broadcast on the network and a blockchain proof that proves the current state of the chain is valid. -``` -module type Inputs_intf = sig - module A : A.S - module B : B.S - module C : C.S - module D : D.S -end +In Mina, anyone can become a block producer. There is an unbounded number of participants with the chance of producing a block proportional to the funds staked. Funds are not locked and are not subject to slashing. -module type S = sig - include Inputs_intf +In return for staking funds and generating the required blockchain proofs, blocks that are produced and included in the canonical chain are rewarded in the form of a coinbase and transaction fees, less any fees paid to purchase required [transaction SNARK work](/node-operators/snark-workers). - (* ... *) -end +To successfully produce a block, a block producer must have the current state of the blockchain. A block producer must have enough available compute to produce a blockchain SNARK within the slot time and be connected to peers to broadcast the generated block within an acceptable delay as defined by the network consensus parameters. -module Make (Inputs : Inputs_intf) - : S - with module A = Inputs.A - and module B = Inputs.B - and module C = Inputs.C - and module D = Inputs.D = -struct - open Inputs +## Block Producers - (* ... *) -end -``` +This section describes how to run a Block Producer on the Mina protocol. -# Code Idiosyncrasies +- [Getting Started](block-producer-node/getting-started) - How to install and get started running a block producer. +- [Hot and Cold Block Production ](block-producer-node/hot-cold-block-production) - How to get started running a block producer. -We use a particular style of OCaml. Here's some of the important things. +--- +url: /node-operators/block-producer-node/staking-service-guidelines +--- -## Parameterized records +# Staking Service Guidelines -```ocaml -type ('payload, 'pk, 'signature) t_ = - {payload: 'payload; sender: 'pk; signature: 'signature} -[@@deriving eq, sexp, hash] +The important parts of running a staking service are predicting/determining winning slots in which you can produce blocks and paying out participants. The Mina protocol does not automatically payout rewards to delegates, so part of running a staking service is manually paying out participants by [sending many transactions](#sending-many-transactions). -type t = (Payload.t, Public_key.t, Signature.t) t_ -[@@deriving eq, sexp, hash] +This document aims to explain the different components that you should think about when managing those payouts. Specifically, this document provides an understanding of odds of winning blocks, gathering data from the ledger for later use, and computing relevant staking payout information from this data. -(* ... *) +## Staking Rewards -type var = (Payload.var, Public_key.var, Signature.var) t_ -``` +The coinbase reward for producing a block is 720 tokens. -We're defining a base type `t_` with type variables for all types of record fields. Then we define the record using these type variables. Finally, we instantiate the record with `type t`, this is the OCaml type. And also `type var` this is the type of this value in a SNARK circuit. We'll cover this more later. Whenever we want something to be programmable from within a SNARK circuit we define it in this manner so we can reuse the record definition across both types. +## Dumping Staking Ledgers -There is some talk of moving to OCaml object types to do this sort of thing so we don't need to deal with positional arguments. Perhaps I (@bkase) will write up an RFC for that at some point. +In order to compute odds of winning a block for a given epoch, or to retroactively compute the coinbase reward a given account would receive, you need to have the staking ledger from that epoch. Mina daemons only keep around the staking ledger for the current epoch and the staking ledger for the next epoch, so if you want to capture a staking ledger for an epoch, you need to do it before or during that epoch. -### Ppx_deriving +The `mina ledger export` command can be used to export ledgers from a running daemon: -```ocaml -type t = int [@@deriving sexp, eq] ``` +Print the specified ledger (default: staged ledger at the best tip). Note: Exporting snarked ledger is an expensive operation and can take a few seconds -This is the first time we've seen a macro. Here we use `sexp` from [ppx_jane](https://github.com/janestreet/ppx_jane) and `eq` from [ppx_deriving](https://github.com/ocaml-ppx/ppx_deriving). + mina ledger export STAGED-LEDGER|SNARKED-LEDGER|STAKING-EPOCH-LEDGER|NEXT-EPOCH-LEDGER -### Stable.V1 +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--plaintext] Use plaintext input or output + (default: JSON) + (alias: -plaintext) + [--state-hash STATE-HASH] State hash, if printing a staged + ledger (default: state hash for the + best tip) + (alias: -state-hash) + [-help] print this help text and exit + (alias: -?) -```ocaml -module Stable : sig - module V1 : sig - type t = (* ... *) - [@@deriving bin_io, (*...*)] - end -end ``` -Whenever a type is serializable, it's important for us to maintain backwards compatibility once we have a stable release. Ideally, we wouldn't define `bin_io` on any types outside of `Stable.V1`. When we change the structure of the datatype we would create a `V2` under `Stable`. -### Property based tests +It requires an argument to identifier of the ledger you wish to export. The table below describes what each of these identifiers represent. -[Core](https://opensource.janestreet.com/core/) has an implementation of [QuickCheck](https://blog.janestreet.com/quickcheck-for-core/) that we use whenever we can in unit tests. Here is an example signature for a `Quickcheck.Generator.t` of payments. +
-```ocaml -(* Generate a single payment between - * $a, b \in keys$ - * for fee $\in [0,max_fee]$ - * and an amount $\in [1,max_amount]$ - *) +| Identifier | Description | +|----------------------|-----------------------------------------------------------------| +| staking-epoch-ledger | The staking ledger for the current epoch. | +| next-epoch-ledger | The staking ledger for the next epoch (epoch after current). | +| staged-ledger | The most recent staged ledger (from the best tip of that node). | +| snarked-ledger | The most recent snarked ledger (from the best tip of that node).| -val gen : - keys:Signature_keypair.t array - -> max_amount:int - -> max_fee:int - -> t Quickcheck.Generator.t +
+ + +In order to ensure you always have each staking ledger available for use after epochs have expired, we recommend exporting the staking-epoch-ledger every (7140 × 3) ÷ 60 = 357 hours (there are 7140 slots in an epoch, and each slot is 3 minutes long). + +By default, ledgers are exported as json data. See `mina ledger export -help` for documentation of flags which will enable other formats. When output as json, the ledger will be represented as an array of account objects. Below is an example of what an account object in json looks like. + +```json +{ + "pk": "B62qrwZRsNkU39TrGpFwDdpRS2JaCB2yFZKMFNqLFYjcqGE5G5fWA8p", + "balance": "17000", + "delegate": "B62qrwZRsNkU39TrGpFwDdpRS2JaCB2yFZKMFNqLFYjcqGE5G5fWA8p", + "token": "1", + "token_permissions": {}, + "receipt_chain_hash": "2mzbV7WevxLuchs2dAMY4vQBS6XttnCUF8Hvks4XNBQ5qiSGGBQe", + "voting_for": "3NK2tkzqqK5spR2sZ7tujjqPksL45M3UUrcA4WhCkeiPtnugyE2x", + "permissions": { + "stake": true, + "edit_state": "signature", + "send": "signature", + "set_delegate": "signature", + "set_permissions": "signature", + "set_verification_key": "signature" + } +} ``` - +For a running staking service, you are interested in the accounts which you control and the accounts which are staking to accounts you control. As an example, if we only had one account in our staking service, we could grab all the accounts we are interested in for a ledger using a command like: -### Typesafe invariants (help with naming this section) +```sh +mina ledger export staking-epoch-ledger | jq "$(cat < With_valid_signature.t option -``` +Blocks in Mina are produced within distinct time intervals called "slots". Each account in the ledger has a chance of winning each slot on the network, which allows them to produce a block for that slot (or, in the case of delegation, allows the delegated account to produce a block on their behalf). An account computes a random number (via a [Verifiable Random Function, or VRF](/glossary#verifiable-random-function-vrf)) for each slot, and compares that random number against a required threshold to determine if they have "won" that slot and can produce a block at that time. Their chance of winning a slot is thus determined by the threshold, which is a function of their relative stake in the network. -Here we define `With_valid_signature` (usage will be `User_command.With_valid_signature.t`) using `type nonrec t = private t` to allow upcasting to a `User_command.t`, but prevent downcasting. The _only_ way to turn a `User_command.t` into a `User_command.With_valid_signature.t` is to `check` it. Now the compiler will catch our mistakes. +The stake distribution that is sampled when determining the VRF threshold are contained in a special ledger called the "staking ledger". Staking ledgers are fixed ledgers from past epochs in the network (epochs are fixed spans of slots; every epoch is 7140 slots long). Using past fixed ledgers prevents malicious stakers from increasing their chances of winning during an epoch by altering the distribution of stake on the main ledger (the "staged ledger"). Every epoch, the staking ledger changes. Due to this constraint, any account on the system can only check for "winning slots" within the next 2 epochs at any given time. -### Unit Tests +Once you know the staking ledger that will be used for a given epoch, the required VRF threshold for producing blocks in that epoch can be computed for each account. We can compute a stake ratio for a given account by dividing the stake that account controls in the staking ledger by the total amount of currency in the staking ledger. Figure 1 on page 20 of the [Ouroboros Genesis](https://eprint.iacr.org/2018/378.pdf) paper provides the raw function for computing VRF thresholds. Filling in Mina's value for the active slots coefficient f gives us the following function: φ(α) = 1 - (1/4)^α. This function takes as a parameter α, which is the stake ratio we compute for an account. -We use [ppx_inline_test](https://github.com/janestreet/ppx_inline_test) for unit testing. Of course whenever we can, we combine that with `QuickCheck`. +Since each VRF is compared against this threshold (which is between 0 and 1), and each VRF is a pseudo-random number between 0 and 1, the VRF threshold for a given account is essentially the probability that a slot will be won by that account. Thus, we can extrapolate this function for computing VRF thresholds to compute the mean number of blocks we can expect a given account to win within an epoch. This can be done merely by summing all the probabilities for winning each slot of an epoch, and since the probability for an entire epoch is fixed, the mathematical expression to compute this simplifies to φ(α) × 7140. -```ocaml -let%test_unit = - Quickcheck.test ~sexp:[%sexp_of: Int.t] Int.quickcheck_generator - ~f:(fun x -> assert (Int.equal (f_inv (f x)) x)) -``` +As an example, let's say there is an account on the network which has 10^6 (1 million) mina tokens in the staking ledger epoch ep. This same staking ledger for epoch ep has a total supply of 8×10^8 (800 million) mina tokens. We can compute the odds that this account will win an individual slot in ep using φ(10^6 ÷ 8×10^8) = 0.0017313674. This gives us a ~0.17% chance that this account will win each slot during epoch ep. We can then compute the mean number of blocks we expect this account to produce for this epoch by multiplying the result by 7140, giving us a probabilistic mean of ~12.36 blocks for the epoch. -### Functors +Another way to calculate the probability is 1 - 0.25^(10^6 ÷ 8×10^8) which gives the same result as above. -We are in the process of migrating to using module signature equalities -- see [the above section](#functor-signature-equalities) and [the rfc for rationale](https://github.com/MinaProtocol/mina/blob/master/rfcs/0004-style-guidelines.md), but we still have a lot of code using type substitutions (`with type foo := bar`). +## Sending Many Transactions -First we define the resulting module type of the functor, keeping all types we'll be functoring in abstract. +It is important to pay out rewards to participants who have delegated their stake to your staking service. -```ocaml -module type S = sig - type boolean_var - type curve - type curve_var - (*...*) -end -``` +To learn how, see [Sending Many Transactions](/mina-protocol/sending-a-payment#sending-many-transactions). -Then we define the functor: +--- +url: /node-operators/data-and-history +--- -```ocaml -module Schnorr - (Impl : Snark_intf.S) - (Curve : sig (*...*) end) - (Message : Message_intf - with type boolean_var := Impl.Boolean.var - (*...*)) -: S with type boolean_var := Impl.Boolean.var - and type curve := Curve.t - and type curve_var := Curve.var - (*...*) -= struct - (* here we implement the signature described in S *) -end -``` +# About Data and History -### Custom SNARK circuit logic +Mina is a succinct blockchain, and as a result, consensus nodes store only the recent history of the chain before discarding it (the last k blocks, currently 290). You can access recent history by using graphQL from a running mina daemon. -This is also the first time we see custom SNARK circuit logic. A pattern we've been using is to scope all operations that you'd want to run inside a SNARK under a submodule `module Checked`. +While prior transaction history is not required to prove the current state is valid (this is handled by using a recursive zero-knowledge proof), many applications need to access prior transaction history. Examples include block explorers and wallets. -For example, inside [sgn.mli](https://github.com/MinaProtocol/mina/blob/master/src/lib/sgn/sgn.mli) we see: +You can optionally run an [archive node](/node-operators/archive-node/getting-started) to access historic data. The archive node process stores a summary of each block in a Postgres database. The archive node requires a running mina-archive daemon that connects to a running mina daemon and a [PostgreSQL](https://www.postgresql.org/) database. -```ocaml -(* ... *) -val negate : t -> t +## In This Section -module Checked : sig - val negate : var -> var -end -``` +- [Rosetta API](/node-operators/data-and-history/rosetta) - A standardized API for blockchain integration, exchange support, and historical data queries. -`negate` is the version of the function that runs in OCaml, and `Checked.negate` is the one that runs inside of a SNARK circuit. +For querying data from a running node via GraphQL, see [Querying Data](/node-operators/validator-node/querying-data). --- -url: /node-operators/archive-node/archive-redundancy +url: /node-operators/data-and-history/rosetta --- -# Archive Redundancy +# Rosetta API -The [archive node](/node-operators/archive-node/getting-started) stores its data in a PostgreSQL database that node operators host on a provider of their choice, including self-hosting. For redundancy, archive node data can also be stored to an object storage like [Google Cloud Storage](#upload-block-data-to-google-cloud-storage); soon S3 and others) or to a [`mina.log`](#save-block-data-from-logs) file that reside on your computer or be streamed to any typical logging service, for example, LogDNA. -Archive data is critical for applications that require historical lookup. -On the protocol side, archive data is important for disaster recovery to reconstruct a certain state. A single [archive node](/node-operators/archive-node/getting-started) set up might not be sufficient. + +A standardized API for blockchain integration — query historical data, build transactions, and integrate with exchanges. + -If the daemon that sends blocks to the archive process or if the archive process itself fails for some reason, there can be missing blocks in the database. To minimize the risk of archive data loss, employ redundancy techniques. +[Rosetta API](https://www.rosetta-api.org/) (rebranded as [Mesh](https://docs.cdp.coinbase.com/mesh/docs/welcome/) by Coinbase) is an open-source specification and set of tools that make deploying and interacting with blockchains quick and easy. Mina implements a subset of the Rosetta specification — not all endpoints defined in the spec are available. Mina's Rosetta implementation is primarily used by exchanges to integrate MINA deposits, withdrawals, and balance queries. -A single archive node setup has a daemon sending blocks to an archive process that writes them to the database. +:::note +The Rosetta API is auxiliary to Mina's existing [GraphQL API](/node-operators/reference/mina-cli-reference) and [Archive Node](/node-operators/archive-node). While GraphQL provides access to current network state, historical and persistence data requires the Archive database. Rosetta bundles both data sources behind a standardized interface and exists primarily to satisfy exchange integration requirements. +::: -To connect multiple daemons to the archive process, specify the address of an archive process in multiple daemons to reduce the dependency on a single daemon to provide blocks to the archive process. +## Architecture -For example, the server port of an archive process is 3086. The daemons can connect to that port using the flag `archive-address` +The Rosetta stack consists of four components that work together: -``` -mina daemon \ - ..... - --archive-address :3086\ -``` +| Component | Default Port | Description | +|---|---|---| +| **Mina Daemon** | 8302 (P2P), 3085 (GraphQL) | Syncs with the network, produces/validates blocks | +| **Archive Node** | 3086 | Stores historical block data in PostgreSQL | +| **PostgreSQL** | 5432 | Database backend for the archive node | +| **Rosetta API** | 3087 (online), 3088 (offline) | Translates Mina data into the Rosetta specification | -Similarly, it is possible to have multiple archive processes write to the same database. In this case, the `postgres-uri` passed to the archive process is the same across multiple archive processes. +All ports listed above are defaults and can be overridden via configuration. When using Docker, these are the ports inside the container — map them to your desired host ports with `-p`. -However, multiple archive processes concurrently writing to a database could cause data inconsistencies (explained in https://github.com/MinaProtocol/mina/issues/7567). To avoid this, set the transaction isolation level of the archive database to `Serializable` with the following query: +## Installation - ALTER DATABASE `` SET DEFAULT_TRANSACTION_ISOLATION TO SERIALIZABLE ; +There are three ways to run Rosetta, depending on your needs. -Set the transaction level after you create the [database](/node-operators/archive-node/getting-started) and before you connect an archive process to it. +### Option 1: All-in-One Docker Image (Recommended for getting started) -## Back up block data +The all-in-one image bundles the daemon, archive node, PostgreSQL, and Rosetta API into a single container. It automatically initializes the archive database from public o1Labs backups. -To ensure that archive data can be restored, use the following features to back up and restore block data. +**Requirements:** Docker with at least 12 GB RAM allocated (16 GB recommended). -A mechanism for logging a high-fidelity machine-readable representation of blocks using JSON includes some opaque information deep within. +#### Mainnet -These logs are used internally to quickly replay blocks to get to certain chain states for debugging. This information suffices to recreate exact states of the network. +```bash +docker run -it --rm --name rosetta \ + --entrypoint=./docker-start.sh \ + -p 8302:8302 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ + minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet +``` -Some of the internal data look like this: +#### Devnet -```json -{"data":["Signed_command",{"payload":{"common":{"fee":"100","fee_token":"1","fee_payer_pk":"B62qixbmBBmCmv1xH5SeF1hw6EqwSNVPi9B28epa3phqVMSyuZk9EoH","nonce":"340","valid_until":"4294967295","memo":"E4YM2vTHhWEg66xpj52JErHUBU4pZ1yageL4TVDDpTTSsv8mK6YaH"},"body":["Payment",{"source_pk":"B62qixbmBBmCmv1xH5SeF1hw6EqwSNVPi9B28epa3phqVMSyuZk9EoH","receiver_pk":"B62qm2GCuGCEK79mEjeyaeiFoukThuZLJCHGe9HAzuAnfbtS5FHtPnP","token_id":"1","amount":"100000000"}]},"signer":"B62qixbmBBmCmv1xH5SeF1hw6EqwSNVPi9B28epa3phqVMSyuZk9EoH","signature":"7mXGz8Df1gu92HVWGue24wcrGxDWkgQrDK59xQGXc627PKFQvVAPSzZn7JMkHtfdBUXavDHcgLBZy4iR4UmA5seRCPMkFDci"}],"status":["Applied",{"fee_payer_account_creation_fee_paid":null,"receiver_account_creation_fee_paid":null,"created_token":null},{"fee_payer_balance":"31866000100000","source_balance":"31866000100000","receiver_balance":"34099000000"}]}],"coinbase":["One",null],"internal_command_balances":[["Coinbase",{"coinbase_receiver_balance":"75477804514901","fee_transfer_receiver_balance":null}],["Fee_transfer",{"receiver1_balance":"65266376010003","receiver2_balance":"78601129170700"}],["Fee_transfer",{"receiver1_balance":"66001820000000","receiver2_balance":"76870784414900"}],["Fee_transfer",{"receiver1_balance":"71158365898775","receiver2_balance":"59264207944722"}],["Fee_transfer",{"receiver1_balance":"68546088449962","receiver2_balance":"66721919100000"}],["Fee_transfer",{"receiver1_balance":"67700798001000","receiver2_balance":"66372760000000"}],["Fee_transfer",{"receiver1_balance":"85383891400000","receiver2_balance":"107174952265469"}],["Fee_transfer",{"receiver1_balance":"65879310000000","receiver2_balance":"66282230000000"}]]}]},"delta_transition_chain_proof":["jxLZWooV57gKCmanzCHHt1CDbHfUpMu6MkynUdqN9ZkBUJi7B1W",[]]} +```bash +docker run -it --rm --name rosetta \ + --entrypoint=./docker-start.sh \ + -p 8302:8302 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ + -e MINA_NETWORK=devnet \ + -e PEER_LIST_URL=https://bootnodes.minaprotocol.com/networks/devnet.txt \ + minaprotocol/mina-rosetta:3.2.0-97ad487-bookworm-devnet ``` -This JSON evolves as the format of the block and transaction payloads evolve in the network. +Initial sync typically takes between 20 minutes and 1 hour depending on your hardware and network connection. You can check sync status with: -### Upload block data to Google Cloud Storage +```bash +docker exec rosetta mina client status +``` -The daemon generates a file for each block with the name `-.json` . These files are called precomputed blocks and have all the fields of a block. +#### Environment Variables -To specify a daemon to upload block data to Google Cloud Storage, pass the flag `--upload-blocks-to-gcloud`. +The all-in-one image supports the following environment variables for customization: -Set the following environment variables: +| Variable | Default | Description | +|---|---|---| +| `MINA_NETWORK` | `mainnet` | Network to connect to (`mainnet` or `devnet`) | +| `PEER_LIST_URL` | Network-specific seed list | URL for the peer list | +| `LOG_LEVEL` | `Debug` | Log level (`Info`, `Debug`, `Warn`, `Error`) | +| `MINA_GRAPHQL_PORT` | `3085` | GraphQL API port | +| `MINA_ARCHIVE_PORT` | `3086` | Archive node port | +| `MINA_ROSETTA_ONLINE_PORT` | `3087` | Rosetta online API port | +| `MINA_ROSETTA_OFFLINE_PORT` | `3088` | Rosetta offline API port | +| `POSTGRES_USERNAME` | `pguser` | PostgreSQL username | +| `POSTGRES_DBNAME` | `archive` | PostgreSQL database name | +| `POSTGRES_DATA_DIR` | `/data/postgresql` | PostgreSQL data directory | +| `MINA_ARCHIVE_DUMP_URL` | `https://storage.googleapis.com/mina-archive-dumps` | Base URL for archive database dumps | +| `MINA_CONFIG_DIR` | `/data/.mina-config` | Mina configuration directory | -- `GCLOUD_KEYFILE`: Key file for authentication +### Option 2: Docker Compose (Recommended for production) -- `NETWORK_NAME`: Network name to use in the filename to easily distinguish between blocks in different networks (Mainnet and Testnets) +For production deployments, use Docker Compose to run each component as a separate container. This gives you more control over resource allocation, logging, and restarts. -- `GCLOUD_BLOCK_UPLOAD_BUCKET`: Google Cloud Storage bucket where the files are uploaded +For production deployments, a complete Docker Compose configuration with all four services (PostgreSQL, archive node, Mina daemon, and Rosetta API) is available in the [Rosetta Docker Compose guide](/node-operators/rosetta/docker-compose). -### Save block data from logs +### Option 3: Debian Packages (Manual setup) -The daemon logs the block data if the flag `-log-precomputed-blocks` is passed. +You can install each component individually via Debian packages. This is useful if you already run a Mina daemon and archive node and only need to add the Rosetta API. -The log to look for is `Saw block with state hash $state_hash` that contains `precomputed_block` in the metadata and has the block information. These precomputed blocks contain the same information that gets uploaded to Google Cloud Storage. +```bash +# Add the Mina repository (if not already configured) +echo "deb [trusted=yes] http://packages.o1test.net/ noble stable" | sudo tee /etc/apt/sources.list.d/mina.list +sudo apt-get update -### Generate block data from another archive database +# Install the Rosetta package for your network +sudo apt-get install mina-rosetta-mainnet +# or: sudo apt-get install mina-rosetta-devnet +``` -From a fully synced archive database, you can generate block data for each block using the `mina-extract-blocks` tool. +Then run the Rosetta API pointing to your existing daemon and archive database: -The `mina-extract-blocks` tool generates a file for each block with name `.json`. +```bash +mina-rosetta \ + --archive-uri postgres://:@:/ \ + --graphql-uri http://localhost:3085/graphql \ + --log-json \ + --port 3087 +``` -The tool takes an `--archive-uri`, an `--end-state-hash`, and an optional `--start-state-hash`, and writes all the blocks in the chain starting from start-state-hash and ending at end-state-hash (including start and end). +This requires a running [Mina daemon](/node-operators/block-producer-node) and [archive node](/node-operators/archive-node) with a populated PostgreSQL database. -If only the end hash is provided, then the tool generates blocks starting with the unparented block closest to the end block. This would be the genesis block if there are no missing blocks in between. The block data in these files are called extensional blocks. Since these blocks are generated from the database, they have only the data stored in the archive database and do not contain any other information pertaining to a block (for example, blockchain SNARK) like the precomputed blocks and can only be used to restore blocks in the archive database. +## Offline Mode -Provide the flag `--all-blocks` to write out all blocks contained in the database. +The Rosetta Construction API requires an "offline" endpoint that can build and sign transactions without network access. Use the standalone entrypoint for this: -## Identify missing blocks +```bash +docker run -it --rm --name rosetta-offline \ + --entrypoint=./docker-standalone-start.sh \ + -p 3088:3088 \ + minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet +``` -To determine any missing blocks in an archive database, use the `mina-missing-block-auditor` tool. +This starts only the Rosetta API process — no daemon, archive, or database. Point your Construction API calls to this endpoint for offline operations. -The tool outputs a list of state hashes of all the blocks in the database that are missing a parent. You can use this list to monitor the archive database for any missing blocks. To specify the URI of the PostgreSQL database, use the `--archive-uri` flag. +## Demo Mode -## Restore blocks +For development and testing, the demo mode launches a local sandbox with a simple genesis ledger and all components running inside a single container: -When you have block data (precomputed or extensional) available from [Back up block data](/node-operators/archive-node/archive-redundancy#back-up-block-data), you can restore missing blocks in an archive database using the tool `mina-archive-blocks`. +```bash +docker run -it --rm --name rosetta-demo \ + --entrypoint=./docker-demo-start.sh \ + -p 3085:3085 -p 3087:3087 \ + minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet +``` -1. Restore precomputed blocks from: +This creates an isolated network with no external connectivity — useful for developing integrations before connecting to mainnet or devnet. - - [Upload block data to Google Cloud Storage](/node-operators/archive-node/archive-redundancy#upload-block-data-to-google-cloud-storage) +## Available Images - - [Save block data from logs](/node-operators/archive-node/archive-redundancy#save-block-data-from-logs) +| Network | Image | Notes | +|---|---|---| +| **Mainnet (noble)** | `minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet` | amd64 | +| **Mainnet (bookworm)** | `minaprotocol/mina-rosetta:3.3.1-7b34378-bookworm-mainnet` | Also available for arm64 | +| **Devnet (bookworm)** | `minaprotocol/mina-rosetta:3.2.0-97ad487-bookworm-devnet` | Latest devnet | -``` - mina-archive-blocks --precomputed --archive-uri FILES -``` +Images are published on [Docker Hub](https://hub.docker.com/r/minaprotocol/mina-rosetta/tags). -2. For extensional blocks: (Generated from option [3](#generate-block-data-from-another-archive-database)) +## Verifying the API + +Once your node is synced, verify the Rosetta API is working. + +**List available networks:** +```bash +curl -s http://localhost:3087/network/list \ + -H 'Content-Type: application/json' \ + -d '{"metadata":{}}' | jq . ``` - mina-archive-blocks --extensional --archive-uri FILES + +**Get network status:** + +```bash +curl -s http://localhost:3087/network/status \ + -H 'Content-Type: application/json' \ + -d '{"network_identifier": {"blockchain": "mina", "network": "mainnet"}}' | jq . ``` -## Staking ledgers +**Query an account balance:** -Staking ledgers are used to determine slot winners for each epoch. Mina daemon stores staking ledger for the current and the next epoch after it is finalized. When transitioning to a new epoch, the "next" staking ledger from the previous epoch is used to determine slot winners of the new epoch and a new "next" staking ledger is chosen. Since staking ledgers for older epochs are no longer accessible, you can still keep them around for reporting or other purposes. +```bash +curl -s http://localhost:3087/account/balance \ + -H 'Content-Type: application/json' \ + -d '{ + "network_identifier": {"blockchain": "mina", "network": "mainnet"}, + "account_identifier": {"address": ""} + }' | jq . +``` -Export these ledgers using the mina cli command: +## Building Your Own Image - mina ledger export [current-staged-ledger|staking-epoch-ledger|next-epoch-ledger] +For most users, the official images are sufficient. If you need to build a custom image, see the [Rosetta README](https://github.com/MinaProtocol/mina/blob/master/src/app/rosetta/README.md) in the Mina repository for build instructions. -Epoch ledger transition happens once every 14 days (given slot-time = 3mins and slots-per-epoch = 7140). +## Questions and Support -The window to backup a staking ledger is ~27 days considering "next" staking ledger is finalized after k (currently 290) blocks in the current epoch and therefore is available for the rest of the current epoch and the entire next epoch. +- Post questions in [Mina GitHub Discussions](https://github.com/MinaProtocol/mina/discussions) +- Report bugs on [GitHub Issues](https://github.com/MinaProtocol/mina/issues) +- Join the [Mina Discord](https://discord.gg/minaprotocol) for community support --- -url: /node-operators/archive-node/docker-compose +url: /node-operators/delegation-program/foundation-delegation-program --- -# Docker Compose Archive - -This example demonstrates how to run a Mina archive node using Docker Compose for the Mainnet network. This Docker Compose setup includes a Postgres database, a bootstrap database with the latest SQL Dump available, an archive node, a Mina node and a Missing Blocks Guardian script to monitor and populate the gaps in the archive database - -Copy and paste the provided configuration into a `docker-compose.yml` file. Then run `docker compose up -d` to start the services, and use `docker compose logs -f` to monitor the logs. +# Mina Foundation Delegation Program -```yaml -services: - postgres: - image: postgres:17 - restart: always - environment: - POSTGRES_PASSWORD: postgres - POSTGRES_DB: archive - healthcheck: - test: ["CMD-SHELL", "psql -U postgres -d archive -tAc \"SELECT COUNT(*) FROM pg_database WHERE datname='archive';\" | grep -q '^1$'"] - interval: 5s - timeout: 10s - retries: 10 - volumes: - - './archive/postgresql/data:/var/lib/postgresql/data' - ports: - - '5432:5432' - bootstrap_db: - image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' - # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet - command: > - bash -c ' - curl -O https://storage.googleapis.com/mina-archive-dumps/mainnet-archive-dump-$(date +%F_0000).sql.tar.gz; - tar -zxvf mainnet-archive-dump-$(date +%F_0000).sql.tar.gz; - psql postgres://postgres:postgres@postgres:5432/archive -c " - ALTER SYSTEM SET max_connections = 500; - ALTER SYSTEM SET max_locks_per_transaction = 100; - ALTER SYSTEM SET max_pred_locks_per_relation = 100; - ALTER SYSTEM SET max_pred_locks_per_transaction = 5000; - " - psql postgres://postgres:postgres@postgres:5432/archive -f mainnet-archive-dump-$(date +%F_0000).sql; - ' - # For Devnet Network, replace "mainnet" references with "devnet" in the block above - depends_on: - postgres: - condition: service_healthy - missing_blocks_guardian: - image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' - # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet - command: > - bash -c ' - curl -O https://raw.githubusercontent.com/MinaFoundation/helm-charts/main/mina-archive/scripts/missing-blocks-guardian-command.sh; - export GUARDIAN_PRECOMPUTED_BLOCKS_URL=https://673156464838-mina-precomputed-blocks.s3.us-west-2.amazonaws.com/mainnet; - export MINA_NETWORK=mainnet; - export PG_CONN=postgres://postgres:postgres@postgres:5432/archive; - while true; do bash missing-blocks-guardian-command.sh; sleep 600; done - ' - # For Devnet Network, replace "mainnet" references with "devnet" in the block above - depends_on: - bootstrap_db: - condition: service_completed_successfully - mina_archive: - image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' - restart: always - command: - - mina-archive - - run - - --postgres-uri - - postgres://postgres:postgres@postgres:5432/archive - - --server-port - - "3086" - volumes: - - './archive/data:/data' - depends_on: - bootstrap_db: - condition: service_completed_successfully - mina_node: - image: 'minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet' - # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet - restart: always - entrypoint: [] - command: > - bash -c ' - mina daemon --archive-address mina_archive:3086 \ - --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ - --insecure-rest-server \ - --rest-port 3085 - ' - # use --peer-list-url https://bootnodes.minaprotocol.com/networks/devnet.txt for Devnet - ports: - - '3085:3085' - - '8302:8302' - depends_on: - bootstrap_db: - condition: service_completed_successfully -``` -Once the services are running, you can access the Mina node graphql endpoint at `http://localhost:3085/graphql` and the postgres database using `psql postgres://postgres:postgres@localhost:5432/archive`. -To retrieve the status of the Mina Node, run `docker compose exec mina_node mina client status` + +How to Participate in the Mina Foundation Delegation Program + ---- -url: /node-operators/archive-node/getting-started ---- +Learn how to receive a delegation from Mina Foundation. -# Archive Nodes Getting Started +:::info -Mina nodes are succinct by default, so they don't need to maintain historical information about the network, block, or transactions. +**PLEASE READ:** This affects the submission of uptime data and the Performance Score results for receiving a delegation. -For some use cases, it is useful to maintain this historical data on an archive node. +The sidecar tracking system was **discontinued on June 14th 2024**. Please use the **SNARK-work-based** uptime tracking system. -:::tip +- [Instructions for the SNARK-work-based uptime system](./uptime-tracking-system) -A zkApp can retrieve events and actions from one or more Mina archive nodes. If your smart contract needs to fetch events and actions from an archive node, see [How to Fetch Events and Actions](/zkapps/writing-a-zkapp/feature-overview/fetch-events-and-actions). +Please follow the latest updates and post questions in the [#delegation-program](https://discord.com/channels/484437221055922177/808895957978447882) channel on Mina Protocol Discord. ::: -An archive node is a regular mina daemon that is connected to a running `mina-archive` process using the `--archive-address` flag. +This Delegation Program is an implementation of the Mina Foundation Delegation Policy, the official policy on the delegations of MINA tokens from the Mina Foundation (referred to as “Foundation” below). -The daemon regularly sends blockchain data to the archive process that stores it in a [PostgreSQL](https://www.postgresql.org/) database. +If you are a block producer looking to participate in the Mina Foundation Delegation Program, this page explains how the program works and how to send uptime data and pay back rewards. -## Archive Node Requirements +:::note -**Software**: Supported environments include macOS, Linux (Debian 10, 11 and Ubuntu 20.04 LTS), and any host machine with Docker. +There are two delegation programs in Mina's ecosystem — one from the Mina Foundation and [one from o1Labs](https://www.o1labs.org/blog/delegation-policy-2024#o1labs-delegation-program-policy), Mina's ecosystem partner. This page details the program only for the Mina Foundation. -**Processor**: Only x86-64 CPU architecture is supported. +::: -**Hardware**: Running an archive node does not require any special hardware. +## Overview -In addition to the [PostgreSQL](https://www.postgresql.org/) database requirements, running an archive node on the Mina network requires at least: +Mina Foundation is committed to decentralizing the governance of the Mina protocol. In order to delegate their voting power, Mina Foundation delegates its tokens to community members through the [Mina Foundation Delegation Program](https://minaprotocol.com/blog/mina-foundation-delegation-policy). -- 8-core processor -- 32 GB of RAM -- 64 GB of free storage +In the Delegation Program, Mina Foundation delegates its tokens to a number of validators. As the validators earn staking rewards associated with the delegation, they must return the remaining rewards to Mina Foundation, **but can keep 8% of the staking rewards**. -Running an archive node requires some knowledge of managing a PostgreSQL database instance. You must set up a database, run the archive node, connect it to a daemon, and run queries on the data. +Block producers are selected on a quarterly basis, known as a delegation cycle, based on their Performance Scores (see below) and requirements outlined in the [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy). -## Install Mina, PostgreSQL, and the archive node package +## Eligibility Requirements -1. Install the latest version of Mina. +### How do I participate in the program? - You must upgrade to the latest version of the daemon. Follow the steps in [Getting Started](../block-producer-node/getting-started). +If you are new to the Mina Foundation Delegation Program, please make sure you have fulfilled the following requirements in order to be eligible for receiving a delegation from the Foundation's token allocation. -1. Download and install [PostgreSQL](https://www.postgresql.org/download/). +#### Step 1: Complete the application -1. Install the archive node package. +Review the [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy) then complete [the application form](https://docs.google.com/forms/d/e/1FAIpQLSduM5EIpwZtf5ohkVepKzs3q0v0--FDEaDfbP2VD4V6GcBepA/viewform). You will be asked to share your public key for receiving a delegation in the application. - - Ubuntu/Debian: +#### Step 2: Run the Uptime Tracking System - ``` - sudo apt-get install mina-archive=3.3.0-8c0c2e6 - ``` +The uptime tracking system sends recent blockchain data from your block producer node's perspective to a service. This service verifies whether this recent data is synced with the chain. If it is synced, it means your node is online. - - Docker: +Instructions for how to run the uptime tracking system: +- [SNARK-based Uptime System](./uptime-tracking-system) - ``` - minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet - ``` +#### Step 3: Pass the KYC/AML requirement (only if selected for delegation) -## Set up the archive node +In order to receive a delegation, you must also meet the KYC/AML requirements of Mina Foundation. -These steps might be different for your operating system, if you're connecting to a cloud instance of PostgreSQL, if your deployment uses Docker, or if you want to run these processes on different machines. +If you have recently been selected as a validator who will be receiving a delegation, but have not yet completed any of the KYC/AML requirements, please look out for an email with further instructions. -:::caution +For more details on this requirement, please review the policies outlined in the [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy). -Using the `--config` parameter ensures genesis accounts are inserted into the database, which is important to avoid gaps in account balances since the archive node stores only incremental changes. +:::note -However, inserting genesis accounts can take significant time and resources. You can skip `--config` if you're connecting to devnet or mainnet and starting from an existing archive database dump rather than an empty archive. +You can only complete this step if you’re receiving a delegation for the next cycle. When you are selected for a delegation, you will be contacted with instructions on how to proceed with completing the KYC/AML requirement. +::: -*Never* use it on long lived network such as mainnet or devnet +## Program Guidelines + +Now that you understand how to become eligible to participate in the program, please review the following guidelines to better understand how to receive a delegation and send back rewards. + +### How do I get selected for a delegation? +#### Measuring Uptime Performance Score + +The **Uptime Performance Score** is an important factor for selecting the public keys to receive a Mina Foundation Delegation Program delegation from a token provider. + +**Uptime** is the measurement of when your block producer node is online. In the Mina Foundation Delegation Program, uptime is measured in 20-minute windows. If your block producer node is online at any time within a given 20-minute window, you will be marked as online for that period. + +:::note +You may run more than one node with the same block producer key if you want to increase your chances of remaining online. ::: - - +:::tip -For production, run the archive database in the background, use your operating system service manager (like systemd) to run it for you, or use a postgres service hosted by a cloud provider. +To participate in the delegation program, your block producer node must stay online and connected to the network at all times. If you cannot maintain uptime, consider [delegating your stake](/node-operators/validator-node/staking-and-snarking) instead. -To run a local archive node in the foreground for testing: +::: -1. Start a local postgres server and connect to port 5432: +**Uptime Performance** is the percentage over a time period in which a block producer node is online and operational. - ```sh - postgres -p 5432 -D /usr/local/var/postgres - ``` +The time period of Mina Foundation's delegation programs is 90 days. - For macOS: +This percentage is based on all possible 20-minute windows over the last time period. - ```sh - brew services start postgres - ``` +### How do I track my performance score? -1. Create a local postgres database called `archive`: +The **leaderboard** shows the ranking of block producers participating in the delegation program based on uptime performance. The sooner you fix issues with your nodes sending uptime data, the sooner you'll be rewarded with a better uptime and position on the leaderboard. - ```sh - psql -p 5432 --h localhost -c "create database archive" - ``` +Your position on this leaderboard is an important factor in your public key being selected to receive delegation from Mina Foundation. -1. Load the mina archive schema into the archive database, (create_schema.sql and zkapp_tables.sql): +See the following uptime leaderboards for the latest uptime performance scores: +- Official Leaderboard: https://uptime.minaprotocol.com +- Community Leaderboards: + - Minataur: https://minataur.net/uptime - ```sh - psql -h localhost -p 5432 -d archive -f <(curl -Ls https://raw.githubusercontent.com/MinaProtocol/mina/release/3.0.2/src/app/archive/create_schema.sql) - ``` +### What happens after I receive a delegation? -1. Start the archive process on port 3086 and connect to the postgres database that runs on port 5432: +#### Payout Procedure - ```sh - mina-archive run \ - --postgres-uri postgres://localhost:5432/archive \ - --server-port 3086 - ``` +As per the Mina Foundation Delegation policy, there is a mandatory requirement on delegatees to use the latest version of the [payout script](https://github.com/jrwashburn/mina-pool-payout) to manage the return of rewards. -1. Start the mina daemon and connect it to the archive process that you started on port 3086: +This payout script includes: +- Mapping of delegating wallet addresses with the correct return wallet address +- Calculation of the Reward amounts and paybacks of the remaining rewards to the delegators. +- Specific memo which includes the md5 hash of Pool’s Public Key (MD5 hash) - ``` - mina daemon \ - ..... - --archive-address 3086 - ``` +For more details and to review the script, see the [README](https://github.com/jrwashburn/mina-pool-payout/blob/main/README.md) file in the `mina-pool-payout` repository. - To connect to an archive process on another machine, specify a hostname with `` i.e. `154.97.53.97:3086`. +#### Payout Addresses - - +You must return rewards to the address specified in the [Mina Delegation Program Return Addresses](https://docs.google.com/spreadsheets/d/1Fm4XSS9Xu4eWAhpM06sdySUKvLClR5SculXfP5o5sSc/edit?usp=sharing) mapping document. This will be covered when running the script in managing reward payback. -1. Install Docker on your machine. For more information, see [Docker](https://docs.docker.com/get-docker/). +#### Payout Frequency -2. Pull the archive node image from Docker Hub. +Rewards must be distributed at least once for a given epoch. You must send one payment in the amount of your obligation to the correct address specified in the [Mina Delegation Program Return Addresses](https://docs.google.com/spreadsheets/d/1Fm4XSS9Xu4eWAhpM06sdySUKvLClR5SculXfP5o5sSc/edit?usp=sharing) mapping document and if applicable, send the correct amount to the burn address. Both payments should have a memo field with the md5 hash value of your block producer public key. This is the easiest method to avoid confusion in tracking payments and will reduce the likelihood you will be incorrectly flagged as delinquent. - ```sh - docker pull minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet - ``` +All the rewards for epoch N must be delivered (ie. accepted in a block, not just sent) no later than slot number 3,500 of the next epoch. This gives you about a week to sort out these payments. -3. Pull and install the postgres image from Docker Hub. - ```sh - docker pull postgres - ``` +### How do I calculate the reward payout? -4. Start the postgres container and expose its networking to other containers. +For reference, this explanation describes how reward returns are calculated. - ```sh - docker run --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=postgres -d postgres - ``` +For details, see the [implementation](https://github.com/jrwashburn/mina-pool-payout/blob/main/src/core/payoutCalculator/PayoutCalculatorIsolateSuperCharge.ts) code. -5. Create a local postgres database called `archive`. - ```sh - docker exec -it postgres createdb -U postgres archive - ``` +#### Reward Specifications -6. Load the mina archive schemas into the archive database, (create_schema.sql and zkapp_tables.sql.) +You must send back at least the amount specified by this mechanism. - ```sh - curl -Ls https://raw.githubusercontent.com/MinaProtocol/mina/1551e2faaa246c01636908aabe5f7981715a10f4/src/app/archive/create_schema.sql | docker exec -i postgres psql -U postgres -d archive +At the end of each epoch, do all of the following: - curl -Ls https://raw.githubusercontent.com/MinaProtocol/mina/1551e2faaa246c01636908aabe5f7981715a10f4/src/app/archive/zkapp_tables.sql | docker exec -i postgres psql -U postgres -d archive - ``` +1. Compute the total stake delegated to your account for the epoch -7. Create a local directory to store the archive node data. +2. Compute the share of stake from the token provider (from both accounts) by dividing the token provider delegation by the total stake. (i.e. `provider_share = provider_delegation / total_stake`). The resulting share should be between 0 and 1. - ```sh - mkdir -p /tmp/archive - ``` +3. For each block produced that has a non-zero block-reward on the canonical chain rewards must be calculated based on 720 MINA. -8. Start the archive node. +4. Calculate the Mina Foundation payout by multiplying the coinbase reward (equal to `720 MINA` ) by the provider share calculated in the previous step minus an 8% percent fee. (i.e. `payout = (provider_share * 0.92) * 720)`. - ```sh - docker run \ - --name archive \ - -p 3086:3086 \ - -v /tmp/archive:/data \ - minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet \ - mina-archive run \ - --postgres-uri postgres://postgres:postgres@postgres:5432/archive \ - --server-port 3086 - ``` +5. Send a transaction to the token provider accounts with the appropriate payout and memo - please follow the rules in the "Payout Attribution" section with your transaction. + More details in the following source code parts: [PayoutCalculatorIsolateSuperCharge.ts](https://github.com/jrwashburn/mina-pool-payout/blob/7f00dbd9e693f76ea6a950c29862120a170625a9/src/core/payoutCalculator/PayoutCalculatorIsolateSuperCharge.ts#L126) and [ConfigurationManager.ts](https://github.com/jrwashburn/mina-pool-payout/blob/7f00dbd9e693f76ea6a950c29862120a170625a9/src/configuration/ConfigurationManager.ts#L21C15-L21C15). -9. Start the mina daemon and connect it to the archive process that you started on port 3086: - ``` - mina daemon \ - ..... - --archive-address 3086 - ``` +The block producer can keep all of the transaction fees or divide them equally amongst the other members of the pool. - To connect to an archive process on another machine, specify a hostname with `` i.e. `154.97.53.97:3086`. +:::tip - - +The canonical chain will be calculated as 12 blocks behind any tip at slot 3,500 of the next epoch. +::: -Docker Compose is a tool for defining and running multi-container Docker applications. -With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration. +#### Example of a Reward Payout Calculation -1. Install Docker and Docker Compose on your machine. For more information, see [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/). +Consider the following: -2. Pull the required images: +- Account A has 2 million MINA +- Account B is controlled by the Mina Foundation and has 6 million MINA which are delegated to Account A via the Delegation Program. +- Account C is controlled by a third party and has 2 million MINA. This account also delegates to Account A and has also agreed to Account A retaining 8% of the staking rewards. - ```sh - docker pull minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet - docker pull postgres - ``` +In this example, the total amount of stake for Account A is 10 million calculated by adding up the balances from all the accounts. -3. Create a local directory to store the archive node data. +Now let's consider Epoch 5. The share of the stake from the Foundation is `6 million MINA / 10 million MINA = 0.6` or 60%. The share of the stake from Account B is `2 million MINA / 10 million MINA = 0.2` or 20%. - ```sh - mkdir -p /tmp/archive - ``` +3 blocks are produced in this epoch that end up on the canonical chain. The blocks were won by Account A. -4. Create a `docker-compose.yml` file with the following contents: +1. Account A retains, 0.2 x 720 MINA x 3 blocks = 432 MINA. +2. Mina Foundation, Account B, payout would be: (0.6 x 0.92) x 720 MINA x 3 blocks = 1,192.32 MINA. +3. Account A retains 8%, (0.6 x 0.08) x 720 MINA x 3 blocks = 103.68 MINA. +3. Account C payout would be: (0.2 x 0.92) x 720 MINA x 3 blocks = 397.44 MINA. +4. Account A retains 8%, (0.2 x 0.08) x 720 MINA x 3 blocks = 34.56 MINA - ```yml - services: - postgres: - image: postgres:17 - environment: - POSTGRES_PASSWORD: postgres - volumes: - - './postgres-data:/var/lib/postgresql/data' - ports: - - '5432:5432' - archive: - image: 'minaprotocol/mina-archive:3.3.0-8c0c2e6-bullseye-mainnet' - command: >- - mina-archive run --postgres-uri - postgres://postgres:postgres@postgres:5432/archive --server-port 3086 - volumes: - - '/tmp/archive:/data' - ports: - - '3086:3086' - depends_on: - - postgres - ``` -5. Start the archive node. +### Relevant Links - ```sh - docker compose up - ``` +- [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy) +- [SNARK-based Uptime System](/node-operators/delegation-program/uptime-tracking-system) -6. Start the mina daemon and connect it to the archive process that you started on port 3086: +#### Need any help? - ``` - mina daemon \ - ..... - --archive-address 3086 - ``` +Post your questions in the [#delegation-program](https://discord.com/channels/484437221055922177/808895957978447882) channel on the Mina Protocol Discord. - To connect to an archive process on another machine, specify a hostname with `` i.e. `154.97.53.97:3086`. +--- +url: /node-operators/delegation-program +--- - - +# Delegation Program +The Mina Foundation Delegation Program enables block producers to receive token delegations from the Mina Foundation and earn staking rewards. -## Using the Archive Node +:::info -Take a look at the tables in the database. +The sidecar tracking system was **discontinued on June 14th 2024**. Please use the **SNARK-work-based** uptime tracking system. See [Uptime Tracking System](/node-operators/delegation-program/uptime-tracking-system) for setup instructions. -To list the tables, run the `\dt` command in psql. The output will look like this: +::: -``` - List of relations - Schema | Name | Type | Owner ---------+-------------------------------+-------+------- - public | account_identifiers | table | mina - public | accounts_accessed | table | mina - public | accounts_created | table | mina - public | blocks | table | mina - public | blocks_internal_commands | table | mina - public | blocks_user_commands | table | mina - public | blocks_zkapp_commands | table | mina - public | epoch_data | table | mina - public | internal_commands | table | mina - public | protocol_versions | table | mina - public | public_keys | table | mina - public | snarked_ledger_hashes | table | mina - public | timing_info | table | mina - public | token_symbols | table | mina - public | tokens | table | mina - public | user_commands | table | mina - public | voting_for | table | mina - public | zkapp_account_precondition | table | mina - public | zkapp_account_update | table | mina - public | zkapp_account_update_body | table | mina - public | zkapp_account_update_failures | table | mina - public | zkapp_accounts | table | mina - public | zkapp_action_states | table | mina - public | zkapp_amount_bounds | table | mina - public | zkapp_balance_bounds | table | mina - public | zkapp_commands | table | mina - public | zkapp_epoch_data | table | mina - public | zkapp_epoch_ledger | table | mina - public | zkapp_events | table | mina - public | zkapp_fee_payer_body | table | mina - public | zkapp_field | table | mina - public | zkapp_field_array | table | mina - public | zkapp_global_slot_bounds | table | mina - public | zkapp_length_bounds | table | mina - public | zkapp_network_precondition | table | mina - public | zkapp_nonce_bounds | table | mina - public | zkapp_permissions | table | mina - public | zkapp_states | table | mina - public | zkapp_states_nullable | table | mina - public | zkapp_timing_info | table | mina - public | zkapp_token_id_bounds | table | mina - public | zkapp_updates | table | mina - public | zkapp_uris | table | mina - public | zkapp_verification_key_hashes | table | mina - public | zkapp_verification_keys | table | mina -(45 rows) - -``` +- **[Foundation Delegation Program](/node-operators/delegation-program/foundation-delegation-program)** -- Eligibility requirements, performance scoring, payout procedures, and reward calculations. +- **[Uptime Tracking System](/node-operators/delegation-program/uptime-tracking-system)** -- Instructions for the SNARK-based uptime tracking system used to measure node uptime. -Use the `\d table_name` to look at the structure of a table in the database. +## Related -For example to see the structure of the user_commands table, run the `\d user_commands` command in psql. The output will look like this: +- [Staking and Snarking](/node-operators/validator-node/staking-and-snarking) -- How staking and delegation work on Mina +- [Block Producers](/node-operators/block-producer-node) -- Getting started with block production -``` - Table "public.user_commands" - Column | Type | Collation | Nullable | Default ---------------+-------------------+-----------+----------+------------------------------------------- - id | integer | | not null | nextval('user_commands_id_seq'::regclass) - command_type | user_command_type | | not null | - fee_payer_id | integer | | not null | - source_id | integer | | not null | - receiver_id | integer | | not null | - nonce | bigint | | not null | - amount | text | | | - fee | text | | not null | - valid_until | bigint | | | - memo | text | | not null | - hash | text | | not null | -Indexes: - "user_commands_pkey" PRIMARY KEY, btree (id) - "user_commands_hash_key" UNIQUE CONSTRAINT, btree (hash) -Foreign-key constraints: - "user_commands_fee_payer_id_fkey" FOREIGN KEY (fee_payer_id) REFERENCES public_keys(id) - "user_commands_receiver_id_fkey" FOREIGN KEY (receiver_id) REFERENCES public_keys(id) - "user_commands_source_id_fkey" FOREIGN KEY (source_id) REFERENCES public_keys(id) -Referenced by: - TABLE "blocks_user_commands" CONSTRAINT "blocks_user_commands_user_command_id_fkey" FOREIGN KEY (user_command_id) REFERENCES user_commands(id) ON DELETE CASCADE +--- +url: /node-operators/delegation-program/uptime-tracking-system +--- -``` +# Uptime Tracking System -Review the full schema at [/archive/create_schema.sql](https://github.com/minaProtocol/mina/blob/master/src/app/archive/create_schema.sql) and [/archive/zkapp_tables.sql](https://github.com/MinaProtocol/mina/blob/berkeley/src/app/archive/zkapp_tables.sql) -## Query the database -Now that you know the structure of the data, try some queries. + +Instructions for the SNARK-based uptime tracking system. + -**Example 1:** Find all blocks that were created by your public key: +Learn how to set up the uptime tracking system for the Mina Foundation Delegation Program -``` -SELECT * -FROM blocks AS b -INNER JOIN public_keys AS pk1 ON b.creator_id = pk1.id -WHERE value = 'MY_PK' -``` +:::info -**Example 2:** Find all payments received by your public key: +**PLEASE READ:** This affects the submission of uptime data and the Performance Score results for receiving a delegation. -``` -SELECT * -FROM user_commands AS uc -JOIN blocks_user_commands AS buc ON uc.id = buc.user_command_id -JOIN public_keys AS pk ON uc.receiver_id = pk.id -WHERE value = 'MY_PK' -AND type = 'payment' -``` -**Example 3:** Find the block at height 12 on the canonical chain: +The sidecar tracking system was **discontinued on June 14th 2024**. Please use the **SNARK-work-based** uptime tracking system. -``` -WITH RECURSIVE chain AS ( - (SELECT ... FROM blocks b WHERE height = (select MAX(height) from blocks) - ORDER BY timestamp ASC - LIMIT 1) +- [Instructions for the SNARK-work-based uptime system](./uptime-tracking-system) - UNION ALL +Please follow the latest updates and post questions in the [#delegation-program](https://discord.com/channels/484437221055922177/808895957978447882) channel on Mina Protocol Discord. - SELECT ... FROM blocks b - INNER JOIN chain - ON b.id = chain.parent_id AND chain.id <> chain.parent_id -) SELECT ..., pk.value as creator FROM chain c - INNER JOIN public_keys pk - ON pk.id = c.creator_id - WHERE c.height = 12 -``` +::: -**Example 3:** List the counts of blocks created by each public key and sort them in descending order" +In order to maintain eligibility for various grants and the Mina Foundation Delegation Program, you must connect to our uptime tracking endpoint to run the SNARK-work-based uptime tracking system with your daemon to report node uptime. -``` -SELECT p.value, COUNT(*) FROM blocks -INNER JOIN public_keys AS p ON creator_id = ip.id -GROUP BY p.value -ORDER BY count DESC; -``` +If you are required to keep your node online for a grant or specific program, you must run a small uptime tracking program that will report your daemon's uptime. This tutorial will walk you through the process of installing, configuring and running the uptime tracking system. -**Example 4:** List the counts of applied payments created by each public key and sort them in descending order: +:::note -``` -SELECT p.value, COUNT(*) FROM user_commands -INNER JOIN public_keys AS p ON source_id = p.id -WHERE status = 'applied' -AND type = 'payment' -GROUP BY p.value ORDER BY count DESC; -``` -**Example 5** Get the latest block: +

We recommend that you save some of your SNARK work data logs. You can also share the logs with us if you’re interested in helping out with data checks.

-``` -SELECT -height as blockheight, -global_slot_since_genesis as globalslotsincegenesis, -global_slot_since_hard_fork as globalslot, -state_hash as statehash, -parent_hash as parenthash, -ledger_hash as ledgerhash, -to_char(to_timestamp(cast ("timestamp" as bigint) / 1000) AT TIME ZONE 'UTC', 'YYYY-MM-DD"T"HH24:MI:SS') || '.' || - LPAD(((cast("timestamp" as bigint) % 1000)::text), 3, '0') || 'Z' as datetime -FROM blocks -WHERE id in (SELECT MAX(id) FROM blocks); +::: -``` +### Pre-requisites -**Example 6** Identify blocks with missing parents, between blockheight 500 and blockheight 5000 -``` -SELECT height -FROM blocks -WHERE parent_id is null AND height >= 500 AND height <= 5000 and height > 1; +Make sure you updated your node to at least release (3.0.0+): -``` +## How to set up the uptime system ---- -url: /node-operators/archive-node ---- +The SNARK-work-based uptime system is built into the mina daemon. The new uptime tracking system no longer requires importing your keypair and supports a new flag `--uptime-submitter-key`, which takes the path to your private key, just like `--block-producer-key`. -# About Archive Nodes +To get started, pass in the following information to the daemon: -Mina nodes are succinct by default, so they don't need to maintain historical information about the network, block, or transactions. An archive node is a Mina node that stores the historical chain data to a persistent data source, PostgreSQL, so it can later be retrieved. For some use cases, it is useful to maintain this historical data on an archive node. +- The path to the private key with the flag: +`--uptime-submitter-key ` -:::tip +- The URL of our testing backend server with the flag: +`--uptime-url https://uptime-backend.minaprotocol.com/v1/submit` -A zkApp can retrieve events and actions from one or more Mina archive nodes. If your smart contract needs to fetch events and actions from an archive node, see [How to Fetch Events and Actions](../../zkapps/writing-a-zkapp/feature-overview/fetch-events-and-actions). +- The password for the keypair associated with the given public key has the environment variable `UPTIME_PRIVKEY_PASS=`. If you are using a .mina-env file on Debian then this value should be on its own line, not included in `EXTRA_FLAGS=`. -::: +Here’s an example of what your .mina-env file should look like for Debian: -An archive node is a regular mina daemon that is connected to a running `mina-archive` process. +~~~ +EXTRA_FLAGS="--block-producer-key /home/mina/my-wallet --uptime-submitter-key /home/mina/my-wallet --uptime-url https://uptime-backend.minaprotocol.com/v1/submit" +UPTIME_PRIVKEY_PASS= +MINA_PRIVKEY_PASS= +LOG_LEVEL=Info +FILE_LOG_LEVEL=Debug +~~~ -The mina daemon regularly sends blockchain data to the `mina-archive` process that then stores it in a [PostgreSQL](https://www.postgresql.org/) database. -Running an archive node requires some knowledge of managing a PostgreSQL database instance. You must set up a database, run the archive node, connect it to a daemon, and run queries on the data. +### Relevant Links -# Archive Node +- [Latest Release Notes for Mina Protocol](https://github.com/MinaProtocol/mina/releases) +- [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy) +- [Mina Docs: Mina Foundation Delegation Program](/node-operators/delegation-program) -This section describes how to set up and run an Archive node within the Mina protocol. -- [Getting Started with Archive Nodes](archive-node/getting-started) - A beginner's guide to launching an Archive Node. -- [Ensuring Archive Redundancy](archive-node/archive-redundancy) - Strategies to mitigate the risk of losing archived data. +#### Need any help? Post your questions in the [#Delegation-Program](https://discord.gg/ywDzwmGABT) channel on Discord. --- -url: /node-operators/block-producer-node/docker-compose +url: /node-operators/downgrading-to-older-versions --- -# Docker Compose Block Producer +# Downgrading to Older Versions -This example demonstrates how to run a Mina Block Producer node using Docker Compose for the Mainnet network. The Docker Compose setup includes a Mina Block Producer node, and another script to generate a wallet key. +If you are running a Mina node on a version above 3.3.0 and need to roll back to 3.3.0 or below, you can convert the on-disk state in place using `mina-storage-converter`. This avoids a full rebootstrap from a remote S3 ledger bucket, which can save significant time. -Copy and paste the provided configuration into a `docker-compose.yml` file. Then run `docker compose up -d` to start the services, and use `docker compose logs -f` to monitor the logs. +## When is downgrading needed? -```yaml -services: - generate_wallet_key: - image: 'minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet' - # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet - environment: - MINA_PRIVKEY_PASS: PssW0rD - entrypoint: [] - command: > - bash -c ' - mina advanced generate-keypair --privkey-path /root/.mina-config/keys/wallet-key - chmod -R 0700 /root/.mina-config/keys - chmod -R 0600 /root/.mina-config/keys/wallet-key - ' - volumes: - - './node/mina-config:/root/.mina-config' - mina_block_producer: - image: 'minaprotocol/mina-daemon:3.3.0-8c0c2e6-bullseye-mainnet' - # image: 'gcr.io/o1labs-192920/mina-daemon:3.3.0-alpha1-6929a7e-bullseye-devnet' # Use this image for Devnet - restart: always - environment: - MINA_PRIVKEY_PASS: PssW0rD - entrypoint: [] - command: > - bash -c ' - mina daemon \ - --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ - --block-producer-key /root/.mina-config/keys/wallet-key - ' - # use --peer-list-url https://bootnodes.minaprotocol.com/networks/devnet.txt for Devnet - volumes: - - './node/mina-config:/root/.mina-config' - ports: - - '8302:8302' - depends_on: - generate_wallet_key: - condition: service_completed_successfully -``` +Downgrading is typically necessary when a fork fails and there is a RocksDB version bump between the stop slot release and the pre-stop slot release. In such cases, the development team will notify node operators that a downgrade is required to continue operating on the correct chain. ---- -url: /node-operators/block-producer-node/getting-started ---- +## Debian/Ubuntu -# Block Producer Getting Started +### 1. Stop the Mina daemon -:::note +Ensure your Mina node is fully shut down before proceeding: -Before following this guide, complete the Validator Node setup — from [Requirements](/node-operators/validator-node/requirements) through [Connect to Mainnet or Devnet](/node-operators/validator-node/connecting-to-the-network). You should have a synced node and a [key pair](/node-operators/validator-node/generating-a-keypair) before proceeding. +```sh +mina client stop daemon +``` -::: +Or however you normally stop your node process. Verify it is no longer running before continuing. -## Start the daemon +### 2. Install storage toolbox packages -Run the daemon with the `--block-producer-key` flag pointing to your wallet key: +Install the required toolbox packages that provide `mina-storage-converter` and the RocksDB scanners: ```sh -mina daemon --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ - --block-producer-key ~/keys/my-wallet +sudo apt-get install -y mina-daemon-storage-toolbox mina-daemon-recovery-storage-toolbox ``` -To send block rewards to a different account (e.g. a cold wallet), add the `--coinbase-receiver` flag: +### 3. Convert on-disk state -```sh -mina daemon --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt \ - --block-producer-key ~/keys/my-wallet \ - --coinbase-receiver $RECEIVER_PUBLICKEY -``` +Run `mina-storage-converter` to convert the local database to the format expected by the older version. -## Verify block production - -Run `mina client status` and check the `Block producers running` field: - -```text -Block producers running: 1 (B62q...) -Coinbase receiver: Block producer -Next block will be produced in: in 7.077h for slot: ... +```sh +mina-storage-converter \ + --node-dir ${NODE_DIR} \ + --current-scanner /usr/lib/mina/storage/*/${SOURCE_VERSION}/mina-rocksdb-scanner \ + --stable-scanner /usr/lib/mina/storage/*/${TARGET_VERSION}/mina-rocksdb-scanner ``` ---- -url: /node-operators/block-producer-node/hot-cold-block-production ---- - -# Hot and Cold Block Production +Where: -Block production requires a node connected to the internet, which means the block producer's private key is exposed on an online machine. To mitigate this risk, Mina supports a hot/cold wallet pattern: +- `NODE_DIR` is the path to your node's config directory. This is usually `~/.mina-config` if you haven't set it explicitly. +- `SOURCE_VERSION` is the version you are downgrading from. +- `TARGET_VERSION` is the version you are downgrading to (e.g. `3.3.0`). -- A **[hot wallet](/glossary#hot-wallet)** has its private key on an internet-connected machine. It is used to run the block producer node but should hold minimal funds. -- A **[cold wallet](/glossary#cold-wallet)** has its private key stored offline (e.g. generated on an air-gapped laptop or a hardware wallet like [Ledger](https://shop.ledger.com/)). It holds the majority of your stake. +The `*` wildcard lets bash resolve the RocksDB version directory automatically, so you don't need to know which RocksDB version is bundled with each Mina release. -By delegating from your cold wallet to your hot wallet, you can produce blocks — and earn all associated rewards — while keeping the bulk of your funds in cold storage. +The tool will prompt for confirmation before making changes. -## Setup +### 4. Install the target version -You need at least two accounts: one cold wallet and one hot wallet. See [Generating a Key Pair](/node-operators/validator-node/generating-a-keypair) for key generation instructions. +Install the older Mina package. For example, to install 3.3.0: -### 1. Create a hot wallet +```sh +sudo apt-get install --allow-downgrades -y mina-mainnet=3.3.0 +``` -Generate a key pair on your block producer machine. Do **not** use a Ledger or other hardware security module (HSM) for this key — the private key must be accessible to the mina daemon. +### 5. Start the Mina daemon -Take note of your hot wallet public key. +Start your node as usual: -### 2. Fund the hot wallet +```sh +mina daemon ${YOUR_EXTRA_DAEMON_ARGS_HERE} +``` -Your hot wallet must be present in the consensus ledger before it can be used for staking. +Your node should resume from the converted local state without needing to rebootstrap. -- If your hot wallet address is present in the genesis ledger, no further action is needed. -- Otherwise, send at least enough MINA to cover the account creation fee to your hot wallet address. +## Docker -### 3. Create a cold wallet +On-disk state conversion is only possible if your `mina-config` directory is persisted as a volume outside the container (e.g. via `--mount "type=bind,source=$(pwd)/.mina-config,dst=/root/.mina-config"`). -Generate a key pair using the most secure method available — preferably on a machine disconnected from the internet, or on a hardware wallet. +:::caution -### 4. Fund the cold wallet +If your `mina-config` is not persisted outside of the container, there is no way to convert the on-disk state. You will need to rebootstrap from scratch after switching to the older image. -Your cold wallet must be present in the consensus ledger before its stake can be counted when delegated to your hot wallet. +::: -- If your cold wallet address is present in the genesis ledger, no further action is needed. -- Otherwise, send enough MINA to meaningfully participate in consensus to your cold wallet address. +Since the Docker image is a Debian/Ubuntu environment with the Mina Debian package installed, you can run the same conversion steps inside the container. The default `NODE_DIR` inside the container is `/root/.mina-config`. -### 5. Delegate from cold to hot +### 1. Stop the running container -Delegate your cold wallet's stake to your hot wallet: +Assume your mina daemon container is running with name `mina-node` ```sh -mina client delegate-stake \ - --sender $COLD_PUBLIC_KEY \ - --receiver $HOT_PUBLIC_KEY +docker stop mina-node ``` -If your cold wallet is on a Ledger, follow the delegate instructions in the [Mina Ledger app README](https://github.com/jspada/ledger-app-mina/tree/v1.0.0-beta.2). - -### 6. Start producing blocks +### 2. Install toolbox packages and run the converter -Follow the [Getting Started](/node-operators/block-producer-node/getting-started) guide using your hot wallet key as the `--block-producer-key`. +Use the current (newer) image to install the toolbox packages and run the conversion against your mounted `mina-config` volume: -## Why can't I use an HSM directly? +```sh +docker run -it --rm \ + --entrypoint bash \ + --mount "type=bind,source=$(pwd)/.mina-config,dst=/root/.mina-config" \ + minaprotocol/mina-daemon:${SOURCE_VERSION}-bullseye-mainnet \ + -c "apt-get update && apt-get install -y mina-daemon-storage-toolbox mina-daemon-recovery-storage-toolbox && mina-storage-converter --node-dir /root/.mina-config --current-scanner /usr/lib/mina/storage/*/${SOURCE_VERSION}/mina-rocksdb-scanner --stable-scanner /usr/lib/mina/storage/*/${TARGET_VERSION}/mina-rocksdb-scanner" +``` -You may wonder why Mina can't let you produce blocks directly from a secure enclave or HSM. Two components of block production require the private key in ways that make this impractical: +### 3. Start a new container with the target version -### Finding eligible slots +Remove the old container and start with the target image: -A block producer determines slot eligibility by evaluating a VRF (verifiable random function) with their private key. The VRF must be evaluated for your account and every account that delegates to you, for all slots within an epoch. +```sh +docker rm mina-node -### Creating the blockchain SNARK +docker run --name mina-node -d \ + -p 8302:8302 \ + --restart=always \ + --mount "type=bind,source=$(pwd)/.mina-config,dst=/root/.mina-config" \ + minaprotocol/mina-daemon:${TARGET_VERSION}-bullseye-mainnet \ + daemon ${YOUR_EXTRA_DAEMON_ARGS_HERE} +``` -When a block producer wins a slot, they must create a SNARK proof that the new block is a valid extension of the existing chain. This proof embeds VRF information using the private key — replacing the simple signature that other protocols use. Creating this proof is computationally expensive and relies on advanced cryptography that is extremely difficult, and likely impossible, to perform quickly enough inside today's secure hardware. +Your node should resume from the converted local state without needing to rebootstrap. --- -url: /node-operators/block-producer-node +url: /node-operators/exchange-operators --- -# About Block Producers +# Exchange Operators -The role of a block producer in Mina is to achieve [consensus](https://minaprotocol.com/blog/what-is-ouroboros-samasika) and provide security to the blockchain. The block producer is responsible for creating new blocks that include recent transactions broadcast on the network and a blockchain proof that proves the current state of the chain is valid. -In Mina, anyone can become a block producer. There is an unbounded number of participants with the chance of producing a block proportional to the funds staked. Funds are not locked and are not subject to slashing. -In return for staking funds and generating the required blockchain proofs, blocks that are produced and included in the canonical chain are rewarded in the form of a coinbase and transaction fees, less any fees paid to purchase required [transaction SNARK work](/node-operators/snark-workers). + +Exchange operators are node operators who run additional infrastructure for blockchain integration. + -To successfully produce a block, a block producer must have the current state of the blockchain. A block producer must have enough available compute to produce a blockchain SNARK within the slot time and be connected to peers to broadcast the generated block within an acceptable delay as defined by the network consensus parameters. +To list MINA on your exchange, you need to run a Mina node with archive and Rosetta API support. This page provides an overview of the components involved and links to the relevant setup guides. -## Block Producers +## Components -This section describes how to run a Block Producer on the Mina protocol. +### Validator Node -- [Getting Started](block-producer-node/getting-started) - How to install and get started running a block producer. -- [Hot and Cold Block Production ](block-producer-node/hot-cold-block-production) - How to get started running a block producer. +A Mina daemon connected to the network. This is the base requirement for any node operator. See [Validator Node](/node-operators/validator-node) for setup instructions. ---- -url: /node-operators/block-producer-node/staking-service-guidelines ---- +### Archive Node -# Staking Service Guidelines +An archive node stores the full history of the blockchain in a PostgreSQL database. Exchanges need this to query historical transactions and track deposits. -The important parts of running a staking service are predicting/determining winning slots in which you can produce blocks and paying out participants. The Mina protocol does not automatically payout rewards to delegates, so part of running a staking service is manually paying out participants by [sending many transactions](#sending-many-transactions). +- [Archive Node](/node-operators/archive-node) — set up and run an archive node -This document aims to explain the different components that you should think about when managing those payouts. Specifically, this document provides an understanding of odds of winning blocks, gathering data from the ledger for later use, and computing relevant staking payout information from this data. +### Rosetta API -## Staking Rewards +[Rosetta](https://www.rosetta-api.org/) is a standardized API for blockchain integration. It is the recommended way to integrate MINA deposits, withdrawals, and balance queries. -The coinbase reward for producing a block is 720 tokens. +- [Rosetta API Overview](/node-operators/data-and-history/rosetta) — architecture and available endpoints +- [Run with Docker](/node-operators/rosetta/run-with-docker) — quickest way to get started +- [Docker Compose](/node-operators/rosetta/docker-compose) — production setup with all services +- [Build from Sources](/node-operators/rosetta/build-from-sources) — build and run from source +- [Code Samples](/node-operators/rosetta/samples) — example integrations for deposits, withdrawals, and block scanning -## Dumping Staking Ledgers +## FAQ -In order to compute odds of winning a block for a given epoch, or to retroactively compute the coinbase reward a given account would receive, you need to have the staking ledger from that epoch. Mina daemons only keep around the staking ledger for the current epoch and the staking ledger for the next epoch, so if you want to capture a staking ledger for an epoch, you need to do it before or during that epoch. +See the [Exchange Integration FAQ](/node-operators/faq#exchange-integration) for answers to common questions about listing MINA, account creation fees, transaction memos, staking, and more. -The `mina ledger export` command can be used to export ledgers from a running daemon: +--- +url: /node-operators/faq +--- -``` -Print the specified ledger (default: staged ledger at the best tip). Note: Exporting snarked ledger is an expensive operation and can take a few seconds +# FAQ - mina ledger export STAGED-LEDGER|SNARKED-LEDGER|STAKING-EPOCH-LEDGER|NEXT-EPOCH-LEDGER +## General -=== flags === +### What is Mina Signer? - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--plaintext] Use plaintext input or output - (default: JSON) - (alias: -plaintext) - [--state-hash STATE-HASH] State hash, if printing a staged - ledger (default: state hash for the - best tip) - (alias: -state-hash) - [-help] print this help text and exit - (alias: -?) +The [Mina Signer](https://github.com/o1-labs/o1js/blob/main/src/mina-signer/README.md) NodeJS SDK allows you to sign strings, payments, and delegations using Mina's key pairs for various specified networks. -``` +The Mina Signer supersedes the deprecated Client SDK. +## SNARKs and SNARK Workers -It requires an argument to identifier of the ledger you wish to export. The table below describes what each of these identifiers represent. +### If I run a SNARK worker, how do I get paid for SNARKs that I generate? -
+Block producers (the validators who add new blocks to the blockchain) are required to buy SNARKs from the network (called the snarketplace) and pay out some of their block reward as fees to the SNARK workers who generated SNARKs. This workflow creates a secondary incentive mechanism in the network to reward nodes that help compress transactions. -| Identifier | Description | -|----------------------|-----------------------------------------------------------------| -| staking-epoch-ledger | The staking ledger for the current epoch. | -| next-epoch-ledger | The staking ledger for the next epoch (epoch after current). | -| staged-ledger | The most recent staged ledger (from the best tip of that node). | -| snarked-ledger | The most recent snarked ledger (from the best tip of that node).| +### Is generating SNARKs similar to Proof-of-Work (PoW) mining? -
+No, they are different in several ways: +- SNARK proof-of-work is deterministic, while PoW mining requires randomly calculating hashes to try and solve a puzzle. There is no luck element in SNARK work — if a SNARK worker wants to generate a SNARK of a transaction, they only need to generate the proof once. This means SNARK work is much less expensive and less environmentally wasteful, as the compute is all spent towards a productive goal. +- There is no difficulty increase for SNARK work, as there is with PoW mining. In fact, as SNARK constructions, and proof generation times improve, the difficulty can actually decrease. +- SNARK work is not directly involved in consensus. SNARK workers play no role in determining the next state of the blockchain. Their role is to simply generate SNARKs of transactions observed in the network +- As a SNARK worker, there is no requirement for uptime. PoW miners need to run their rigs non-stop to ensure they don't miss out on a potential block. SNARK workers can come online and offline as they please — it is more like Uber, where there is always be work to be done, and nobody needs to say ahead of time when they want to work. -In order to ensure you always have each staking ledger available for use after epochs have expired, we recommend exporting the staking-epoch-ledger every (7140 × 3) ÷ 60 = 357 hours (there are 7140 slots in an epoch, and each slot is 3 minutes long). +### Why have my SNARKs not been included? (A.K.A. How should I price my SNARKs?) -By default, ledgers are exported as json data. See `mina ledger export -help` for documentation of flags which will enable other formats. When output as json, the ledger will be represented as an array of account objects. Below is an example of what an account object in json looks like. +Even though your SNARK worker might be producing SNARKs at a breakneck pace, if someone else produces a cheaper proof for a particular job you have already completed, their SNARK is preferred due to its lower fee. -```json -{ - "pk": "B62qrwZRsNkU39TrGpFwDdpRS2JaCB2yFZKMFNqLFYjcqGE5G5fWA8p", - "balance": "17000", - "delegate": "B62qrwZRsNkU39TrGpFwDdpRS2JaCB2yFZKMFNqLFYjcqGE5G5fWA8p", - "token": "1", - "token_permissions": {}, - "receipt_chain_hash": "2mzbV7WevxLuchs2dAMY4vQBS6XttnCUF8Hvks4XNBQ5qiSGGBQe", - "voting_for": "3NK2tkzqqK5spR2sZ7tujjqPksL45M3UUrcA4WhCkeiPtnugyE2x", - "permissions": { - "stake": true, - "edit_state": "signature", - "send": "signature", - "set_delegate": "signature", - "set_permissions": "signature", - "set_verification_key": "signature" - } -} -``` +Pricing your SNARKs is a delicate balance between the cost of compute, the market environment (demand for SNARKs), your SNARK throughput, and the speed at which each of your SNARK worker processes can produce SNARKs. Sometimes, it might even be economically prudent to turn off your SNARK worker altogether until the market improves. -For a running staking service, you are interested in the accounts which you control and the accounts which are staking to accounts you control. As an example, if we only had one account in our staking service, we could grab all the accounts we are interested in for a ledger using a command like: +### Will SNARK workers require more storage and computing power over time? What about compared to Mina full nodes? -```sh -mina ledger export staking-epoch-ledger | jq "$(cat <) where one supplier of a product seeks to exhaust competing suppliers in the market by undercutting the market price. The supplier prices their goods much cheaper than the market rate in order to drive out competitors, even if it means incurring short-term losses. After the market has been cleared, the dominant supplier then increases prices [above competitive market rates](https://en.wikipedia.org/wiki/Supracompetitive_pricing), as competition has been extinguished. -Since each VRF is compared against this threshold (which is between 0 and 1), and each VRF is a pseudo-random number between 0 and 1, the VRF threshold for a given account is essentially the probability that a slot will be won by that account. Thus, we can extrapolate this function for computing VRF thresholds to compute the mean number of blocks we can expect a given account to win within an epoch. This can be done merely by summing all the probabilities for winning each slot of an epoch, and since the probability for an entire epoch is fixed, the mathematical expression to compute this simplifies to φ(α) × 7140. +However, this strategy is effective only in markets where there are high barriers to entry. Meaning, competitors who were crowded out in the predation stage are unable to rejoin the market. -As an example, let's say there is an account on the network which has 10^6 (1 million) mina tokens in the staking ledger epoch ep. This same staking ledger for epoch ep has a total supply of 8×10^8 (800 million) mina tokens. We can compute the odds that this account will win an individual slot in ep using φ(10^6 ÷ 8×10^8) = 0.0017313674. This gives us a ~0.17% chance that this account will win each slot during epoch ep. We can then compute the mean number of blocks we expect this account to produce for this epoch by multiplying the result by 7140, giving us a probabilistic mean of ~12.36 blocks for the epoch. +This is not the case for SNARK work because the barriers to entry are low. Anyone who has spare compute can join the snarketplace and produce as little as one SNARK work and profit on that unit of work. The only barrier to entry is the initial capital expense on hardware, but hardware requirements are low so that users with spare equipment can come online and participate. If any SNARK worker succeeds in driving out the market and increases prices, newcomers are anticipated to reappear and drive prices back down. -Another way to calculate the probability is 1 - 0.25^(10^6 ÷ 8×10^8) which gives the same result as above. +### Does speed of producing SNARKs matter? If my computer is slower, will I be at a disadvantage? -## Sending Many Transactions +No, provided that the SNARK work produced is still required by block producers, it doesn't matter who produced it first — only the price matters to block producers. The caveat here is that earlier inclusion into the SNARK mempool is obviously beneficial, as block producers are likely to "see" the work earlier. -It is important to pay out rewards to participants who have delegated their stake to your staking service. +However, you can envision a scenario where a set of SNARK workers are favored because they produced the most number of SNARK works that are profitable, and buying proofs from as few entities as possible would allow for more transactions to be included in any block. -To learn how, see [Sending Many Transactions](/mina-protocol/sending-a-payment#sending-many-transactions). +There is also a threshold at which time becomes a factor, but this scenario applies only to very underpowered devices. ---- -url: /node-operators/data-and-history ---- +### Will a full node need to store all intermediate SNARK proofs? Will the storage requirements grow linearly with blocks? -# About Data and History +No, when a new block is generated, Mina computes the proof recursively over the prior proof and the new block. This is the advantage of recursive composition -- at any given time, nodes need to store only the most recent proof. Intermediate proofs are not needed. For historical clarity on how this architecture emerged, see the [Using zkSNARKs to create a blockless blockchain](https://www.youtube.com/watch?v=eWVGATxEB6M) talk. -Mina is a succinct blockchain, and as a result, consensus nodes store only the recent history of the chain before discarding it (the last k blocks, currently 290). You can access recent history by using graphQL from a running mina daemon. +### How do you control or limit the number of threads that SNARK workers use? -While prior transaction history is not required to prove the current state is valid (this is handled by using a recursive zero-knowledge proof), many applications need to access prior transaction history. Examples include block explorers and wallets. +When you start the mina daemon, use the `-snark-worker-parallelism` flag. This flag is equivalent to setting `OMP_NUM_THREADS`, but doesn't affect block production. -You can optionally run an [archive node](/node-operators/archive-node/getting-started) to access historic data. The archive node process stores a summary of each block in a Postgres database. The archive node requires a running mina-archive daemon that connects to a running mina daemon and a [PostgreSQL](https://www.postgresql.org/) database. +## Exchange Integration -## In This Section +#### Where can I find third-party audit reports for Mina? -- [Rosetta API](/node-operators/data-and-history/rosetta) - A standardized API for blockchain integration, exchange support, and historical data queries. +The latest third-party audit reports are publicly available here: -For querying data from a running node via GraphQL, see [Querying Data](/node-operators/validator-node/querying-data). +- [https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review](https://research.nccgroup.com/2020/05/13/public-report-coda-cryptographic-review/) +- [https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs](https://leastauthority.com/blog/audit-of-mina-ledger-application-for-o1-labs/) +- [https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review](https://research.nccgroup.com/2022/02/22/public-report-o1-labs-mina-client-sdk-signature-library-and-base-components-cryptography-and-implementation-review) ---- -url: /node-operators/data-and-history/rosetta ---- - -# Rosetta API - - - - -A standardized API for blockchain integration — query historical data, build transactions, and integrate with exchanges. - +:::note -[Rosetta API](https://www.rosetta-api.org/) (rebranded as [Mesh](https://docs.cdp.coinbase.com/mesh/docs/welcome/) by Coinbase) is an open-source specification and set of tools that make deploying and interacting with blockchains quick and easy. Mina implements a subset of the Rosetta specification — not all endpoints defined in the spec are available. Mina's Rosetta implementation is primarily used by exchanges to integrate MINA deposits, withdrawals, and balance queries. +Any news and updates related to exchange listing shared by the Mina Foundation are on [www.minaprotocol.com](https://minaprotocol.com) or the official [Mina Protocol](https://x.com/MinaProtocol) X (Twitter). Mina Foundation cannot individually answer any listing questions. -:::note -The Rosetta API is auxiliary to Mina's existing [GraphQL API](/node-operators/reference/mina-cli-reference) and [Archive Node](/node-operators/archive-node). While GraphQL provides access to current network state, historical and persistence data requires the Archive database. Rosetta bundles both data sources behind a standardized interface and exists primarily to satisfy exchange integration requirements. ::: -## Architecture - -The Rosetta stack consists of four components that work together: +#### Why do you recommend using Rosetta for integrating Mina to our exchange? -| Component | Default Port | Description | -|---|---|---| -| **Mina Daemon** | 8302 (P2P), 3085 (GraphQL) | Syncs with the network, produces/validates blocks | -| **Archive Node** | 3086 | Stores historical block data in PostgreSQL | -| **PostgreSQL** | 5432 | Database backend for the archive node | -| **Rosetta API** | 3087 (online), 3088 (offline) | Translates Mina data into the Rosetta specification | +Rosetta is an open-source specification that helps exchanges and developers integrate blockchains. Since Rosetta is actively maintained and specifically designed to enable simpler, faster, and more reliable blockchain integrations, we highly recommend using Rosetta to integrate Mina blockchain with your exchange. -All ports listed above are defaults and can be overridden via configuration. When using Docker, these are the ports inside the container — map them to your desired host ports with `-p`. +#### Is there an account creation fee? -## Installation +Yes, Mina Protocol charges a fee of 1 MINA when you create a new account. This fee helps protect the network from denial of service-type attacks. Over time, this fee can change. -There are three ways to run Rosetta, depending on your needs. +#### What is the maximum size of the mempool? How do we work around this? -### Option 1: All-in-One Docker Image (Recommended for getting started) +The max mempool size is 3,000. After it hits that size, transactions with the lowest fees are discarded. -The all-in-one image bundles the daemon, archive node, PostgreSQL, and Rosetta API into a single container. It automatically initializes the archive database from public o1Labs backups. +Set your fee to an amount higher than 0.001 MINA, the current average fee for transactions in the pool. -**Requirements:** Docker with at least 12 GB RAM allocated (16 GB recommended). +You can view the fees for pending transactions and adjust your fees accordingly: [https://minascan.io/mainnet/txs/pending-txs](https://minascan.io/mainnet/txs/pending-txs). -#### Mainnet +#### Why do some users appear to have lost their funds when sending to exchanges? -```bash -docker run -it --rm --name rosetta \ - --entrypoint=./docker-start.sh \ - -p 8302:8302 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ - minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet -``` +:::tip -#### Devnet +While Mina and its SDKs do support the memo field when sending a transaction, the recommended best practice is do NOT require a memo for deposits. -```bash -docker run -it --rm --name rosetta \ - --entrypoint=./docker-start.sh \ - -p 8302:8302 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ - -e MINA_NETWORK=devnet \ - -e PEER_LIST_URL=https://bootnodes.minaprotocol.com/networks/devnet.txt \ - minaprotocol/mina-rosetta:3.2.0-97ad487-bookworm-devnet -``` +::: -Initial sync typically takes between 20 minutes and 1 hour depending on your hardware and network connection. You can check sync status with: +To associate the deposit with the user's account, some exchanges require their users to include a unique memo field when sending MINA deposits to the exchange's address. -```bash -docker exec rosetta mina client status -``` +If the user does not include this unique memo when sending their deposit, the receiving exchange may not be able to associate the deposit properly with the user's exchange account. -#### Environment Variables +These funds are NOT lost. The exchanges have received the funds at the exchange's address, but the exchange may not be able to automatically associate the deposit with the user's exchange account without such a memo. -The all-in-one image supports the following environment variables for customization: +To prevent this issue, we recommend that exchanges do NOT require a memo for deposits. At the same time, exchanges and wallet creators are recommended to expose an optional memo field during a Mina send transaction. -| Variable | Default | Description | -|---|---|---| -| `MINA_NETWORK` | `mainnet` | Network to connect to (`mainnet` or `devnet`) | -| `PEER_LIST_URL` | Network-specific seed list | URL for the peer list | -| `LOG_LEVEL` | `Debug` | Log level (`Info`, `Debug`, `Warn`, `Error`) | -| `MINA_GRAPHQL_PORT` | `3085` | GraphQL API port | -| `MINA_ARCHIVE_PORT` | `3086` | Archive node port | -| `MINA_ROSETTA_ONLINE_PORT` | `3087` | Rosetta online API port | -| `MINA_ROSETTA_OFFLINE_PORT` | `3088` | Rosetta offline API port | -| `POSTGRES_USERNAME` | `pguser` | PostgreSQL username | -| `POSTGRES_DBNAME` | `archive` | PostgreSQL database name | -| `POSTGRES_DATA_DIR` | `/data/postgresql` | PostgreSQL data directory | -| `MINA_ARCHIVE_DUMP_URL` | `https://storage.googleapis.com/mina-archive-dumps` | Base URL for archive database dumps | -| `MINA_CONFIG_DIR` | `/data/.mina-config` | Mina configuration directory | +#### What is the maximum number of rollback blocks? -### Option 2: Docker Compose (Recommended for production) +The table in [Lifecycle of a Payment](/mina-protocol/lifecycle-of-a-payment) describes how many blocks you wait for a transaction to be confirmed. -For production deployments, use Docker Compose to run each component as a separate container. This gives you more control over resource allocation, logging, and restarts. +#### How should I calculate transaction fees? -For production deployments, a complete Docker Compose configuration with all four services (PostgreSQL, archive node, Mina daemon, and Rosetta API) is available in the [Rosetta Docker Compose guide](/exchange-operators/rosetta/docker-compose). +To calculate your transaction fees, use [https://fees.mina.tools](https://fees.mina.tools/). -### Option 3: Debian Packages (Manual setup) +#### My Mina node gets stuck sometimes. How can I detect this and fix it? -You can install each component individually via Debian packages. This is useful if you already run a Mina daemon and archive node and only need to add the Rosetta API. +This is a known issue for some earlier releases. Restart your mina node whenever this issue is detected. -```bash -# Add the Mina repository (if not already configured) -echo "deb [trusted=yes] http://packages.o1test.net/ noble stable" | sudo tee /etc/apt/sources.list.d/mina.list -sudo apt-get update +You can use the following script to run a cron job every 3 minutes (the slot length) or more frequently: -# Install the Rosetta package for your network -sudo apt-get install mina-rosetta-mainnet -# or: sudo apt-get install mina-rosetta-devnet ``` +MINA_STATUS=$($MINA client status --json) +HIGHESTBLOCK="$(echo $MINA_STATUS | jq .highest_block_length_received)" +HIGHESTUNVALIDATEDBLOCK="$(echo $MINA_STATUS | jq .highest_unvalidated_block_length_received)" -Then run the Rosetta API pointing to your existing daemon and archive database: +# Calculate difference between validated and unvalidated blocks. +# If block height is more than 4 block behind, something is likely wrong. -```bash -mina-rosetta \ - --archive-uri postgres://:@:/ \ - --graphql-uri http://localhost:3085/graphql \ - --log-json \ - --port 3087 +DELTAVALIDATED="$(($HIGHESTUNVALIDATEDBLOCK-$HIGHESTBLOCK))" +if [[ "$DELTAVALIDATED" -gt 4 ]]; then + $MINA client stop +fi ``` -This requires a running [Mina daemon](/node-operators/block-producer-node) and [archive node](/node-operators/archive-node) with a populated PostgreSQL database. - -## Offline Mode - -The Rosetta Construction API requires an "offline" endpoint that can build and sign transactions without network access. Use the standalone entrypoint for this: - -```bash -docker run -it --rm --name rosetta-offline \ - --entrypoint=./docker-standalone-start.sh \ - -p 3088:3088 \ - minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet -``` +:::tip -This starts only the Rosetta API process — no daemon, archive, or database. Point your Construction API calls to this endpoint for offline operations. +Be sure your Mina daemon is monitored by something such as systemd, so it can auto-restart. -## Demo Mode +::: -For development and testing, the demo mode launches a local sandbox with a simple genesis ledger and all components running inside a single container: +#### My archive node is missing block information after a restart. How can I recover the data? -```bash -docker run -it --rm --name rosetta-demo \ - --entrypoint=./docker-demo-start.sh \ - -p 3085:3085 -p 3087:3087 \ - minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet -``` +Archive node operators often choose to run redundant archive nodes to store block data to one or more locations of their choice (for example, PostgreSQL, GCP, local files, or a logging service) and to backfill any missed block data if needed. -This creates an isolated network with no external connectivity — useful for developing integrations before connecting to mainnet or devnet. +For convenience, [mina_network_block_data](https://console.cloud.google.com/storage/browser/mina_network_block_data) from the archive node is available to help others in the community backfill any missing information. -## Available Images +This bucket contains blocks from various Mina networks — for example, Mainnet and the most recent Devnet `devnet2`. Filter by filename for the network you want. Note that this bucket contains blocks for various other networks too, such as QAnet, which is not recommended for your testing. QAnet is used by o1Labs during targeted iterative development. -| Network | Image | Notes | -|---|---|---| -| **Mainnet (noble)** | `minaprotocol/mina-rosetta:3.3.1-7b34378-noble-mainnet` | amd64 | -| **Mainnet (bookworm)** | `minaprotocol/mina-rosetta:3.3.1-7b34378-bookworm-mainnet` | Also available for arm64 | -| **Devnet (bookworm)** | `minaprotocol/mina-rosetta:3.2.0-97ad487-bookworm-devnet` | Latest devnet | +Filenames contain the network name, block height, and state hash of the block. Blocks older than height 25,705 include only the network name and state hash in the filename. -Images are published on [Docker Hub](https://hub.docker.com/r/minaprotocol/mina-rosetta/tags). +Example filenames: -## Verifying the API +(Recent) -Once your node is synced, verify the Rosetta API is working. +``` +mainnet-30627-3NLfKanQ53X2MRKx5ZRvb9nVCEB9eJpcnssGCTpT3J1cojhB5M19.json +``` -**List available networks:** +(Older) -```bash -curl -s http://localhost:3087/network/list \ - -H 'Content-Type: application/json' \ - -d '{"metadata":{}}' | jq . +``` +mainnet-3NKUBmkc7UZ7ik5JyCM4WNzkN1HG5heMB5zNDUkf3Kgta1MFY6LY.json ``` -**Get network status:** +You can download a specific block using curl: -```bash -curl -s http://localhost:3087/network/status \ - -H 'Content-Type: application/json' \ - -d '{"network_identifier": {"blockchain": "mina", "network": "mainnet"}}' | jq . +``` +curl https://mina_network_block_data.storage.googleapis.com/ ``` -**Query an account balance:** +You can import this file using the mina archive blocks tool. The command for it is: -```bash -curl -s http://localhost:3087/account/balance \ - -H 'Content-Type: application/json' \ - -d '{ - "network_identifier": {"blockchain": "mina", "network": "mainnet"}, - "account_identifier": {"address": ""} - }' | jq . +``` +mina-archive-blocks --precomputed --archive-uri FILE. ``` -## Building Your Own Image +#### How do I query for the canonical block at a certain height from the archive node? -For most users, the official images are sufficient. If you need to build a custom image, see the [Rosetta README](https://github.com/MinaProtocol/mina/blob/master/src/app/rosetta/README.md) in the Mina repository for build instructions. +Use a recursive query. See [Query the database](/node-operators/archive-node/getting-started#query-the-database) examples in the Archive Node docs. -## Questions and Support +#### Why am I getting this error message: "Not able to connect to the network"? -- Post questions in [Mina GitHub Discussions](https://github.com/MinaProtocol/mina/discussions) -- Report bugs on [GitHub Issues](https://github.com/MinaProtocol/mina/issues) -- Join the [Mina Discord](https://discord.gg/minaprotocol) for community support +This error message usually occurs due to a chain ID mismatch from running a Devnet build on Mainnet, or vice versa. ---- -url: /node-operators/delegation-program/foundation-delegation-program ---- +To check whether you are running a devnet or mainnet build, run `Mina client status`. Next, compare the output's chain ID of your node to the expected chain ID of the network you are trying to connect to. You can find required information for comparison within the [GitHub announcements](https://github.com/MinaProtocol/mina/discussions/categories/announcements) or [Discord](https://discord.com/channels/484437221055922177/601171209287368715) server. -# Mina Foundation Delegation Program +#### Are there any official broadcast nodes that can be used? +No, there are no official broadcast nodes at this time. However, you can broadcast transactions using [https://minascan.io/mainnet/broadcast/payment](https://minascan.io/mainnet/broadcast/payment). Use this method as a backup, the recommended method is to broadcast transactions yourself. +#### Should I be staking my funds? - -How to Participate in the Mina Foundation Delegation Program - +Since Mina is a Proof of Stake (PoS) consensus network without lockup for staked tokens, it is recommended to stake these funds to support the quality of the Mina network. Additionally, by not staking, you are missing out on staking rewards that you can otherwise be receiving from the Mina blockchain. -Learn how to receive a delegation from Mina Foundation. +You can look into staking this wallet, either by running your own block production node or just by delegating your funds to a staking pool on the network. Delegating to a staking pool is simpler to set up. -:::info +:::note -**PLEASE READ:** This affects the submission of uptime data and the Performance Score results for receiving a delegation. +Newly staked accounts incur a delay of 18 to 29 days before you start receiving rewards. -The sidecar tracking system was **discontinued on June 14th 2024**. Please use the **SNARK-work-based** uptime tracking system. +::: -- [Instructions for the SNARK-work-based uptime system](./uptime-tracking-system) +#### Why is there a delay for staking to take effect? -Please follow the latest updates and post questions in the [#delegation-program](https://discord.com/channels/484437221055922177/808895957978447882) channel on Mina Protocol Discord. +For purposes of ensuring consensus, there is a delay between when delegations are sent on the blockchain and when they take effect with respect to staking on the network. The staking ledger always operates between 18 to 29 days behind the live ledger. -::: +#### How long is the delay and when is the next staking snapshot? -This Delegation Program is an implementation of the Mina Foundation Delegation Policy, the official policy on the delegations of MINA tokens from the Mina Foundation (referred to as “Foundation” below). +The timing of the next staking snapshot varies. -If you are a block producer looking to participate in the Mina Foundation Delegation Program, this page explains how the program works and how to send uptime data and pay back rewards. +Since the timing is based on a combination of consensus timing (epochs) and snarketplace throughput, it is difficult to determine exactly how long this delay can be. -:::note +A conservative estimate is that delegations sent 3 days before the epoch transition can take effect in the upcoming epoch. This means that, for any given delegation, there is an average of 18 to 29 days delay before this delegation updates block production. -There are two delegation programs in Mina's ecosystem — one from the Mina Foundation and [one from o1Labs](https://www.o1labs.org/blog/delegation-policy-2024#o1labs-delegation-program-policy), Mina's ecosystem partner. This page details the program only for the Mina Foundation. +You can use this Delegation Calculator tool built by Carbonara to see the next staking ledger cutoff: +[https://epoch.mina.tools](https://epoch.mina.tools/). -::: +#### What is the best way to test tooling and integration with Mina? -## Overview +Test tooling and integrations on Devnet before going live on Mainnet. The Devnet network is dedicated for developers building on top of the Mina protocol and is designed for testing and experimentation. Be sure to simulate expected Mainnet conditions, such as transaction volume and frequency, to help identify and solve potential issues ahead of time. -Mina Foundation is committed to decentralizing the governance of the Mina protocol. In order to delegate their voting power, Mina Foundation delegates its tokens to community members through the [Mina Foundation Delegation Program](https://minaprotocol.com/blog/mina-foundation-delegation-policy). +See [Connect to Devnet](/node-operators/validator-node/connecting-to-the-network). -In the Delegation Program, Mina Foundation delegates its tokens to a number of validators. As the validators earn staking rewards associated with the delegation, they must return the remaining rewards to Mina Foundation, **but can keep 8% of the staking rewards**. +--- +url: /node-operators +--- -Block producers are selected on a quarterly basis, known as a delegation cycle, based on their Performance Scores (see below) and requirements outlined in the [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy). +# Introduction -## Eligibility Requirements +The [Mina Protocol](/mina-protocol) is a layer 1 blockchain that is secured by [proof of stake consensus](/mina-protocol/proof-of-stake). -### How do I participate in the program? +Node operators are people who run Mina nodes. Mina node operators participate in consensus to create new blocks and help compress data by generating zk-SNARKs. -If you are new to the Mina Foundation Delegation Program, please make sure you have fulfilled the following requirements in order to be eligible for receiving a delegation from the Foundation's token allocation. +## Mina Node Roles -#### Step 1: Complete the application +A node is a machine running the [Mina daemon](/glossary#daemon). Different nodes fulfill different roles within the Mina network: -Review the [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy) then complete [the application form](https://docs.google.com/forms/d/e/1FAIpQLSduM5EIpwZtf5ohkVepKzs3q0v0--FDEaDfbP2VD4V6GcBepA/viewform). You will be asked to share your public key for receiving a delegation in the application. +1. Validator - A plain Mina daemon node that participates in network consensus and can be used for operations such as sending and signing payments. -#### Step 2: Run the Uptime Tracking System +2. [Block Producer](/glossary#block-producer) - A node that participates in a process to determine what blocks it is allowed to produce and then produces blocks containing transactions that can be broadcast to the network. People who run [block producer](https://docs.minaprotocol.com/mina-protocol/block-producers) nodes are also called block producers. -The uptime tracking system sends recent blockchain data from your block producer node's perspective to a service. This service verifies whether this recent data is synced with the chain. If it is synced, it means your node is online. +3. [SNARK Coordinators](/glossary#snark-coordinator) - A role on a Mina node in the Mina network that distributes work to a series of SNARK workers in parallel to block production. -Instructions for how to run the uptime tracking system: -- [SNARK-based Uptime System](./uptime-tracking-system) +4. [SNARK Workers](https://docs.minaprotocol.com/glossary#snark-worker) - SNARK workers create [zk-SNARKs](https://minaprotocol.com/blog/what-are-zk-snarks) for each transaction. These zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of a block, and in turn, these zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of the network. These zk-SNARKs help provide the Mina Protocol with succinctness. -#### Step 3: Pass the KYC/AML requirement (only if selected for delegation) +5. [Archive Nodes](/node-operators/archive-node) - A regular mina daemon that is connected to a running `mina-archive` process. The daemon regularly sends blockchain data to the archive process that stores it in a [PostgreSQL](https://www.postgresql.org/) database. -In order to receive a delegation, you must also meet the KYC/AML requirements of Mina Foundation. +6. [Seed Nodes](https://docs.minaprotocol.com/glossary#seed-nodes) - Keep a record of nodes in the network and enable nodes joining the network to connect to peer nodes. -If you have recently been selected as a validator who will be receiving a delegation, but have not yet completed any of the KYC/AML requirements, please look out for an email with further instructions. +## Operating a Node -For more details on this requirement, please review the policies outlined in the [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy). +Node operators are network participants who run Mina nodes on a Mina network. -:::note +This section describes how to run each node role and where to find the operational references. -You can only complete this step if you’re receiving a delegation for the next cycle. When you are selected for a delegation, you will be contacted with instructions on how to proceed with completing the KYC/AML requirement. -::: +### Validators +- [Requirements](/node-operators/validator-node/requirements) - Hardware, software, and network requirements +- [Generating a Key Pair](/node-operators/validator-node/generating-a-keypair) - How to generate key pairs +- [Querying Data](/node-operators/validator-node/querying-data) - Query blockchain data via GraphQL +- [Staking and Snarking](/node-operators/validator-node/staking-and-snarking) - How to stake your MINA +- [Logging](/node-operators/validator-node/logging) - Log files, levels, and export -## Program Guidelines +### Specialized Node Types +- [Block Producers](/node-operators/block-producer-node) - Running a Block Producer +- [SNARK Coordinator & Workers](/node-operators/snark-workers) - Running SNARK coordinator and worker roles +- [Archive Nodes](/node-operators/archive-node) - Running an Archive Node +- [Seed Node](/node-operators/seed-peers) - Running a Seed Node -Now that you understand how to become eligible to participate in the program, please review the following guidelines to better understand how to receive a delegation and send back rewards. +### Data and Programs +- [Data and History](/node-operators/data-and-history) - Rosetta API and blockchain integration +- [Delegation Program](/node-operators/delegation-program) - The Mina Foundation Delegation Program for Block Producers -### How do I get selected for a delegation? -#### Measuring Uptime Performance Score +### Reference +- [Mina CLI Reference](/node-operators/reference/mina-cli-reference) - Guide to CLI interactions with Mina networks +- [Mina Signer](/node-operators/mina-signer) - Key generation and transaction signing -The **Uptime Performance Score** is an important factor for selecting the public keys to receive a Mina Foundation Delegation Program delegation from a token provider. +### Support +- [Troubleshooting](/node-operators/troubleshooting) - Solutions to common problems +- [FAQ](/node-operators/faq) - Frequently asked questions -**Uptime** is the measurement of when your block producer node is online. In the Mina Foundation Delegation Program, uptime is measured in 20-minute windows. If your block producer node is online at any time within a given 20-minute window, you will be marked as online for that period. +## Exchange Integration -:::note +[Exchange Operators](/node-operators/exchange-operators) describes how exchanges can integrate with the Mina blockchain using Rosetta API, archive nodes, and validator nodes. -You may run more than one node with the same block producer key if you want to increase your chances of remaining online. -::: +--- +url: /node-operators/mina-signer +--- -:::tip +# Mina Signer -To participate in the delegation program, your block producer node must stay online and connected to the network at all times. If you cannot maintain uptime, consider [delegating your stake](/node-operators/validator-node/staking-and-snarking) instead. +[Mina Signer](/mina-signer) is a NodeJS/Browser-compatible library for signing transactions and generating keys. The Rosetta stack also ships an offline signer CLI tool. Both can be used to sign transactions offline for later submission via the [Construction API](/node-operators/rosetta/samples/send-transactions). -::: +## Migration from o1labs/client-sdk -**Uptime Performance** is the percentage over a time period in which a block producer node is online and operational. +The signing library `o1labs/client-sdk` is deprecated and will stop working after the Mina mainnet upgrade. All users should migrate to [mina-signer](https://www.npmjs.com/package/mina-signer). -The time period of Mina Foundation's delegation programs is 90 days. +When migrating: -This percentage is based on all possible 20-minute windows over the last time period. +1. Adjust the `nonce` to the correct nonce on the sender account +2. Update the `url` variable with an existing Mina Node GraphQL endpoint -### How do I track my performance score? +See [Broadcasting a Signed Payment](/mina-signer#broadcasting-a-signed-payment) for a complete example. -The **leaderboard** shows the ranking of block producers participating in the delegation program based on uptime performance. The sooner you fix issues with your nodes sending uptime data, the sooner you'll be rewarded with a better uptime and position on the leaderboard. +## Generating a key pair with mina-signer -Your position on this leaderboard is an important factor in your public key being selected to receive delegation from Mina Foundation. +See [Generating Keys](/mina-signer#generating-keys) for full details. A quick example: -See the following uptime leaderboards for the latest uptime performance scores: -- Official Leaderboard: https://uptime.minaprotocol.com -- Community Leaderboards: - - Minataur: https://minataur.net/uptime +```ts -### What happens after I receive a delegation? +const mina = new Client({network: 'testnet'}) +const keypair = mina.genKeys() +``` -#### Payout Procedure +## Generating a key pair with signer CLI -As per the Mina Foundation Delegation policy, there is a mandatory requirement on delegatees to use the latest version of the [payout script](https://github.com/jrwashburn/mina-pool-payout) to manage the return of rewards. +In a native build, the signer is at `_build/default/src/app/rosetta/ocaml-signer/signer.exe`. In Docker, it's at `/rosetta/app/mina-ocaml-signer`. Examples below use `signer` as an alias. -This payout script includes: -- Mapping of delegating wallet addresses with the correct return wallet address -- Calculation of the Reward amounts and paybacks of the remaining rewards to the delegators. -- Specific memo which includes the md5 hash of Pool’s Public Key (MD5 hash) +Generate a private key: -For more details and to review the script, see the [README](https://github.com/jrwashburn/mina-pool-payout/blob/main/README.md) file in the `mina-pool-payout` repository. +```bash +signer generate-private-key +``` -#### Payout Addresses +Derive the public key and account address: -You must return rewards to the address specified in the [Mina Delegation Program Return Addresses](https://docs.google.com/spreadsheets/d/1Fm4XSS9Xu4eWAhpM06sdySUKvLClR5SculXfP5o5sSc/edit?usp=sharing) mapping document. This will be covered when running the script in managing reward payback. +```bash +signer derive-public-key --private-key +``` -#### Payout Frequency +## Signing a transaction with mina-signer -Rewards must be distributed at least once for a given epoch. You must send one payment in the amount of your obligation to the correct address specified in the [Mina Delegation Program Return Addresses](https://docs.google.com/spreadsheets/d/1Fm4XSS9Xu4eWAhpM06sdySUKvLClR5SculXfP5o5sSc/edit?usp=sharing) mapping document and if applicable, send the correct amount to the burn address. Both payments should have a memo field with the md5 hash value of your block producer public key. This is the easiest method to avoid confusion in tracking payments and will reduce the likelihood you will be incorrectly flagged as delinquent. +```ts +mina.signRosettaTransaction(payload, privateKey) +``` -All the rewards for epoch N must be delivered (ie. accepted in a block, not just sent) no later than slot number 3,500 of the next epoch. This gives you about a week to sort out these payments. +:::note +We recommend using `mina.rosettaCombinePayload` to sign and prepare a payload for the `/construction/combine` request. See [Sending Transactions](/node-operators/rosetta/samples/send-transactions) for the full flow. +::: +## Signing a transaction with signer CLI -### How do I calculate the reward payout? +```bash +signer sign --private-key --unsigned-transaction +``` -For reference, this explanation describes how reward returns are calculated. +Replace `` with the unsigned transaction string from the `/construction/payloads` endpoint. See [Sending Transactions](/node-operators/rosetta/samples/send-transactions) for the full flow. -For details, see the [implementation](https://github.com/jrwashburn/mina-pool-payout/blob/main/src/core/payoutCalculator/PayoutCalculatorIsolateSuperCharge.ts) code. +--- +url: /node-operators/reference +--- +# Reference -#### Reward Specifications +- **[Mina CLI Reference](/node-operators/reference/mina-cli-reference)** -- Complete reference for the `mina` command line interface. +- **[Mina Signer](/node-operators/mina-signer)** -- Key generation and transaction signing for node operators. +- **[Troubleshooting](/node-operators/troubleshooting)** -- Solutions to common problems when running a Mina node. +- **[FAQ](/node-operators/faq)** -- Frequently asked questions about node operations, SNARKs, and the Mina network. -You must send back at least the amount specified by this mechanism. +--- +url: /node-operators/reference/mina-cli-reference +--- -At the end of each epoch, do all of the following: +# Mina CLI Reference -1. Compute the total stake delegated to your account for the epoch +The Mina CLI (Command Line Interface) is the primary way for users to interact with the Mina network. It provides standard client functionality to create accounts, send transactions, and participate in consensus. There are also advanced client and daemon commands for power users. -2. Compute the share of stake from the token provider (from both accounts) by dividing the token provider delegation by the total stake. (i.e. `provider_share = provider_delegation / total_stake`). The resulting share should be between 0 and 1. +The Mina CLI is installed when you [install Mina](/node-operators/validator-node/installing-on-ubuntu-and-debian). -3. For each block produced that has a non-zero block-reward on the canonical chain rewards must be calculated based on 720 MINA. +:::tip -4. Calculate the Mina Foundation payout by multiplying the coinbase reward (equal to `720 MINA` ) by the provider share calculated in the previous step minus an 8% percent fee. (i.e. `payout = (provider_share * 0.92) * 720)`. +Mina APIs are always improving. See `mina help` for the most up-to-date version. -5. Send a transaction to the token provider accounts with the appropriate payout and memo - please follow the rules in the "Payout Attribution" section with your transaction. - More details in the following source code parts: [PayoutCalculatorIsolateSuperCharge.ts](https://github.com/jrwashburn/mina-pool-payout/blob/7f00dbd9e693f76ea6a950c29862120a170625a9/src/core/payoutCalculator/PayoutCalculatorIsolateSuperCharge.ts#L126) and [ConfigurationManager.ts](https://github.com/jrwashburn/mina-pool-payout/blob/7f00dbd9e693f76ea6a950c29862120a170625a9/src/configuration/ConfigurationManager.ts#L21C15-L21C15). +::: +## mina +``` +Mina -The block producer can keep all of the transaction fees or divide them equally amongst the other members of the pool. + mina SUBCOMMAND -:::tip +=== subcommands === -The canonical chain will be calculated as 12 blocks behind any tip at slot 3,500 of the next epoch. -::: + accounts Client commands concerning account management + daemon Mina daemon + client Lightweight client commands + advanced Advanced client commands + ledger Ledger commands + libp2p Libp2p commands + internal Internal commands + parallel-worker internal use only + transaction-snark-profiler transaction snark profiler + version print version information + help explain a given subcommand (perhaps recursively) -#### Example of a Reward Payout Calculation +``` -Consider the following: +## mina accounts +``` +Client commands concerning account management -- Account A has 2 million MINA -- Account B is controlled by the Mina Foundation and has 6 million MINA which are delegated to Account A via the Delegation Program. -- Account C is controlled by a third party and has 2 million MINA. This account also delegates to Account A and has also agreed to Account A retaining 8% of the staking rewards. + mina accounts SUBCOMMAND -In this example, the total amount of stake for Account A is 10 million calculated by adding up the balances from all the accounts. +=== subcommands === -Now let's consider Epoch 5. The share of the stake from the Foundation is `6 million MINA / 10 million MINA = 0.6` or 60%. The share of the stake from Account B is `2 million MINA / 10 million MINA = 0.2` or 20%. + list List all owned accounts + create Create new account + import Import a password protected private key to be tracked by the daemon. + Set MINA_PRIVKEY_PASS environment variable to use non-interactively + (key will be imported using the same password). + export Export a tracked account so that it can be saved or transferred + between machines. + Set MINA_PRIVKEY_PASS environment variable to use non-interactively + (key will be exported using the same password). + unlock Unlock a tracked account + lock Lock a tracked account + help explain a given subcommand (perhaps recursively) -3 blocks are produced in this epoch that end up on the canonical chain. The blocks were won by Account A. +``` -1. Account A retains, 0.2 x 720 MINA x 3 blocks = 432 MINA. -2. Mina Foundation, Account B, payout would be: (0.6 x 0.92) x 720 MINA x 3 blocks = 1,192.32 MINA. -3. Account A retains 8%, (0.6 x 0.08) x 720 MINA x 3 blocks = 103.68 MINA. -3. Account C payout would be: (0.2 x 0.92) x 720 MINA x 3 blocks = 397.44 MINA. -4. Account A retains 8%, (0.2 x 0.08) x 720 MINA x 3 blocks = 34.56 MINA +### mina accounts list +``` +List all owned accounts + mina accounts list -### Relevant Links +=== flags === -- [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy) -- [SNARK-based Uptime System](/node-operators/delegation-program/uptime-tracking-system) + [--config-directory DIR] Configuration directory + (alias: -config-directory) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -#### Need any help? +``` -Post your questions in the [#delegation-program](https://discord.com/channels/484437221055922177/808895957978447882) channel on the Mina Protocol Discord. +### mina accounts create +``` +Create new account ---- -url: /node-operators/delegation-program ---- + mina accounts create -# Delegation Program +=== flags === -The Mina Foundation Delegation Program enables block producers to receive token delegations from the Mina Foundation and earn staking rewards. + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -:::info +``` -The sidecar tracking system was **discontinued on June 14th 2024**. Please use the **SNARK-work-based** uptime tracking system. See [Uptime Tracking System](/node-operators/delegation-program/uptime-tracking-system) for setup instructions. +### mina accounts import +``` +Import a password protected private key to be tracked by the daemon. +Set MINA_PRIVKEY_PASS environment variable to use non-interactively (key will be imported using the same password). -::: + mina accounts import -- **[Foundation Delegation Program](/node-operators/delegation-program/foundation-delegation-program)** -- Eligibility requirements, performance scoring, payout procedures, and reward calculations. -- **[Uptime Tracking System](/node-operators/delegation-program/uptime-tracking-system)** -- Instructions for the SNARK-based uptime tracking system used to measure node uptime. +=== flags === -## Related + --privkey-path FILE File to read private key from + (alias: -privkey-path) + [--config-directory DIR] Configuration directory + (alias: -config-directory) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -- [Staking and Snarking](/node-operators/validator-node/staking-and-snarking) -- How staking and delegation work on Mina -- [Block Producers](/node-operators/block-producer-node) -- Getting started with block production +``` ---- -url: /node-operators/delegation-program/uptime-tracking-system ---- +### mina accounts export +``` +Export a tracked account so that it can be saved or transferred between machines. + Set MINA_PRIVKEY_PASS environment variable to use non-interactively (key will be exported using the same password). -# Uptime Tracking System + mina accounts export +=== flags === + --privkey-path FILE File to write private key into (public key + will be FILE.pub) + (alias: -privkey-path) + --public-key PUBLICKEY Public key of account to be exported + (alias: -public-key) + [--config-directory DIR] Configuration directory + (alias: -config-directory) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) - -Instructions for the SNARK-based uptime tracking system. - +``` -Learn how to set up the uptime tracking system for the Mina Foundation Delegation Program +### mina accounts unlock +``` +Unlock a tracked account -:::info - -**PLEASE READ:** This affects the submission of uptime data and the Performance Score results for receiving a delegation. + mina accounts unlock +=== flags === -The sidecar tracking system was **discontinued on June 14th 2024**. Please use the **SNARK-work-based** uptime tracking system. + --public-key PUBLICKEY Public key to be unlocked + (alias: -public-key) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -- [Instructions for the SNARK-work-based uptime system](./uptime-tracking-system) +``` -Please follow the latest updates and post questions in the [#delegation-program](https://discord.com/channels/484437221055922177/808895957978447882) channel on Mina Protocol Discord. +### mina accounts lock +``` +Lock a tracked account -::: + mina accounts lock -In order to maintain eligibility for various grants and the Mina Foundation Delegation Program, you must connect to our uptime tracking endpoint to run the SNARK-work-based uptime tracking system with your daemon to report node uptime. +=== flags === -If you are required to keep your node online for a grant or specific program, you must run a small uptime tracking program that will report your daemon's uptime. This tutorial will walk you through the process of installing, configuring and running the uptime tracking system. + --public-key PUBLICKEY Public key of account to be locked + (alias: -public-key) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -:::note +``` -

We recommend that you save some of your SNARK work data logs. You can also share the logs with us if you’re interested in helping out with data checks.

+### mina accounts help +``` +explain a given subcommand (perhaps recursively) -::: + mina accounts help [SUBCOMMAND] -### Pre-requisites +=== flags === -Make sure you updated your node to at least release (3.0.0+): + [-expand-dots] expand subcommands in recursive help + [-flags] show flags as well in recursive help + [-recursive] show subcommands of subcommands, etc. + [-help] print this help text and exit + (alias: -?) -## How to set up the uptime system +``` -The SNARK-work-based uptime system is built into the mina daemon. The new uptime tracking system no longer requires importing your keypair and supports a new flag `--uptime-submitter-key`, which takes the path to your private key, just like `--block-producer-key`. +## mina client +``` +Lightweight client commands -To get started, pass in the following information to the daemon: + mina client SUBCOMMAND -- The path to the private key with the flag: -`--uptime-submitter-key ` +=== subcommands === -- The URL of our testing backend server with the flag: -`--uptime-url https://uptime-backend.minaprotocol.com/v1/submit` + get-balance Get balance associated with a public key + get-tokens Get all token IDs that a public key has accounts for + send-payment Send payment to an address + delegate-stake Delegate your stake to another public key + cancel-transaction Cancel a transaction -- this submits a replacement + transaction with a fee larger than the cancelled + transaction. + set-snark-worker Set key you wish to snark work with or disable snark + working + set-snark-work-fee Set fee reward for doing transaction snark work + export-logs Export daemon logs to tar archive + export-local-logs Export local logs (no daemon) to tar archive + stop-daemon Stop the daemon + status Get running daemon status + help explain a given subcommand (perhaps recursively) -- The password for the keypair associated with the given public key has the environment variable `UPTIME_PRIVKEY_PASS=`. If you are using a .mina-env file on Debian then this value should be on its own line, not included in `EXTRA_FLAGS=`. +``` -Here’s an example of what your .mina-env file should look like for Debian: +### mina client get-balance +``` +Get balance associated with a public key -~~~ -EXTRA_FLAGS="--block-producer-key /home/mina/my-wallet --uptime-submitter-key /home/mina/my-wallet --uptime-url https://uptime-backend.minaprotocol.com/v1/submit" -UPTIME_PRIVKEY_PASS= -MINA_PRIVKEY_PASS= -LOG_LEVEL=Info -FILE_LOG_LEVEL=Debug -~~~ + mina client get-balance +=== flags === -### Relevant Links + --public-key PUBLICKEY Public key for which you want to check the + balance + (alias: -public-key) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [--token TOKEN_ID] The token ID for the account + (alias: -token) + [-help] print this help text and exit + (alias: -?) -- [Latest Release Notes for Mina Protocol](https://github.com/MinaProtocol/mina/releases) -- [Mina Foundation Delegation Policy](https://minaprotocol.com/blog/mina-foundation-delegation-policy) -- [Mina Docs: Mina Foundation Delegation Program](/node-operators/delegation-program) +``` +### mina client get-tokens +``` +Get all token IDs that a public key has accounts for -#### Need any help? Post your questions in the [#Delegation-Program](https://discord.gg/ywDzwmGABT) channel on Discord. + mina client get-tokens ---- -url: /node-operators/downgrading-to-older-versions ---- +=== flags === -# Downgrading to Older Versions + --public-key PUBLICKEY Public key for which you want to find + accounts + (alias: -public-key) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -If you are running a Mina node on a version above 3.3.0 and need to roll back to 3.3.0 or below, you can convert the on-disk state in place using `mina-storage-converter`. This avoids a full rebootstrap from a remote S3 ledger bucket, which can save significant time. +``` -## When is downgrading needed? +### mina client send-payment +``` +Send payment to an address -Downgrading is typically necessary when a fork fails and there is a RocksDB version bump between the stop slot release and the pre-stop slot release. In such cases, the development team will notify node operators that a downgrade is required to continue operating on the correct chain. + mina client send-payment -## Debian/Ubuntu +=== flags === -### 1. Stop the Mina daemon + --amount VALUE Payment amount you want to send + (alias: -amount) + --receiver PUBLICKEY Public key to which you want to send money + (alias: -receiver) + --sender PUBLICKEY Public key from which you want to send the + transaction + (alias: -sender) + [--fee FEE] Amount you are willing to pay to process + the transaction (default: 0.25) (minimum: + 0.001) + (alias: -fee) + [--memo STRING] Memo accompanying the transaction + (alias: -memo) + [--nonce NONCE] Nonce that you would like to set for your + transaction (default: nonce of your + account on the best ledger or the + successor of highest value nonce of your + sent transactions from the transaction + pool ) + (alias: -nonce) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -Ensure your Mina node is fully shut down before proceeding: +``` -```sh -mina client stop daemon +### mina client delegate-stake ``` +Delegate your stake to another public key -Or however you normally stop your node process. Verify it is no longer running before continuing. + mina client delegate-stake -### 2. Install storage toolbox packages +=== flags === -Install the required toolbox packages that provide `mina-storage-converter` and the RocksDB scanners: + --receiver PUBLICKEY Public key to which you want to delegate + your stake + (alias: -receiver) + --sender PUBLICKEY Public key from which you want to send the + transaction + (alias: -sender) + [--fee FEE] Amount you are willing to pay to process + the transaction (default: 0.25) (minimum: + 0.001) + (alias: -fee) + [--memo STRING] Memo accompanying the transaction + (alias: -memo) + [--nonce NONCE] Nonce that you would like to set for your + transaction (default: nonce of your + account on the best ledger or the + successor of highest value nonce of your + sent transactions from the transaction + pool ) + (alias: -nonce) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -```sh -sudo apt-get install -y mina-daemon-storage-toolbox mina-daemon-recovery-storage-toolbox ``` -### 3. Convert on-disk state - -Run `mina-storage-converter` to convert the local database to the format expected by the older version. - -```sh -mina-storage-converter \ - --node-dir ${NODE_DIR} \ - --current-scanner /usr/lib/mina/storage/*/${SOURCE_VERSION}/mina-rocksdb-scanner \ - --stable-scanner /usr/lib/mina/storage/*/${TARGET_VERSION}/mina-rocksdb-scanner +### mina client cancel-transaction ``` +Cancel a transaction -- this submits a replacement transaction with a fee larger than the cancelled transaction. -Where: - -- `NODE_DIR` is the path to your node's config directory. This is usually `~/.mina-config` if you haven't set it explicitly. -- `SOURCE_VERSION` is the version you are downgrading from. -- `TARGET_VERSION` is the version you are downgrading to (e.g. `3.3.0`). - -The `*` wildcard lets bash resolve the RocksDB version directory automatically, so you don't need to know which RocksDB version is bundled with each Mina release. + mina client cancel-transaction -The tool will prompt for confirmation before making changes. +=== flags === -### 4. Install the target version + --id ID Transaction ID to be cancelled + (alias: -id) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -Install the older Mina package. For example, to install 3.3.0: +``` -```sh -sudo apt-get install --allow-downgrades -y mina-mainnet=3.3.0 +### mina client set-snark-worker ``` +Set key you wish to snark work with or disable snark working -### 5. Start the Mina daemon - -Start your node as usual: - -```sh -mina daemon ${YOUR_EXTRA_DAEMON_ARGS_HERE} -``` - -Your node should resume from the converted local state without needing to rebootstrap. - -## Docker - -On-disk state conversion is only possible if your `mina-config` directory is persisted as a volume outside the container (e.g. via `--mount "type=bind,source=$(pwd)/.mina-config,dst=/root/.mina-config"`). - -:::caution - -If your `mina-config` is not persisted outside of the container, there is no way to convert the on-disk state. You will need to rebootstrap from scratch after switching to the older image. - -::: - -Since the Docker image is a Debian/Ubuntu environment with the Mina Debian package installed, you can run the same conversion steps inside the container. The default `NODE_DIR` inside the container is `/root/.mina-config`. + mina client set-snark-worker -### 1. Stop the running container +=== flags === -Assume your mina daemon container is running with name `mina-node` + [--address PUBLICKEY] Public-key address you wish to start + snark-working on; null to stop doing any + snark work. Warning: If the key is from a + zkApp account, the account's receive + permission must be None. + (alias: -address) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -```sh -docker stop mina-node ``` -### 2. Install toolbox packages and run the converter - -Use the current (newer) image to install the toolbox packages and run the conversion against your mounted `mina-config` volume: - -```sh -docker run -it --rm \ - --entrypoint bash \ - --mount "type=bind,source=$(pwd)/.mina-config,dst=/root/.mina-config" \ - minaprotocol/mina-daemon:${SOURCE_VERSION}-bullseye-mainnet \ - -c "apt-get update && apt-get install -y mina-daemon-storage-toolbox mina-daemon-recovery-storage-toolbox && mina-storage-converter --node-dir /root/.mina-config --current-scanner /usr/lib/mina/storage/*/${SOURCE_VERSION}/mina-rocksdb-scanner --stable-scanner /usr/lib/mina/storage/*/${TARGET_VERSION}/mina-rocksdb-scanner" +### mina client set-snark-work-fee ``` +Set fee reward for doing transaction snark work -### 3. Start a new container with the target version + mina client set-snark-work-fee FEE -Remove the old container and start with the target image: +=== flags === -```sh -docker rm mina-node + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) -docker run --name mina-node -d \ - -p 8302:8302 \ - --restart=always \ - --mount "type=bind,source=$(pwd)/.mina-config,dst=/root/.mina-config" \ - minaprotocol/mina-daemon:${TARGET_VERSION}-bullseye-mainnet \ - daemon ${YOUR_EXTRA_DAEMON_ARGS_HERE} ``` -Your node should resume from the converted local state without needing to rebootstrap. - ---- -url: /node-operators/faq ---- - -# FAQ - -## General - -### What is Mina Signer? - -The [Mina Signer](https://github.com/o1-labs/o1js/blob/main/src/mina-signer/README.md) NodeJS SDK allows you to sign strings, payments, and delegations using Mina's key pairs for various specified networks. - -The Mina Signer supersedes the deprecated Client SDK. - -## SNARKs and SNARK Workers - -### If I run a SNARK worker, how do I get paid for SNARKs that I generate? - -Block producers (the validators who add new blocks to the blockchain) are required to buy SNARKs from the network (called the snarketplace) and pay out some of their block reward as fees to the SNARK workers who generated SNARKs. This workflow creates a secondary incentive mechanism in the network to reward nodes that help compress transactions. - -### Is generating SNARKs similar to Proof-of-Work (PoW) mining? - -No, they are different in several ways: - -- SNARK proof-of-work is deterministic, while PoW mining requires randomly calculating hashes to try and solve a puzzle. There is no luck element in SNARK work — if a SNARK worker wants to generate a SNARK of a transaction, they only need to generate the proof once. This means SNARK work is much less expensive and less environmentally wasteful, as the compute is all spent towards a productive goal. -- There is no difficulty increase for SNARK work, as there is with PoW mining. In fact, as SNARK constructions, and proof generation times improve, the difficulty can actually decrease. -- SNARK work is not directly involved in consensus. SNARK workers play no role in determining the next state of the blockchain. Their role is to simply generate SNARKs of transactions observed in the network -- As a SNARK worker, there is no requirement for uptime. PoW miners need to run their rigs non-stop to ensure they don't miss out on a potential block. SNARK workers can come online and offline as they please — it is more like Uber, where there is always be work to be done, and nobody needs to say ahead of time when they want to work. - -### Why have my SNARKs not been included? (A.K.A. How should I price my SNARKs?) - -Even though your SNARK worker might be producing SNARKs at a breakneck pace, if someone else produces a cheaper proof for a particular job you have already completed, their SNARK is preferred due to its lower fee. - -Pricing your SNARKs is a delicate balance between the cost of compute, the market environment (demand for SNARKs), your SNARK throughput, and the speed at which each of your SNARK worker processes can produce SNARKs. Sometimes, it might even be economically prudent to turn off your SNARK worker altogether until the market improves. - -### Will SNARK workers require more storage and computing power over time? What about compared to Mina full nodes? - -SNARK workers will not need more storage or computing power over time. SNARK workers simply query the mempool for pending transactions requiring SNARK proofs, and then generate said proof -- this does not require syncing historical data. In addition, the underlying proving cost of SNARK work doesn't get more expensive with time. - -If we are comparing SNARK worker nodes with full nodes on Mina, then yes SNARK workers benefit from specialized hardware as generating SNARK proofs can be compute intensive. Again, however, with the explosion of SNARK research, this is likely to change and become more accessible to consumer hardware. - -### What is the difference between a SNARK, a SNARK proof, and SNARK work? - -SNARKs are a very overloaded term. When you read **SNARK**, it could be referring to the concept of succinct non-interactive proof systems (for example, SNARKs vs Bulletproofs), the specific technical implementation of the proof system (for example, the construction, the circuit, or the prover), or the individual instance of the proof itself (for example, the blockchain SNARK). - -The general terminology is: - -- SNARK: the general concept of succinct, non-interactive zero knowledge proofs -- SNARK circuit: the specific circuit and prover, as pertaining to an app -- SNARK proof: an individual proof that is generated by a SNARK prover -- SNARK work: a Mina protocol data structure that is a wrapper around one or two SNARK proofs and a price to be paid to the SNARK worker that generated the proof or proofs - -### Is there any concern about a single large SNARK worker dumping work in the snarketplace, and then raising prices after monopolizing the market? - -In economics, there is a pricing strategy called [predatory pricing (or dumping)]() where one supplier of a product seeks to exhaust competing suppliers in the market by undercutting the market price. The supplier prices their goods much cheaper than the market rate in order to drive out competitors, even if it means incurring short-term losses. After the market has been cleared, the dominant supplier then increases prices [above competitive market rates](https://en.wikipedia.org/wiki/Supracompetitive_pricing), as competition has been extinguished. - -However, this strategy is effective only in markets where there are high barriers to entry. Meaning, competitors who were crowded out in the predation stage are unable to rejoin the market. +### mina client export-logs +``` +Export daemon logs to tar archive -This is not the case for SNARK work because the barriers to entry are low. Anyone who has spare compute can join the snarketplace and produce as little as one SNARK work and profit on that unit of work. The only barrier to entry is the initial capital expense on hardware, but hardware requirements are low so that users with spare equipment can come online and participate. If any SNARK worker succeeds in driving out the market and increases prices, newcomers are anticipated to reappear and drive prices back down. + mina client export-logs -### Does speed of producing SNARKs matter? If my computer is slower, will I be at a disadvantage? +=== flags === -No, provided that the SNARK work produced is still required by block producers, it doesn't matter who produced it first — only the price matters to block producers. The caveat here is that earlier inclusion into the SNARK mempool is obviously beneficial, as block producers are likely to "see" the work earlier. + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [--tarfile STRING] Basename of the tar archive (default: + date_time) + (alias: -tarfile) + [-help] print this help text and exit + (alias: -?) -However, you can envision a scenario where a set of SNARK workers are favored because they produced the most number of SNARK works that are profitable, and buying proofs from as few entities as possible would allow for more transactions to be included in any block. +``` -There is also a threshold at which time becomes a factor, but this scenario applies only to very underpowered devices. +### mina client export-local-logs +``` +Export local logs (no daemon) to tar archive -### Will a full node need to store all intermediate SNARK proofs? Will the storage requirements grow linearly with blocks? + mina client export-local-logs -No, when a new block is generated, Mina computes the proof recursively over the prior proof and the new block. This is the advantage of recursive composition -- at any given time, nodes need to store only the most recent proof. Intermediate proofs are not needed. For historical clarity on how this architecture emerged, see the [Using zkSNARKs to create a blockless blockchain](https://www.youtube.com/watch?v=eWVGATxEB6M) talk. +=== flags === -### How do you control or limit the number of threads that SNARK workers use? + [--config-directory DIR] Configuration directory + (alias: -config-directory) + [--tarfile STRING] Basename of the tar archive (default: date_time) + (alias: -tarfile) + [-help] print this help text and exit + (alias: -?) -When you start the mina daemon, use the `-snark-worker-parallelism` flag. This flag is equivalent to setting `OMP_NUM_THREADS`, but doesn't affect block production. +``` ---- -url: /node-operators ---- +### mina client stop-daemon +``` +Stop the daemon -# Introduction + mina client stop-daemon -The [Mina Protocol](/mina-protocol) is a layer 1 blockchain that is secured by [proof of stake consensus](/mina-protocol/proof-of-stake). +=== flags === -Node operators are people who run Mina nodes. Mina node operators participate in consensus to create new blocks and help compress data by generating zk-SNARKs. + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) -## Mina Node Roles +``` -A node is a machine running the [Mina daemon](/glossary#daemon). Different nodes fulfill different roles within the Mina network: +### mina client status +``` +Get running daemon status -1. Validator - A plain Mina daemon node that participates in network consensus and can be used for operations such as sending and signing payments. + mina client status -2. [Block Producer](/glossary#block-producer) - A node that participates in a process to determine what blocks it is allowed to produce and then produces blocks containing transactions that can be broadcast to the network. People who run [block producer](https://docs.minaprotocol.com/mina-protocol/block-producers) nodes are also called block producers. +=== flags === -3. [SNARK Coordinators](/glossary#snark-coordinator) - A role on a Mina node in the Mina network that distributes work to a series of SNARK workers in parallel to block production. + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--json] Use JSON output (default: plaintext) + (alias: -json) + [--performance] Include performance histograms in + status output (default: don't + include) + (alias: -performance) + [-help] print this help text and exit + (alias: -?) -4. [SNARK Workers](https://docs.minaprotocol.com/glossary#snark-worker) - SNARK workers create [zk-SNARKs](https://minaprotocol.com/blog/what-are-zk-snarks) for each transaction. These zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of a block, and in turn, these zk-SNARKs are used to create recursive zk-SNARKs that prove the correctness of the network. These zk-SNARKs help provide the Mina Protocol with succinctness. +``` -5. [Archive Nodes](/node-operators/archive-node) - A regular mina daemon that is connected to a running `mina-archive` process. The daemon regularly sends blockchain data to the archive process that stores it in a [PostgreSQL](https://www.postgresql.org/) database. +### mina client help +``` +explain a given subcommand (perhaps recursively) -6. [Seed Nodes](https://docs.minaprotocol.com/glossary#seed-nodes) - Keep a record of nodes in the network and enable nodes joining the network to connect to peer nodes. + mina client help [SUBCOMMAND] -## Operating a Node +=== flags === -Node operators are network participants who run Mina nodes on a Mina network. + [-expand-dots] expand subcommands in recursive help + [-flags] show flags as well in recursive help + [-recursive] show subcommands of subcommands, etc. + [-help] print this help text and exit + (alias: -?) -This section describes how to run each node role and where to find the operational references. +``` -### Validators -- [Requirements](/node-operators/validator-node/requirements) - Hardware, software, and network requirements -- [Generating a Key Pair](/node-operators/validator-node/generating-a-keypair) - How to generate key pairs -- [Querying Data](/node-operators/validator-node/querying-data) - Query blockchain data via GraphQL -- [Staking and Snarking](/node-operators/validator-node/staking-and-snarking) - How to stake your MINA -- [Logging](/node-operators/validator-node/logging) - Log files, levels, and export +## mina daemon +``` +Mina daemon -### Specialized Node Types -- [Block Producers](/node-operators/block-producer-node) - Running a Block Producer -- [SNARK Coordinator & Workers](/node-operators/snark-workers) - Running SNARK coordinator and worker roles -- [Archive Nodes](/node-operators/archive-node) - Running an Archive Node -- [Seed Node](/node-operators/seed-peers) - Running a Seed Node + mina daemon -### Data and Programs -- [Data and History](/node-operators/data-and-history) - Rosetta API and blockchain integration -- [Delegation Program](/node-operators/delegation-program) - The Mina Foundation Delegation Program for Block Producers - -### Reference -- [Mina CLI Reference](/node-operators/reference/mina-cli-reference) - Guide to CLI interactions with Mina networks -- [Mina Signer](/node-operators/reference/mina-signer) - NodeJS SDK for signing - -### Support -- [Troubleshooting](/node-operators/troubleshooting) - Solutions to common problems -- [FAQ](/node-operators/faq) - Frequently asked questions - ---- -url: /node-operators/reference ---- - -# Reference - -Reference documentation and support resources for Mina node operators. - -- **[Mina CLI Reference](/node-operators/reference/mina-cli-reference)** -- Complete reference for the `mina` command line interface. -- **[Mina Signer](/node-operators/reference/mina-signer)** -- NodeJS SDK for signing payments and delegations. -- **[Troubleshooting](/node-operators/troubleshooting)** -- Solutions to common problems when running a Mina node. -- **[FAQ](/node-operators/faq)** -- Frequently asked questions about node operations, SNARKs, and the Mina network. - ---- -url: /node-operators/reference/mina-cli-reference ---- - -# Mina CLI Reference - -The Mina CLI (Command Line Interface) is the primary way for users to interact with the Mina network. It provides standard client functionality to create accounts, send transactions, and participate in consensus. There are also advanced client and daemon commands for power users. - -The Mina CLI is installed when you [install Mina](/node-operators/validator-node/installing-on-ubuntu-and-debian). - -:::tip - -Mina APIs are always improving. See `mina help` for the most up-to-date version. - -::: - -## mina -``` -Mina - - mina SUBCOMMAND - -=== subcommands === - - accounts Client commands concerning account management - daemon Mina daemon - client Lightweight client commands - advanced Advanced client commands - ledger Ledger commands - libp2p Libp2p commands - internal Internal commands - parallel-worker internal use only - transaction-snark-profiler transaction snark profiler - version print version information - help explain a given subcommand (perhaps recursively) - -``` - -## mina accounts -``` -Client commands concerning account management - - mina accounts SUBCOMMAND - -=== subcommands === - - list List all owned accounts - create Create new account - import Import a password protected private key to be tracked by the daemon. - Set MINA_PRIVKEY_PASS environment variable to use non-interactively - (key will be imported using the same password). - export Export a tracked account so that it can be saved or transferred - between machines. - Set MINA_PRIVKEY_PASS environment variable to use non-interactively - (key will be exported using the same password). - unlock Unlock a tracked account - lock Lock a tracked account - help explain a given subcommand (perhaps recursively) - -``` - -### mina accounts list -``` -List all owned accounts - - mina accounts list - -=== flags === - - [--config-directory DIR] Configuration directory - (alias: -config-directory) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina accounts create -``` -Create new account - - mina accounts create - -=== flags === - - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina accounts import -``` -Import a password protected private key to be tracked by the daemon. -Set MINA_PRIVKEY_PASS environment variable to use non-interactively (key will be imported using the same password). - - mina accounts import - -=== flags === - - --privkey-path FILE File to read private key from - (alias: -privkey-path) - [--config-directory DIR] Configuration directory - (alias: -config-directory) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina accounts export -``` -Export a tracked account so that it can be saved or transferred between machines. - Set MINA_PRIVKEY_PASS environment variable to use non-interactively (key will be exported using the same password). - - mina accounts export - -=== flags === - - --privkey-path FILE File to write private key into (public key - will be FILE.pub) - (alias: -privkey-path) - --public-key PUBLICKEY Public key of account to be exported - (alias: -public-key) - [--config-directory DIR] Configuration directory - (alias: -config-directory) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina accounts unlock -``` -Unlock a tracked account - - mina accounts unlock - -=== flags === - - --public-key PUBLICKEY Public key to be unlocked - (alias: -public-key) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina accounts lock -``` -Lock a tracked account - - mina accounts lock - -=== flags === - - --public-key PUBLICKEY Public key of account to be locked - (alias: -public-key) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina accounts help -``` -explain a given subcommand (perhaps recursively) - - mina accounts help [SUBCOMMAND] - -=== flags === - - [-expand-dots] expand subcommands in recursive help - [-flags] show flags as well in recursive help - [-recursive] show subcommands of subcommands, etc. - [-help] print this help text and exit - (alias: -?) - -``` - -## mina client -``` -Lightweight client commands - - mina client SUBCOMMAND - -=== subcommands === - - get-balance Get balance associated with a public key - get-tokens Get all token IDs that a public key has accounts for - send-payment Send payment to an address - delegate-stake Delegate your stake to another public key - cancel-transaction Cancel a transaction -- this submits a replacement - transaction with a fee larger than the cancelled - transaction. - set-snark-worker Set key you wish to snark work with or disable snark - working - set-snark-work-fee Set fee reward for doing transaction snark work - export-logs Export daemon logs to tar archive - export-local-logs Export local logs (no daemon) to tar archive - stop-daemon Stop the daemon - status Get running daemon status - help explain a given subcommand (perhaps recursively) - -``` - -### mina client get-balance -``` -Get balance associated with a public key - - mina client get-balance - -=== flags === - - --public-key PUBLICKEY Public key for which you want to check the - balance - (alias: -public-key) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [--token TOKEN_ID] The token ID for the account - (alias: -token) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client get-tokens -``` -Get all token IDs that a public key has accounts for - - mina client get-tokens - -=== flags === - - --public-key PUBLICKEY Public key for which you want to find - accounts - (alias: -public-key) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client send-payment -``` -Send payment to an address - - mina client send-payment - -=== flags === - - --amount VALUE Payment amount you want to send - (alias: -amount) - --receiver PUBLICKEY Public key to which you want to send money - (alias: -receiver) - --sender PUBLICKEY Public key from which you want to send the - transaction - (alias: -sender) - [--fee FEE] Amount you are willing to pay to process - the transaction (default: 0.25) (minimum: - 0.001) - (alias: -fee) - [--memo STRING] Memo accompanying the transaction - (alias: -memo) - [--nonce NONCE] Nonce that you would like to set for your - transaction (default: nonce of your - account on the best ledger or the - successor of highest value nonce of your - sent transactions from the transaction - pool ) - (alias: -nonce) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client delegate-stake -``` -Delegate your stake to another public key - - mina client delegate-stake - -=== flags === - - --receiver PUBLICKEY Public key to which you want to delegate - your stake - (alias: -receiver) - --sender PUBLICKEY Public key from which you want to send the - transaction - (alias: -sender) - [--fee FEE] Amount you are willing to pay to process - the transaction (default: 0.25) (minimum: - 0.001) - (alias: -fee) - [--memo STRING] Memo accompanying the transaction - (alias: -memo) - [--nonce NONCE] Nonce that you would like to set for your - transaction (default: nonce of your - account on the best ledger or the - successor of highest value nonce of your - sent transactions from the transaction - pool ) - (alias: -nonce) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client cancel-transaction -``` -Cancel a transaction -- this submits a replacement transaction with a fee larger than the cancelled transaction. - - mina client cancel-transaction - -=== flags === - - --id ID Transaction ID to be cancelled - (alias: -id) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client set-snark-worker -``` -Set key you wish to snark work with or disable snark working - - mina client set-snark-worker - -=== flags === - - [--address PUBLICKEY] Public-key address you wish to start - snark-working on; null to stop doing any - snark work. Warning: If the key is from a - zkApp account, the account's receive - permission must be None. - (alias: -address) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client set-snark-work-fee -``` -Set fee reward for doing transaction snark work - - mina client set-snark-work-fee FEE - -=== flags === - - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client export-logs -``` -Export daemon logs to tar archive - - mina client export-logs - -=== flags === - - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [--tarfile STRING] Basename of the tar archive (default: - date_time) - (alias: -tarfile) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client export-local-logs -``` -Export local logs (no daemon) to tar archive - - mina client export-local-logs - -=== flags === - - [--config-directory DIR] Configuration directory - (alias: -config-directory) - [--tarfile STRING] Basename of the tar archive (default: date_time) - (alias: -tarfile) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client stop-daemon -``` -Stop the daemon - - mina client stop-daemon - -=== flags === - - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client status -``` -Get running daemon status - - mina client status - -=== flags === - - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--json] Use JSON output (default: plaintext) - (alias: -json) - [--performance] Include performance histograms in - status output (default: don't - include) - (alias: -performance) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina client help -``` -explain a given subcommand (perhaps recursively) - - mina client help [SUBCOMMAND] - -=== flags === - - [-expand-dots] expand subcommands in recursive help - [-flags] show flags as well in recursive help - [-recursive] show subcommands of subcommands, etc. - [-help] print this help text and exit - (alias: -?) - -``` - -## mina daemon -``` -Mina daemon - - mina daemon - -=== flags === +=== flags === [--all-peers-seen-metric true|false] whether to track the set of all peers ever seen @@ -9489,91 +8204,656 @@ Mina daemon ``` -## mina advanced +## mina advanced +``` +Advanced client commands + + mina advanced SUBCOMMAND + +=== subcommands === + + add-peers Add peers to the daemon + Addresses take the format + /ip4/IPADDR/tcp/PORT/p2p/PEERID + archive-blocks Archive a block from a file. + If an archive address is given, this process will + communicate with the archive node directly; + otherwise it will communicate through the daemon + over the rest-server + batch-send-payments Send multiple payments from a file + client-trustlist Client trustlist management + compile-time-constants Print a JSON map of the compile-time consensus + parameters + compute-receipt-chain-hash Compute the next receipt chain hash from the + previous hash and transaction ID + constraint-system-digests Print MD5 digest of each SNARK constraint + dump-keypair Print out a keypair from a private key file + generate-hardfork-config Generate reference hardfork configuration + generate-keypair Generate a new public, private keypair + get-nonce Get the current nonce for an account + get-peers List the peers currently connected to the daemon + get-public-keys Get public keys + get-trust-status Get the trust status associated with an IP address + get-trust-status-all Get trust statuses for all peers known to the + trust system + hash-transaction Compute the hash of a transaction from its + transaction ID + node-status Get node statuses for a set of peers + object-lifetime-statistics Dump internal object lifetime statistics to JSON + pending-snark-work List of snark works in JSON format that are not + available in the pool yet + pooled-user-commands Retrieve all the user commands that are pending + inclusion + pooled-zkapp-commands Retrieve all the zkApp commands that are pending + inclusion + print-signature-kind Print the signature kind that this binary is + compiled with + reset-trust-status Reset the trust status associated with an IP + address + runtime-config Compute the runtime configuration used by a + running daemon + send-rosetta-transactions Dispatch one or more transactions, provided to + stdin in rosetta format + set-coinbase-receiver Set the coinbase receiver + snark-job-list List of snark jobs in JSON format that are yet to + be included in the blocks + snark-pool-list List of snark works in the snark pool in JSON + format + start-internal-tracing Start internal tracing to + $config-directory/internal-tracing/internal-trace.jsonl + start-tracing Start async tracing to + $config-directory/trace/$pid.trace + status-clear-hist Clear histograms reported in status + stop-internal-tracing Stop internal tracing + stop-tracing Stop async tracing + test Testing-only commands + thread-graph Return a Graphviz Dot graph representation of the + internal thread hierarchy + time-offset Get the time offset in seconds used by the daemon + to convert real time into blockchain time + validate-keypair Validate a public, private keypair + validate-transaction Validate the signature on one or more + transactions, provided to stdin in rosetta format + verify-receipt Verify a receipt of a sent payment + visualization Visualize data structures special to Mina + vrf Commands for vrf evaluations + wrap-key Wrap a private key into a private key file + help explain a given subcommand (perhaps recursively) + +``` + +### mina advanced add-peers +``` +Add peers to the daemon + +Addresses take the format /ip4/IPADDR/tcp/PORT/p2p/PEERID + + mina advanced add-peers PEER [PEER ...] + +=== flags === + + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [--seed true/false] Whether to add these peers as 'seed' + peers, which may perform peer exchange. + Default: true + (alias: -seed) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced archive-blocks +``` +Archive a block from a file. + +If an archive address is given, this process will communicate with the archive node directly; otherwise it will communicate through the daemon over the rest-server + + mina advanced archive-blocks [FILES ...] + +=== flags === + + [--archive-address HOST:PORT/LOCALHOST-PORT] Daemon to archive process + communication. If HOST is + omitted, then localhost is + assumed to be HOST. (examples: + 3086, 154.97.53.97:3086) + (alias: -archive-address) + [--extensional] Blocks are in extensional JSON + format + (alias: -extensional) + [--failed-files PATH] Appends the list of files that + failed to be processed + (alias: -failed-files) + [--log-successful true/false] Whether to log messages for + files that were processed + successfully + (alias: -log-successful) + [--precomputed] Blocks are in precomputed JSON + format + (alias: -precomputed) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon + interaction (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [--successful-files PATH] Appends the list of files that + were processed successfully + (alias: -successful-files) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced batch-send-payments +``` +Send multiple payments from a file + + mina advanced batch-send-payments PAYMENTS-FILE + +=== flags === + + --privkey-path FILE File to read private key from + (alias: -privkey-path) + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced client-trustlist +``` +Client trustlist management + + mina advanced client-trustlist SUBCOMMAND + +=== subcommands === + + add Add an IP to the trustlist + list List the CIDR masks in the trustlist + remove Remove a CIDR mask from the trustlist + help explain a given subcommand (perhaps recursively) + +``` + +### mina advanced compile-time-constants +``` +Print a JSON map of the compile-time consensus parameters + + mina advanced compile-time-constants + +=== flags === + + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced compute-receipt-chain-hash +``` +Compute the next receipt chain hash from the previous hash and transaction ID + + mina advanced compute-receipt-chain-hash + +=== flags === + + --previous-hash HASH Previous receipt chain hash, + Base58Check-encoded + --transaction-id TRANSACTION_ID Transaction ID, Base64-encoded + [--index NN] For a zkApp, 0 for fee payer or + 1-based index of account update + [--signature-kind mainnet|testnet|] Signature kind to use (default: + value compiled into this binary) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced constraint-system-digests +``` +Print MD5 digest of each SNARK constraint + + mina advanced constraint-system-digests + +=== flags === + + [--signature-kind mainnet|testnet|] Signature kind to use (default: + value compiled into this binary) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced dump-keypair +``` +Print out a keypair from a private key file + + mina advanced dump-keypair + +=== flags === + + --privkey-path FILE File to read private key from + (alias: -privkey-path) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced generate-hardfork-config +``` +Generate reference hardfork configuration + + mina advanced generate-hardfork-config + +=== flags === + + --hardfork-config-dir DIR Directory to generate hardfork + configuration, relative to the + daemon working directory + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--generate-fork-validation BOOL] whether generating the fork + validation folder. Defaults to true + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced generate-keypair +``` +Generate a new public, private keypair + + mina advanced generate-keypair + +=== flags === + + --privkey-path FILE File to write private key into (public key will be + FILE.pub) + (alias: -privkey-path) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced get-nonce +``` +Get the current nonce for an account + + mina advanced get-nonce + +=== flags === + + --address PUBLICKEY Public-key address you want the + nonce for + (alias: -address) + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--token TOKEN_ID] The token ID for the account + (alias: -token) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced get-peers +``` +List the peers currently connected to the daemon + + mina advanced get-peers + +=== flags === + + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced get-public-keys +``` +Get public keys + + mina advanced get-public-keys + +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--json] Use JSON output (default: plaintext) + (alias: -json) + [--with-details] Show extra details (eg. balance, + nonce) in addition to public keys + (alias: -with-details) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced get-trust-status +``` +Get the trust status associated with an IP address + + mina advanced get-trust-status + +=== flags === + + --ip-address IP An IPv4 or IPv6 address for which + you want to query the trust status + (alias: -ip-address) + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--json] Use JSON output (default: plaintext) + (alias: -json) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced get-trust-status-all +``` +Get trust statuses for all peers known to the trust system + + mina advanced get-trust-status-all + +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--json] Use JSON output (default: plaintext) + (alias: -json) + [--nonzero-only] Only show trust statuses whose trust + score is nonzero + (alias: -nonzero-only) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced hash-transaction +``` +Compute the hash of a transaction from its transaction ID + + mina advanced hash-transaction + +=== flags === + + --transaction-id ID ID of the transaction to hash + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced node-status +``` +Get node statuses for a set of peers + + mina advanced node-status + +=== flags === + + [--daemon-peers] Get node statuses for peers known to + the daemon + (alias: -daemon-peers) + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--peers CSV-LIST] Peer multiaddrs for obtaining node + status + (alias: -peers) + [--show-errors] Include error responses in output + (alias: -show-errors) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced object-lifetime-statistics +``` +Dump internal object lifetime statistics to JSON + + mina advanced object-lifetime-statistics + +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced pending-snark-work ``` -Advanced client commands +List of snark works in JSON format that are not available in the pool yet - mina advanced SUBCOMMAND + mina advanced pending-snark-work -=== subcommands === +=== flags === - add-peers Add peers to the daemon - Addresses take the format - /ip4/IPADDR/tcp/PORT/p2p/PEERID - archive-blocks Archive a block from a file. - If an archive address is given, this process will - communicate with the archive node directly; - otherwise it will communicate through the daemon - over the rest-server - batch-send-payments Send multiple payments from a file - client-trustlist Client trustlist management - compile-time-constants Print a JSON map of the compile-time consensus - parameters - compute-receipt-chain-hash Compute the next receipt chain hash from the - previous hash and transaction ID - constraint-system-digests Print MD5 digest of each SNARK constraint - dump-keypair Print out a keypair from a private key file - generate-hardfork-config Generate reference hardfork configuration - generate-keypair Generate a new public, private keypair - get-nonce Get the current nonce for an account - get-peers List the peers currently connected to the daemon - get-public-keys Get public keys - get-trust-status Get the trust status associated with an IP address - get-trust-status-all Get trust statuses for all peers known to the - trust system - hash-transaction Compute the hash of a transaction from its - transaction ID - node-status Get node statuses for a set of peers - object-lifetime-statistics Dump internal object lifetime statistics to JSON - pending-snark-work List of snark works in JSON format that are not - available in the pool yet - pooled-user-commands Retrieve all the user commands that are pending - inclusion - pooled-zkapp-commands Retrieve all the zkApp commands that are pending - inclusion - print-signature-kind Print the signature kind that this binary is - compiled with - reset-trust-status Reset the trust status associated with an IP - address - runtime-config Compute the runtime configuration used by a - running daemon - send-rosetta-transactions Dispatch one or more transactions, provided to - stdin in rosetta format - set-coinbase-receiver Set the coinbase receiver - snark-job-list List of snark jobs in JSON format that are yet to - be included in the blocks - snark-pool-list List of snark works in the snark pool in JSON - format - start-internal-tracing Start internal tracing to - $config-directory/internal-tracing/internal-trace.jsonl - start-tracing Start async tracing to - $config-directory/trace/$pid.trace - status-clear-hist Clear histograms reported in status - stop-internal-tracing Stop internal tracing - stop-tracing Stop async tracing - test Testing-only commands - thread-graph Return a Graphviz Dot graph representation of the - internal thread hierarchy - time-offset Get the time offset in seconds used by the daemon - to convert real time into blockchain time - validate-keypair Validate a public, private keypair - validate-transaction Validate the signature on one or more - transactions, provided to stdin in rosetta format - verify-receipt Verify a receipt of a sent payment - visualization Visualize data structures special to Mina - vrf Commands for vrf evaluations - wrap-key Wrap a private key into a private key file - help explain a given subcommand (perhaps recursively) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced pooled-user-commands +``` +Retrieve all the user commands that are pending inclusion + + mina advanced pooled-user-commands [PUBLIC-KEY] + +=== flags === + + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced pooled-zkapp-commands +``` +Retrieve all the zkApp commands that are pending inclusion + + mina advanced pooled-zkapp-commands [PUBLIC-KEY] + +=== flags === + + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced print-signature-kind +``` +Print the signature kind that this binary is compiled with + + mina advanced print-signature-kind + +=== flags === + + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced reset-trust-status +``` +Reset the trust status associated with an IP address + + mina advanced reset-trust-status + +=== flags === + + --ip-address IP An IPv4 or IPv6 address for which + you want to reset the trust status + (alias: -ip-address) + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [--json] Use JSON output (default: plaintext) + (alias: -json) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced runtime-config +``` +Compute the runtime configuration used by a running daemon + + mina advanced runtime-config + +=== flags === + + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced send-rosetta-transactions +``` +Dispatch one or more transactions, provided to stdin in rosetta format + + mina advanced send-rosetta-transactions + +=== flags === + + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced set-coinbase-receiver +``` +Set the coinbase receiver + + mina advanced set-coinbase-receiver + +=== flags === + + [--block-producer] Send coinbase rewards to the block + producer's public key + (alias: -block-producer) + [--public-key PUBLICKEY] Public key of account to send coinbase + rewards to + (alias: -public-key) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced snark-job-list +``` +List of snark jobs in JSON format that are yet to be included in the blocks + + mina advanced snark-job-list + +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced add-peers +### mina advanced snark-pool-list ``` -Add peers to the daemon - -Addresses take the format /ip4/IPADDR/tcp/PORT/p2p/PEERID +List of snark works in the snark pool in JSON format - mina advanced add-peers PEER [PEER ...] + mina advanced snark-pool-list === flags === @@ -9584,136 +8864,179 @@ Addresses take the format /ip4/IPADDR/tcp/PORT/p2p/PEERID (default: 3085 or http://localhost:3085/graphql) (alias: -rest-server) - [--seed true/false] Whether to add these peers as 'seed' - peers, which may perform peer exchange. - Default: true - (alias: -seed) [-help] print this help text and exit (alias: -?) ``` -### mina advanced archive-blocks +### mina advanced start-internal-tracing ``` -Archive a block from a file. +Start internal tracing to $config-directory/internal-tracing/internal-trace.jsonl -If an archive address is given, this process will communicate with the archive node directly; otherwise it will communicate through the daemon over the rest-server + mina advanced start-internal-tracing - mina advanced archive-blocks [FILES ...] +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced start-tracing +``` +Start async tracing to $config-directory/trace/$pid.trace + + mina advanced start-tracing === flags === - [--archive-address HOST:PORT/LOCALHOST-PORT] Daemon to archive process - communication. If HOST is - omitted, then localhost is - assumed to be HOST. (examples: - 3086, 154.97.53.97:3086) - (alias: -archive-address) - [--extensional] Blocks are in extensional JSON - format - (alias: -extensional) - [--failed-files PATH] Appends the list of files that - failed to be processed - (alias: -failed-files) - [--log-successful true/false] Whether to log messages for - files that were processed - successfully - (alias: -log-successful) - [--precomputed] Blocks are in precomputed JSON - format - (alias: -precomputed) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon - interaction (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [--successful-files PATH] Appends the list of files that - were processed successfully - (alias: -successful-files) - [-help] print this help text and exit - (alias: -?) + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced batch-send-payments +### mina advanced status-clear-hist ``` -Send multiple payments from a file +Clear histograms reported in status - mina advanced batch-send-payments PAYMENTS-FILE + mina advanced status-clear-hist === flags === - --privkey-path FILE File to read private key from - (alias: -privkey-path) [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon communication. If HOST is omitted, then localhost is assumed to be HOST. (examples: 8301, 154.97.53.97:8301) (default: 8301) (alias: -daemon-port) + [--json] Use JSON output (default: plaintext) + (alias: -json) + [--performance] Include performance histograms in + status output (default: don't + include) + (alias: -performance) [-help] print this help text and exit (alias: -?) ``` -### mina advanced client-trustlist +### mina advanced stop-internal-tracing ``` -Client trustlist management +Stop internal tracing - mina advanced client-trustlist SUBCOMMAND + mina advanced stop-internal-tracing + +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced stop-tracing +``` +Stop async tracing + + mina advanced stop-tracing + +=== flags === + + [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon + communication. If HOST is omitted, + then localhost is assumed to be + HOST. (examples: 8301, + 154.97.53.97:8301) (default: 8301) + (alias: -daemon-port) + [-help] print this help text and exit + (alias: -?) + +``` + +### mina advanced test +``` +Testing-only commands + + mina advanced test SUBCOMMAND === subcommands === - add Add an IP to the trustlist - list List the CIDR masks in the trustlist - remove Remove a CIDR mask from the trustlist - help explain a given subcommand (perhaps recursively) + create-genesis Test genesis creation + submit-to-archive Generate blocks with zkApp transactions and payments. + Optionally submit to archive node or save to file for + analysis. + help explain a given subcommand (perhaps recursively) ``` -### mina advanced compile-time-constants +### mina advanced thread-graph ``` -Print a JSON map of the compile-time consensus parameters +Return a Graphviz Dot graph representation of the internal thread hierarchy - mina advanced compile-time-constants + mina advanced thread-graph === flags === - [-help] print this help text and exit - (alias: -?) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced compute-receipt-chain-hash +### mina advanced time-offset ``` -Compute the next receipt chain hash from the previous hash and transaction ID +Get the time offset in seconds used by the daemon to convert real time into blockchain time - mina advanced compute-receipt-chain-hash + mina advanced time-offset === flags === - --previous-hash HASH Previous receipt chain hash, - Base58Check-encoded - --transaction-id TRANSACTION_ID Transaction ID, Base64-encoded - [--index NN] For a zkApp, 0 for fee payer or - 1-based index of account update - [--signature-kind mainnet|testnet|] Signature kind to use (default: - value compiled into this binary) - [-help] print this help text and exit - (alias: -?) + [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction + (examples: 3085 or + http://localhost:3085/graphql, + /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) + (default: 3085 or + http://localhost:3085/graphql) + (alias: -rest-server) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced constraint-system-digests +### mina advanced validate-keypair ``` -Print MD5 digest of each SNARK constraint +Validate a public, private keypair - mina advanced constraint-system-digests + mina advanced validate-keypair === flags === + --privkey-path FILE File to write private key into + (public key will be FILE.pub) + (alias: -privkey-path) [--signature-kind mainnet|testnet|] Signature kind to use (default: value compiled into this binary) [-help] print this help text and exit @@ -9721,50 +9044,101 @@ Print MD5 digest of each SNARK constraint ``` -### mina advanced dump-keypair +### mina advanced validate-transaction ``` -Print out a keypair from a private key file +Validate the signature on one or more transactions, provided to stdin in rosetta format - mina advanced dump-keypair + mina advanced validate-transaction === flags === - --privkey-path FILE File to read private key from - (alias: -privkey-path) - [-help] print this help text and exit - (alias: -?) + [--signature-kind mainnet|testnet|] Signature kind to use (default: + value compiled into this binary) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced generate-hardfork-config +### mina advanced verify-receipt ``` -Generate reference hardfork configuration +Verify a receipt of a sent payment - mina advanced generate-hardfork-config + mina advanced verify-receipt === flags === - --hardfork-config-dir DIR Directory to generate hardfork - configuration, relative to the - daemon working directory + --address PUBLICKEY Public-key address of sender + (alias: -address) + --payment-path PAYMENTPATH File to read json version of + verifying payment + (alias: -payment-path) + --proof-path PROOFFILE File to read json version of payment + receipt + (alias: -proof-path) [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon communication. If HOST is omitted, then localhost is assumed to be HOST. (examples: 8301, 154.97.53.97:8301) (default: 8301) (alias: -daemon-port) - [--generate-fork-validation BOOL] whether generating the fork - validation folder. Defaults to true + [--legacy] Use legacy json format (zkapp + command with hashes) + [--token TOKEN_ID] The token ID for the account + (alias: -token) [-help] print this help text and exit (alias: -?) ``` -### mina advanced generate-keypair +### mina advanced visualization +``` +Visualize data structures special to Mina + + mina advanced visualization SUBCOMMAND + +=== subcommands === + + registered-masks Produce a visualization of the registered-masks + transition-frontier Produce a visualization of the transition-frontier + help explain a given subcommand (perhaps recursively) + +``` + +### mina advanced vrf +``` +Commands for vrf evaluations + + mina advanced vrf SUBCOMMAND + +=== subcommands === + + batch-check-witness Check a batch of vrf evaluation witnesses read on + stdin. Outputs the verified vrf evaluations (or no vrf + output if the witness is invalid), and whether the vrf + output satisfies the threshold values if given. The + threshold should be included in the JSON for each vrf + as the 'vrfThreshold' field, of format + {delegatedStake: 1000, totalStake: 1000000000}. The + threshold is not checked against a ledger; this should + be done manually to confirm whether threshold_met in + the output corresponds to an actual won block. + batch-generate-witness Generate a batch of vrf evaluation witnesses from + {"globalSlot": _, "epochSeed": _, "delegatorIndex": _} + JSON message objects read on stdin + generate-witness Generate a vrf evaluation witness. This may be used to + calculate whether a given private key will win a given + slot (by checking threshold_met = true in the JSON + output), or to generate a witness that a 3rd + account_update can use to verify a vrf evaluation. + help explain a given subcommand (perhaps recursively) + +``` + +### mina advanced wrap-key ``` -Generate a new public, private keypair +Wrap a private key into a private key file - mina advanced generate-keypair + mina advanced wrap-key === flags === @@ -9776,103 +9150,62 @@ Generate a new public, private keypair ``` -### mina advanced get-nonce -``` -Get the current nonce for an account - - mina advanced get-nonce - -=== flags === - - --address PUBLICKEY Public-key address you want the - nonce for - (alias: -address) - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--token TOKEN_ID] The token ID for the account - (alias: -token) - [-help] print this help text and exit - (alias: -?) - -``` - -### mina advanced get-peers +### mina advanced help ``` -List the peers currently connected to the daemon +explain a given subcommand (perhaps recursively) - mina advanced get-peers + mina advanced help [SUBCOMMAND] === flags === - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) + [-expand-dots] expand subcommands in recursive help + [-flags] show flags as well in recursive help + [-recursive] show subcommands of subcommands, etc. + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced get-public-keys +## mina ledger ``` -Get public keys +Ledger commands - mina advanced get-public-keys + mina ledger SUBCOMMAND -=== flags === +=== subcommands === - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--json] Use JSON output (default: plaintext) - (alias: -json) - [--with-details] Show extra details (eg. balance, - nonce) in addition to public keys - (alias: -with-details) - [-help] print this help text and exit - (alias: -?) + currency Print the total currency for each token present in the ledger + contained in the specified file + export Print the specified ledger (default: staged ledger at the best tip). + Note: Exporting snarked ledger is an expensive operation and can + take a few seconds + hash Print the Merkle root of the ledger contained in the specified file + test Testing-only commands + help explain a given subcommand (perhaps recursively) ``` -### mina advanced get-trust-status +### mina ledger currency ``` -Get the trust status associated with an IP address +Print the total currency for each token present in the ledger contained in the specified file - mina advanced get-trust-status + mina ledger currency === flags === - --ip-address IP An IPv4 or IPv6 address for which - you want to query the trust status - (alias: -ip-address) - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--json] Use JSON output (default: plaintext) - (alias: -json) - [-help] print this help text and exit - (alias: -?) + --ledger-file LEDGER-FILE File containing an exported ledger + [--plaintext] Use plaintext input or output (default: JSON) + (alias: -plaintext) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced get-trust-status-all +### mina ledger export ``` -Get trust statuses for all peers known to the trust system +Print the specified ledger (default: staged ledger at the best tip). Note: Exporting snarked ledger is an expensive operation and can take a few seconds - mina advanced get-trust-status-all + mina ledger export STAGED-LEDGER|SNARKED-LEDGER|STAKING-EPOCH-LEDGER|NEXT-EPOCH-LEDGER === flags === @@ -9882,761 +9215,881 @@ Get trust statuses for all peers known to the trust system HOST. (examples: 8301, 154.97.53.97:8301) (default: 8301) (alias: -daemon-port) - [--json] Use JSON output (default: plaintext) - (alias: -json) - [--nonzero-only] Only show trust statuses whose trust - score is nonzero - (alias: -nonzero-only) + [--plaintext] Use plaintext input or output + (default: JSON) + (alias: -plaintext) + [--state-hash STATE-HASH] State hash, if printing a staged + ledger or snarked ledger (default: + state hash for the best tip) + (alias: -state-hash) [-help] print this help text and exit (alias: -?) ``` -### mina advanced hash-transaction +### mina ledger hash ``` -Compute the hash of a transaction from its transaction ID +Print the Merkle root of the ledger contained in the specified file - mina advanced hash-transaction + mina ledger hash === flags === - --transaction-id ID ID of the transaction to hash - [-help] print this help text and exit - (alias: -?) + --ledger-file LEDGER-FILE File containing an exported ledger + [--plaintext] Use plaintext input or output (default: JSON) + (alias: -plaintext) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced node-status +### mina ledger test ``` -Get node statuses for a set of peers +Testing-only commands - mina advanced node-status + mina ledger test SUBCOMMAND -=== flags === +=== subcommands === - [--daemon-peers] Get node statuses for peers known to - the daemon - (alias: -daemon-peers) - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--peers CSV-LIST] Peer multiaddrs for obtaining node - status - (alias: -peers) - [--show-errors] Include error responses in output - (alias: -show-errors) - [-help] print this help text and exit - (alias: -?) + apply Test ledger application + generate-accounts Generate a ledger for testing + help explain a given subcommand (perhaps recursively) ``` -### mina advanced object-lifetime-statistics +### mina ledger help ``` -Dump internal object lifetime statistics to JSON +explain a given subcommand (perhaps recursively) - mina advanced object-lifetime-statistics + mina ledger help [SUBCOMMAND] === flags === - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [-help] print this help text and exit - (alias: -?) + [-expand-dots] expand subcommands in recursive help + [-flags] show flags as well in recursive help + [-recursive] show subcommands of subcommands, etc. + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced pending-snark-work +## mina libp2p ``` -List of snark works in JSON format that are not available in the pool yet +Libp2p commands - mina advanced pending-snark-work + mina libp2p SUBCOMMAND -=== flags === +=== subcommands === - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) + dump-keypair Print an existing libp2p keypair + generate-keypair Generate a new libp2p keypair and print out the peer ID + help explain a given subcommand (perhaps recursively) ``` -### mina advanced pooled-user-commands +### mina libp2p dump-keypair ``` -Retrieve all the user commands that are pending inclusion +Print an existing libp2p keypair - mina advanced pooled-user-commands [PUBLIC-KEY] + mina libp2p dump-keypair === flags === - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) + --privkey-path FILE File to read private key from + (alias: -privkey-path) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced pooled-zkapp-commands +### mina libp2p generate-keypair ``` -Retrieve all the zkApp commands that are pending inclusion +Generate a new libp2p keypair and print out the peer ID - mina advanced pooled-zkapp-commands [PUBLIC-KEY] + mina libp2p generate-keypair === flags === - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) + --privkey-path FILE File to write private key into (public key will be + FILE.pub) + (alias: -privkey-path) + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced print-signature-kind +### mina libp2p help ``` -Print the signature kind that this binary is compiled with +explain a given subcommand (perhaps recursively) - mina advanced print-signature-kind + mina libp2p help [SUBCOMMAND] === flags === - [-help] print this help text and exit - (alias: -?) + [-expand-dots] expand subcommands in recursive help + [-flags] show flags as well in recursive help + [-recursive] show subcommands of subcommands, etc. + [-help] print this help text and exit + (alias: -?) ``` -### mina advanced reset-trust-status -``` -Reset the trust status associated with an IP address +--- +url: /node-operators/rosetta/build-from-sources +--- - mina advanced reset-trust-status +# Building and running Rosetta from source code -=== flags === +In [Running with Docker](run-with-docker), you learned that the Docker container runs the mina daemon, mina-archive, and Rosetta API. - --ip-address IP An IPv4 or IPv6 address for which - you want to reset the trust status - (alias: -ip-address) - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--json] Use JSON output (default: plaintext) - (alias: -json) - [-help] print this help text and exit - (alias: -?) +If you already have mina-archive up and running, you can also build and run the Rosetta API natively and wire it to the existing mina daemon and mina-archive. -``` +:::tip -### mina advanced runtime-config -``` -Compute the runtime configuration used by a running daemon +To run mina daemon and mina-archive natively, follow the [Archive Node page](/node-operators/archive-node/getting-started) instructions. - mina advanced runtime-config +::: -=== flags === +The easiest way to build Rosetta natively is to use the Nix development environment. - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) +1. + Clone the official [mina repository](https://github.com/MinaProtocol/mina.git) and switch to the `compatible` branch. -``` +1. + If you don't already have Nix on your machine, install it following the steps in the [nix/README.md](https://github.com/MinaProtocol/mina/blob/develop/nix/README.md). -### mina advanced send-rosetta-transactions -``` -Dispatch one or more transactions, provided to stdin in rosetta format +1. + Run the `./nix/pin.sh` script to enable submodules to be available to the build: - mina advanced send-rosetta-transactions + ```shell + ./nix/pin.sh + ``` -=== flags === +1. + Launch the development shell: - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) + ```shell + nix develop mina + ``` -``` +1. + Build Rosetta app: -### mina advanced set-coinbase-receiver -``` -Set the coinbase receiver + ```shell + dune build --profile=mainnet src/app/rosetta + ``` - mina advanced set-coinbase-receiver +1. + After a successful build, the Rosetta app is available here: `_build/default/src/app/rosetta/rosetta.exe` -=== flags === + You can run it with following command: + ```shell + MINA_ROSETTA_MAX_DB_POOL_SIZE=64 \ + _build/default/src/app/rosetta/rosetta.exe --port 3087 \ + --graphql-uri + --archive-uri postgres://:@:/ + ``` - [--block-producer] Send coinbase rewards to the block - producer's public key - (alias: -block-producer) - [--public-key PUBLICKEY] Public key of account to send coinbase - rewards to - (alias: -public-key) - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) +--- +url: /node-operators/rosetta/docker-compose +--- -``` +# Docker Compose Rosetta -### mina advanced snark-job-list -``` -List of snark jobs in JSON format that are yet to be included in the blocks +For production deployments, use Docker Compose to run each Rosetta component as a separate container. This gives you control over resource allocation, logging, and restarts. - mina advanced snark-job-list +The full Docker Compose configuration — including `docker-compose.yml`, example environment files for mainnet and devnet, a `Makefile`, and a `README` — is maintained in the Mina repository: -=== flags === +**[`mina/src/app/rosetta/docker-compose/`](https://github.com/MinaProtocol/mina/tree/compatible/src/app/rosetta/docker-compose)** - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [-help] print this help text and exit - (alias: -?) +## Quick start -``` +:::tip +Before running the commands below, review the [Configuration](#configuration) section to see all available options — including image tags, network selection, ports, and database settings. +::: -### mina advanced snark-pool-list -``` -List of snark works in the snark pool in JSON format +```bash +git clone https://github.com/MinaProtocol/mina.git +cd mina/src/app/rosetta/docker-compose - mina advanced snark-pool-list +# For mainnet +cp example.mainnet.env .env -=== flags === +# For devnet +cp example.devnet.env .env - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) +# Edit .env — set MINA_LIBP2P_PASS and review POSTGRES_PASSWORD +vi .env -``` +# Start all services +docker compose up -d -### mina advanced start-internal-tracing +# Or use make shortcuts +make mainnet # copies mainnet env and starts services +make devnet # copies devnet env and starts services ``` -Start internal tracing to $config-directory/internal-tracing/internal-trace.jsonl - mina advanced start-internal-tracing +## Services overview -=== flags === +The Docker Compose setup includes six services: - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [-help] print this help text and exit - (alias: -?) +| Service | Description | Default Port | +|---------|-------------|--------------| +| **postgres** | PostgreSQL 17 with health checks | 5432 (container), configurable host port | +| **bootstrap_db** | One-shot: downloads and imports the latest daily archive dump | — | +| **mina_archive** | Archive process, stores block data in PostgreSQL | 3086 | +| **mina_node** | Mina daemon with GraphQL API | 3085 (GraphQL), 8302 (P2P) | +| **mina_rosetta** | Rosetta API for exchange integration | 3087 | +| **missing_blocks_guardian** | Monitors and recovers missing blocks between nightly dumps and chain tip | — | -``` +## Configuration -### mina advanced start-tracing -``` -Start async tracing to $config-directory/trace/$pid.trace +All configuration is done through a single `.env` file. Key variables: - mina advanced start-tracing +### Docker images -=== flags === +| Variable | Description | +|----------|-------------| +| `MINA_DAEMON_IMAGE` | Mina daemon Docker image | +| `MINA_ARCHIVE_IMAGE` | Mina archive Docker image | +| `MINA_ROSETTA_IMAGE` | Mina Rosetta Docker image | - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [-help] print this help text and exit - (alias: -?) +For mainnet, images are pulled from [Docker Hub](https://hub.docker.com/u/minaprotocol) (`minaprotocol/mina-*`). +For devnet, images are pulled from the o1Labs GCR registry. -``` +### Network -### mina advanced status-clear-hist -``` -Clear histograms reported in status +| Variable | Description | +|----------|-------------| +| `MINA_NETWORK` | `mainnet` or `devnet` | +| `MINA_PEERLIST_URL` | Bootstrap peers URL | +| `MINA_LIBP2P_PASS` | Passphrase for the libp2p key (required) | - mina advanced status-clear-hist +### Ports + +| Variable | Default | Description | +|----------|---------|-------------| +| `POSTGRES_PORT` | `5433` | Host port mapped to PostgreSQL | +| `MINA_REST_PORT` | `3085` | GraphQL API port | +| `MINA_P2P_PORT` | `8302` | P2P networking port | +| `MINA_ARCHIVE_PORT` | `3086` | Archive server port | +| `MINA_ROSETTA_PORT` | `3087` | Rosetta API port | + +### Archive bootstrap + +| Variable | Description | +|----------|-------------| +| `ARCHIVE_DUMP_BASE_URL` | Base URL for daily archive dumps | +| `ARCHIVE_DUMP_PREFIX` | Dump filename prefix (`mainnet-archive-dump` or `devnet-archive-dump`) | +| `GUARDIAN_PRECOMPUTED_BLOCKS_URL` | S3 bucket URL for precomputed blocks used by the missing blocks guardian | + +## Data persistence + +Bind mounts preserve data across `docker compose down` / `up`: + +| Host path | Container path | Contents | +|-----------|---------------|----------| +| `./archive/postgresql/data` | `/var/lib/postgresql/data` | PostgreSQL data | +| `./archive/data` | `/data` | Archive node data | +| `./mina_node/.mina-config` | `/root/.mina-config` | Daemon config, keys, peers | +| `./mina_rosetta/.mina-config` | `/root/.mina-config` | Rosetta config | -=== flags === +## Make targets - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--json] Use JSON output (default: plaintext) - (alias: -json) - [--performance] Include performance histograms in - status output (default: don't - include) - (alias: -performance) - [-help] print this help text and exit - (alias: -?) +| Command | Description | +|---------|-------------| +| `make devnet` | Copy devnet env and start services | +| `make mainnet` | Copy mainnet env and start services | +| `make stop` | Stop all services | +| `make clean` | Stop services, remove volumes and all persisted data | +| `make logs` | Follow logs for all services | +| `make status` | Show container status | +| `make health` | Check health of Postgres, GraphQL, and Rosetta endpoints | -``` +## Verifying the deployment -### mina advanced stop-internal-tracing -``` -Stop internal tracing +Once services are running and the node is synced: - mina advanced stop-internal-tracing +```bash +# Check container status +make status -=== flags === +# Run health checks +make health - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [-help] print this help text and exit - (alias: -?) +# Check sync status +docker compose exec mina_node mina client status -``` +# Query Rosetta API +curl -s http://localhost:3087/network/list \ + -H 'Content-Type: application/json' -d '{}' | jq . -### mina advanced stop-tracing +# Connect to archive database +psql postgres://postgres:postgres@localhost:5433/archive ``` -Stop async tracing - - mina advanced stop-tracing -=== flags === +## Clean start - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [-help] print this help text and exit - (alias: -?) +To wipe all data and start fresh: +```bash +make clean +docker compose up -d ``` -### mina advanced test -``` -Testing-only commands +--- +url: /node-operators/rosetta/run-with-docker +--- - mina advanced test SUBCOMMAND +# Running with Docker -=== subcommands === +:::info - create-genesis Test genesis creation - submit-to-archive Generate blocks with zkApp transactions and payments. - Optionally submit to archive node or save to file for - analysis. - help explain a given subcommand (perhaps recursively) +There is a known issue that you can't use the docker image on the Apple silicon chip. +To run Mina Rosetta on Apple silicon, you can use the steps in [Building and running Rosetta from source code](build-from-sources). -``` +::: -### mina advanced thread-graph -``` -Return a Graphviz Dot graph representation of the internal thread hierarchy +1. [Install Docker](https://www.docker.com/get-started) and check that your Docker configuration has at least 16 GB RAM (the recommended amount is 32 GB). - mina advanced thread-graph +1. Check the latest release for Mainnet on the official [Mina GitHub releases](https://github.com/MinaProtocol/mina/releases) page. -=== flags === +1. Use the Mina Rosetta Docker image: - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) -``` +:::note -### mina advanced time-offset -``` -Get the time offset in seconds used by the daemon to convert real time into blockchain time +If you want to build your own docker image, you can find more details in [Mina's Rosetta repository](https://github.com/MinaProtocol/mina/blob/develop/src/app/rosetta/README.md). However, for most users, it's not necessary to build your own image in order to interact with the API. - mina advanced time-offset +::: -=== flags === +The container in `/rosetta` includes three entrypoints, which each run a different set of services connected to a particular network. - [--rest-server URI/LOCALHOST-PORT] graphql rest server for daemon interaction - (examples: 3085 or - http://localhost:3085/graphql, - /dns4/peer1-rising-phoenix.o1test.net:3085/graphql) - (default: 3085 or - http://localhost:3085/graphql) - (alias: -rest-server) - [-help] print this help text and exit - (alias: -?) +- **docker-start.sh** (default) - connects the mina node to a network (defaults to Mainnet) and initializes the archive database from publicly-available nightly O(1) Labs backups. This script runs a mina node, mina-archive, a postgresql DB, and mina-rosetta. The script also periodically checks for blocks that may be missing between the nightly backup and the tip of the chain and will fill in those gaps by walking back the linked list of blocks in the canonical chain and importing them one at a time. The script is configurable through environment variables. Take a look at the source for more information about what and how you can configure. +- **docker-standalone-start.sh** - starts only the mina-rosetta API endpoint and any flags passed into the script go to mina-rosetta. You may use this for the "offline" part of the Construction API or if you want a setup with each service in its own container. +- **docker-demo-start.sh** launches a mina node with a very simple 1-address genesis ledger as a sandbox for developing and playing around in. This script starts the full suite of tools (a mina node, mina-archive, a postgresql DB, and mina-rosetta), but for a demo network with all operations occurring inside this container and no external network activity. -``` +Run the container with following command (replace the image tag with one from dockerhub that's compatible with the network you are trying to connect to, also replace `--entrypoint` and any environment variable if needed): -### mina advanced validate-keypair ``` -Validate a public, private keypair + docker run -it --rm --name rosetta \ + --entrypoint=./docker-start.sh \ + -p 10101:10101 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ + minaprotocol/mina-rosetta:3.3.0-8c0c2e6-bullseye-mainnet +``` - mina advanced validate-keypair +You can also create a file with the environment variables and pass it to the docker run command with `--env-file` flag. For example, create a file named `mainnet.env` with the following content: -=== flags === +``` + MINA_NETWORK=mainnet + PEERS_LIST_URL=https://bootnodes.minaprotocol.com/networks/mainnet.txt + MINA_ARCHIVE_DUMP_URL=https://storage.googleapis.com/mina-archive-dumps + MINA_GENESIS_LEDGER_URL=https://storage.googleapis.com/o1labs-gitops-infrastructure/mainnet/mainnet.json + BLOCKS_BUCKET=https://storage.googleapis.com/mina_network_block_data +``` - --privkey-path FILE File to write private key into - (public key will be FILE.pub) - (alias: -privkey-path) - [--signature-kind mainnet|testnet|] Signature kind to use (default: - value compiled into this binary) - [-help] print this help text and exit - (alias: -?) +Then run the container with the following command: ``` - -### mina advanced validate-transaction + docker run -it --rm --name rosetta \ + --entrypoint=./docker-start.sh \ + -p 8302:8302 -p 3085:3085 -p 3086:3086 -p 3087:3087 \ + --env-file mainnet.env \ + minaprotocol/mina-rosetta: ``` -Validate the signature on one or more transactions, provided to stdin in rosetta format - mina advanced validate-transaction +Example environment files for public networks can be found in [this](https://github.com/MinaProtocol/mina/blob/develop/src/app/rosetta/scripts) directory of Mina's Rosetta repository. -=== flags === +:::note - [--signature-kind mainnet|testnet|] Signature kind to use (default: - value compiled into this binary) - [-help] print this help text and exit - (alias: -?) +- It can take 20 min to 1 hour for your node to sync +- Port 8302 is the default P2P port and must be exposed to the open internet +- The GraphQL API runs on port 3085 (accessible via localhost:3085/graphql) +- PostgreSQL runs on port 3086 +- Rosetta runs on port 3087 -``` +::: -### mina advanced verify-receipt -``` -Verify a receipt of a sent payment +--- +url: /node-operators/rosetta/samples +--- - mina advanced verify-receipt +# Code Samples -=== flags === +These samples use `curl` and [`jq`](https://jqlang.github.io/jq/) to interact with the Rosetta API. They assume a running Rosetta instance on `localhost:3087`. - --address PUBLICKEY Public-key address of sender - (alias: -address) - --payment-path PAYMENTPATH File to read json version of - verifying payment - (alias: -payment-path) - --proof-path PROOFFILE File to read json version of payment - receipt - (alias: -proof-path) - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--legacy] Use legacy json format (zkapp - command with hashes) - [--token TOKEN_ID] The token ID for the account - (alias: -token) - [-help] print this help text and exit - (alias: -?) +Set these shell variables before running the examples: +```bash +ROSETTA_URL="http://localhost:3087" +NETWORK='{"blockchain":"mina","network":"mainnet"}' ``` -### mina advanced visualization -``` -Visualize data structures special to Mina +All endpoints except `/network/list` require a `network_identifier` parameter. The samples include it in each request body. - mina advanced visualization SUBCOMMAND +:::tip +Replace `mainnet` with `devnet` if you are testing against a devnet instance. +::: -=== subcommands === +--- +url: /node-operators/rosetta/samples/requests +--- - registered-masks Produce a visualization of the registered-masks - transition-frontier Produce a visualization of the transition-frontier - help explain a given subcommand (perhaps recursively) +# Requests and Responses -``` +The Rosetta API specification defines high-level descriptions of request and response objects. Exact JSON layouts differ between blockchains. This page covers Mina-specific objects and shows how to query each endpoint with curl. -### mina advanced vrf -``` -Commands for vrf evaluations +All examples assume the shell variables from the [Code Samples](/node-operators/rosetta/samples) setup. - mina advanced vrf SUBCOMMAND +## Network endpoints -=== subcommands === +List available networks: - batch-check-witness Check a batch of vrf evaluation witnesses read on - stdin. Outputs the verified vrf evaluations (or no vrf - output if the witness is invalid), and whether the vrf - output satisfies the threshold values if given. The - threshold should be included in the JSON for each vrf - as the 'vrfThreshold' field, of format - {delegatedStake: 1000, totalStake: 1000000000}. The - threshold is not checked against a ledger; this should - be done manually to confirm whether threshold_met in - the output corresponds to an actual won block. - batch-generate-witness Generate a batch of vrf evaluation witnesses from - {"globalSlot": _, "epochSeed": _, "delegatorIndex": _} - JSON message objects read on stdin - generate-witness Generate a vrf evaluation witness. This may be used to - calculate whether a given private key will win a given - slot (by checking threshold_met = true in the JSON - output), or to generate a witness that a 3rd - account_update can use to verify a vrf evaluation. - help explain a given subcommand (perhaps recursively) +```bash +curl -s "$ROSETTA_URL/network/list" \ + -H 'Content-Type: application/json' \ + -d '{"metadata":{}}' | jq . +``` +Sample response: + +```json +{"network_identifiers":[{"blockchain":"mina","network":"mainnet"}]} ``` -### mina advanced wrap-key -``` -Wrap a private key into a private key file +You must pass the `network_identifier` object as a parameter to all other endpoints. In Mina's Rosetta implementation, it exists only for the network you run Rosetta for, so this array always contains one object. - mina advanced wrap-key +Get network status (current block height, sync state, peers): -=== flags === +```bash +curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq . +``` - --privkey-path FILE File to write private key into (public key will be - FILE.pub) - (alias: -privkey-path) - [-help] print this help text and exit - (alias: -?) +Get supported options and operation types: +```bash +curl -s "$ROSETTA_URL/network/options" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq . ``` -### mina advanced help -``` -explain a given subcommand (perhaps recursively) +## Block and transaction queries - mina advanced help [SUBCOMMAND] +Fetch a block by index: -=== flags === +```bash +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":1000}}" | jq . +``` - [-expand-dots] expand subcommands in recursive help - [-flags] show flags as well in recursive help - [-recursive] show subcommands of subcommands, etc. - [-help] print this help text and exit - (alias: -?) +Fetch a block by hash: +```bash +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"hash\":\"BLOCK_HASH\"}}" | jq . ``` -## mina ledger -``` -Ledger commands +List pending transactions in the mempool: - mina ledger SUBCOMMAND +```bash +curl -s "$ROSETTA_URL/mempool" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq . +``` -=== subcommands === +## Account queries - currency Print the total currency for each token present in the ledger - contained in the specified file - export Print the specified ledger (default: staged ledger at the best tip). - Note: Exporting snarked ledger is an expensive operation and can - take a few seconds - hash Print the Merkle root of the ledger contained in the specified file - test Testing-only commands - help explain a given subcommand (perhaps recursively) +Query an account balance: +```bash +curl -s "$ROSETTA_URL/account/balance" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"account_identifier\":{\"address\":\"B62qr...\",\"token_id\":\"wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf\"} + }" | jq . ``` -### mina ledger currency +Search for transactions by address: + +```bash +curl -s "$ROSETTA_URL/search/transactions" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"address\":\"B62qr...\"}" | jq . ``` -Print the total currency for each token present in the ledger contained in the specified file - mina ledger currency +Search for a specific transaction by hash: -=== flags === +```bash +curl -s "$ROSETTA_URL/search/transactions" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"transaction_identifier\":{\"hash\":\"CkpY...\"}}" | jq . +``` - --ledger-file LEDGER-FILE File containing an exported ledger - [--plaintext] Use plaintext input or output (default: JSON) - (alias: -plaintext) - [-help] print this help text and exit - (alias: -?) +Derive an account address from a public key: +```bash +curl -s "$ROSETTA_URL/construction/derive" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"public_key\":{\"hex_bytes\":\"PUBLIC_KEY_HEX\",\"curve_type\":\"pallas\"} + }" | jq . ``` -### mina ledger export -``` -Print the specified ledger (default: staged ledger at the best tip). Note: Exporting snarked ledger is an expensive operation and can take a few seconds +## Operation object - mina ledger export STAGED-LEDGER|SNARKED-LEDGER|STAKING-EPOCH-LEDGER|NEXT-EPOCH-LEDGER +In Rosetta terminology, each transaction consists of one or more operations. In Mina's implementation, each operation has: -=== flags === +- `operation_identifier` and `related_operations`: a mandatory index, and optional array of related operations +- `type`: the operation type +- `account`: the account identifier the operation relates to +- `amount`: an object with `value` — a signed number representing the balance change - [--daemon-port HOST:PORT/LOCALHOST-PORT] Client to local daemon - communication. If HOST is omitted, - then localhost is assumed to be - HOST. (examples: 8301, - 154.97.53.97:8301) (default: 8301) - (alias: -daemon-port) - [--plaintext] Use plaintext input or output - (default: JSON) - (alias: -plaintext) - [--state-hash STATE-HASH] State hash, if printing a staged - ledger or snarked ledger (default: - state hash for the best tip) - (alias: -state-hash) - [-help] print this help text and exit - (alias: -?) +
+Sample operation JSON +```json +{ + "operation_identifier": { "index": 2 }, + "related_operations": [{ "index": 1 }], + "type": "payment_receiver_inc", + "account": { + "address": "B62qqJ1AqK3YQmEEALdJeMw49438Sh6zuQ5cNWUYfCgRsPkduFE2uLU", + "metadata": { "token_id": "1" } + }, + "amount": { + "value": "90486110", + "currency": { "symbol": "MINA", "decimals": 9 } + } +} ``` -### mina ledger hash -``` -Print the Merkle root of the ledger contained in the specified file +
- mina ledger hash +:::note +All possible operation types are available from the `/network/options` endpoint. The most common types are `fee_payment`, `payment_source_dec`, and `payment_receiver_inc`. +::: -=== flags === +## Transfer transaction layout - --ledger-file LEDGER-FILE File containing an exported ledger - [--plaintext] Use plaintext input or output (default: JSON) - (alias: -plaintext) - [-help] print this help text and exit - (alias: -?) +A MINA token transfer is represented by three operations (account updates): -``` +1. Decrease fee payer balance (fee payer = sender) +2. Decrease sender balance by the transfer amount +3. Increase receiver balance by the transfer amount -### mina ledger test +
+Sample transfer transaction JSON + +```json +{ + "transaction_identifier": { + "hash": "CkpYVELyYvzbyAwYcnMQryEeQ7Gd6Ws7mZNXpmF5kEAyvwoTiUfbX" + }, + "operations": [ + { + "operation_identifier": { "index": 0 }, + "type": "fee_payment", + "account": { "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", "metadata": { "token_id": "1" } }, + "amount": { "value": "-37000000", "currency": { "symbol": "MINA", "decimals": 9 } } + }, + { + "operation_identifier": { "index": 1 }, + "type": "payment_source_dec", + "account": { "address": "B62qpLST3UC1rpVT6SHfB7wqW2iQgiopFAGfrcovPgLjgfpDUN2LLeg", "metadata": { "token_id": "1" } }, + "amount": { "value": "-58486000", "currency": { "symbol": "MINA", "decimals": 9 } } + }, + { + "operation_identifier": { "index": 2 }, + "related_operations": [{ "index": 1 }], + "type": "payment_receiver_inc", + "account": { "address": "B62qkiF5CTjeiuV1HSx4SpEytjiCptApsvmjiHHqkb1xpAgVuZTtR14", "metadata": { "token_id": "1" } }, + "amount": { "value": "58486000", "currency": { "symbol": "MINA", "decimals": 9 } } + } + ] +} ``` -Testing-only commands - mina ledger test SUBCOMMAND +
-=== subcommands === +This operations array is what you pass to `/construction/preprocess` and `/construction/payloads` when building a transfer. See [Sending Transactions](send-transactions) for the full flow. - apply Test ledger application - generate-accounts Generate a ledger for testing - help explain a given subcommand (perhaps recursively) +--- +url: /node-operators/rosetta/samples/scan-blocks +--- +# Scanning Blocks + +To poll for new blocks, query `/network/status` for the current block height, then fetch each block sequentially. + +Get the current block height: + +```bash +curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq '.current_block_identifier.index' ``` -### mina ledger help +Fetch a specific block by index: + +```bash +BLOCK_INDEX=1000 + +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$BLOCK_INDEX}}" | jq . ``` -explain a given subcommand (perhaps recursively) - mina ledger help [SUBCOMMAND] +A simple polling loop that waits for new blocks: -=== flags === +```bash +HEIGHT=$(curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq '.current_block_identifier.index') - [-expand-dots] expand subcommands in recursive help - [-flags] show flags as well in recursive help - [-recursive] show subcommands of subcommands, etc. - [-help] print this help text and exit - (alias: -?) +while true; do + BLOCK=$(curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$HEIGHT}}") + if echo "$BLOCK" | jq -e '.block' > /dev/null 2>&1; then + echo "Block $HEIGHT:" + echo "$BLOCK" | jq '.block.transactions[] | .transaction_identifier.hash' + HEIGHT=$((HEIGHT + 1)) + else + sleep 10 + fi +done ``` -## mina libp2p -``` -Libp2p commands +--- +url: /node-operators/rosetta/samples/send-transactions +--- - mina libp2p SUBCOMMAND +# Sending Transactions -=== subcommands === +:::info +This flow follows the [Construction API Overview](https://docs.cloud.coinbase.com/rosetta/docs/construction-api-overview) from the official Rosetta documentation. +::: - dump-keypair Print an existing libp2p keypair - generate-keypair Generate a new libp2p keypair and print out the peer ID - help explain a given subcommand (perhaps recursively) +The steps to send a MINA payment: + +1. Derive the account address from a public key +2. Build the unsigned transaction via preprocess → metadata → payloads +3. Sign offline with the [signer tool](/node-operators/mina-signer) +4. Combine the signature into a signed blob +5. Submit the signed transaction + +## Prerequisites +- A key pair generated with the [offline signer tool](/node-operators/mina-signer) +- The account must have a balance (send test funds on devnet first) +- Set the shell variables from the [Code Samples](/node-operators/rosetta/samples) setup + +Set your keys and transfer parameters: + +```bash +PUBLIC_KEY="YOUR_PUBLIC_KEY_HEX" +SENDER="B62q..." +RECEIVER="B62q..." +TOKEN_ID="wSHV2S4qX9jFsLjQo8r1BsMLH2ZRKsZx6EJd1sbozGPieEC4Jf" +FEE=10000000 # 0.01 MINA in nanomina +VALUE=1000000000 # 1 MINA in nanomina ``` -### mina libp2p dump-keypair +## Step 1: Derive account address + +```bash +curl -s "$ROSETTA_URL/construction/derive" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"public_key\":{\"hex_bytes\":\"$PUBLIC_KEY\",\"curve_type\":\"pallas\"} + }" | jq . ``` -Print an existing libp2p keypair - mina libp2p dump-keypair +## Step 2: Build the operations payload -=== flags === +Construct the three operations that represent a MINA transfer (see [Requests and Responses](requests#transfer-transaction-layout) for details on the structure): - --privkey-path FILE File to read private key from - (alias: -privkey-path) - [-help] print this help text and exit - (alias: -?) +```bash +OPERATIONS='[ + { + "operation_identifier":{"index":0}, + "type":"fee_payment", + "account":{"address":"'"$SENDER"'","metadata":{"token_id":"'"$TOKEN_ID"'"}}, + "amount":{"value":"-'"$FEE"'","currency":{"symbol":"MINA","decimals":9}} + }, + { + "operation_identifier":{"index":1}, + "type":"payment_source_dec", + "account":{"address":"'"$SENDER"'","metadata":{"token_id":"'"$TOKEN_ID"'"}}, + "amount":{"value":"-'"$VALUE"'","currency":{"symbol":"MINA","decimals":9}} + }, + { + "operation_identifier":{"index":2}, + "related_operations":[{"index":1}], + "type":"payment_receiver_inc", + "account":{"address":"'"$RECEIVER"'","metadata":{"token_id":"'"$TOKEN_ID"'"}}, + "amount":{"value":"'"$VALUE"'","currency":{"symbol":"MINA","decimals":9}} + } +]' +``` + +## Step 3: Preprocess + +```bash +PREPROCESS=$(curl -s "$ROSETTA_URL/construction/preprocess" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"operations\":$OPERATIONS}") +echo "$PREPROCESS" | jq . ``` -### mina libp2p generate-keypair +## Step 4: Metadata + +```bash +METADATA=$(curl -s "$ROSETTA_URL/construction/metadata" \ + -H 'Content-Type: application/json' \ + -d "$(echo "$PREPROCESS" | jq -c ". + { + \"network_identifier\":$NETWORK, + \"public_keys\":[{\"hex_bytes\":\"$PUBLIC_KEY\",\"curve_type\":\"pallas\"}] + }")") + +echo "$METADATA" | jq . ``` -Generate a new libp2p keypair and print out the peer ID - mina libp2p generate-keypair +## Step 5: Payloads (unsigned transaction) -=== flags === +```bash +PAYLOADS=$(curl -s "$ROSETTA_URL/construction/payloads" \ + -H 'Content-Type: application/json' \ + -d "$(echo "$METADATA" | jq -c ". + { + \"network_identifier\":$NETWORK, + \"operations\":$OPERATIONS + }")") - --privkey-path FILE File to write private key into (public key will be - FILE.pub) - (alias: -privkey-path) - [-help] print this help text and exit - (alias: -?) +echo "$PAYLOADS" | jq . +UNSIGNED_TX=$(echo "$PAYLOADS" | jq -r '.unsigned_transaction') +``` + +## Step 6: Sign offline + +Use the [signer CLI tool](/node-operators/mina-signer#signing-a-transaction-with-signer-cli): +```bash +SIGNATURE=$(signer sign --private-key "$PRIVATE_KEY" --unsigned-transaction "$UNSIGNED_TX") ``` -### mina libp2p help +## Step 7: Combine + +```bash +SIGNING_PAYLOAD=$(echo "$PAYLOADS" | jq -c '.payloads[0]') + +COMBINE=$(curl -s "$ROSETTA_URL/construction/combine" \ + -H 'Content-Type: application/json' \ + -d "{ + \"network_identifier\":$NETWORK, + \"unsigned_transaction\":\"$UNSIGNED_TX\", + \"signatures\":[{ + \"signing_payload\":$SIGNING_PAYLOAD, + \"public_key\":{\"hex_bytes\":\"$PUBLIC_KEY\",\"curve_type\":\"pallas\"}, + \"signature_type\":\"schnorr_poseidon\", + \"hex_bytes\":\"$SIGNATURE\" + }] + }") + +SIGNED_TX=$(echo "$COMBINE" | jq -r '.signed_transaction') +echo "$COMBINE" | jq . ``` -explain a given subcommand (perhaps recursively) - mina libp2p help [SUBCOMMAND] +## Step 8: Get transaction hash -=== flags === +```bash +curl -s "$ROSETTA_URL/construction/hash" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"signed_transaction\":\"$SIGNED_TX\"}" | jq . +``` - [-expand-dots] expand subcommands in recursive help - [-flags] show flags as well in recursive help - [-recursive] show subcommands of subcommands, etc. - [-help] print this help text and exit - (alias: -?) +## Step 9: Submit +```bash +curl -s "$ROSETTA_URL/construction/submit" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"signed_transaction\":\"$SIGNED_TX\"}" | jq . ``` +After submission, you can monitor for confirmation using the [block scanning](scan-blocks) approach — poll blocks until your transaction hash appears. + --- -url: /node-operators/reference/mina-signer +url: /node-operators/rosetta/samples/track-deposits --- -# Mina Signer for Node Operators +# Tracking Deposits -[Mina Signer](/mina-signer) is a NodeJS SDK that allows you to sign strings, payments, and delegations using Mina's key pairs for various specified networks. See the full [Mina Signer documentation](/mina-signer) for installation, API details, and usage with o1js and Rosetta. +To track deposits, scan each block for `payment_receiver_inc` operations matching your deposit address. -## Migration from o1labs/client-sdk to mina-signer +Fetch a block and filter for deposits to a specific address: -The signing library `o1labs/client-sdk` was deprecated some time ago and will stop working after the Mina mainnet upgrade. All users should upgrade to use the [mina-signer](https://www.npmjs.com/package/mina-signer) library. +```bash +DEPOSIT_ADDRESS="B62qr..." +BLOCK_INDEX=1000 + +curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$BLOCK_INDEX}}" \ + | jq --arg addr "$DEPOSIT_ADDRESS" ' + .block.transactions[] + | { + tx_hash: .transaction_identifier.hash, + deposits: [ + .operations[] + | select(.account.address == $addr and .type == "payment_receiver_inc") + | { amount: .amount.value } + ] + } + | select(.deposits | length > 0) + ' +``` -Please keep in mind the following: +A continuous deposit monitoring loop: -1. Make sure to adjust the `nonce` to the correct nonce on the account you want to use as "sender" -1. Update the `url` variable with an existing Mina Node GraphQL endpoint +```bash +DEPOSIT_ADDRESS="B62qr..." -See the [Broadcasting a Signed Payment](/mina-signer#broadcasting-a-signed-payment) section for a complete example of signing and broadcasting a payment via GraphQL. +HEIGHT=$(curl -s "$ROSETTA_URL/network/status" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK}" | jq '.current_block_identifier.index') + +while true; do + BLOCK=$(curl -s "$ROSETTA_URL/block" \ + -H 'Content-Type: application/json' \ + -d "{\"network_identifier\":$NETWORK,\"block_identifier\":{\"index\":$HEIGHT}}") + + if echo "$BLOCK" | jq -e '.block' > /dev/null 2>&1; then + echo "$BLOCK" | jq --arg addr "$DEPOSIT_ADDRESS" ' + .block.transactions[] + | { + tx_hash: .transaction_identifier.hash, + deposits: [ + .operations[] + | select(.account.address == $addr and .type == "payment_receiver_inc") + | { amount: .amount.value } + ] + } + | select(.deposits | length > 0) + ' + HEIGHT=$((HEIGHT + 1)) + else + sleep 10 + fi +done +``` --- url: /node-operators/seed-peers/docker-compose @@ -11780,7 +11233,7 @@ You can use your [Ledger Nano S](https://www.ledger.com/) hardware wallet to sec ### Mina Signer -You can also use [Mina Signer](/node-operators/reference/mina-signer) to generate key pairs and sign transactions. +You can also use [Mina Signer](/node-operators/mina-signer) to generate key pairs and sign transactions. ## Validate your private key