Skip to content

Mina Mainnet & Devnet Stable Hot Fix 4.0.3 Release

Choose a tag to compare

View all tags
@dkijania dkijania released this 23 Oct 12:22
· 585 commits to compatible since this release
4.0.3
73134bf
This commit was signed with the committer’s verified signature.
dkijania Dariusz Kijania
GPG key ID: A2F8DBA4E961F815
Verified
Learn about vigilant mode.

🚨 The o1Labs team has released the 4.0.3 Hotfix for Mainnet & Devnet. 🚨

4.0.3 is a hotfix for the archive node. Since Mainnet block 565083, the archive node skips every block that touches an account whose token symbol is not valid UTF-8 (#19855), so archive databases have gaps from that point. 4.0.3 fixes this. All archive node operators should upgrade. The daemon and Rosetta behave as in 4.0.2, and there is no protocol change.

If you run into any issues, please include Release: 4.0.3 in the name of any issues you create. Please report any issues you encounter here or join the discussion!

What's fixed in 4.0.3?

Archive node: token symbols and zkApp URIs that are not UTF-8

A token symbol or zkApp URI can contain any bytes. The archive node stored them as UTF-8 text, so:

  • a value that is not valid UTF-8 (for example a single 0xff byte) failed with invalid byte sequence for encoding "UTF8", and the archive node skipped the whole block. Every later block that touched the same account was skipped too. On Mainnet this affected blocks 565083, 565084 and 565087.
  • a value with a zero byte was silently cut at that byte.

The archive node now stores these values byte for byte.

Complete Changelog since 4.0.2:

4.0.2...4.0.3

Upgrading the archive node

The 4.0.3 archive node needs the 4.0.3 schema. Do these steps in this order:

  1. Stop the archive node.

  2. Run the 4.0.3 upgrade script against the archive database. The script is in the 4.0.3 archive package and image at /etc/mina/archive/upgrade.sql, and on GitHub: upgrade.sql.

    psql "postgres://USER:PASSWORD@HOST:5432/archive" -f /etc/mina/archive/upgrade.sql

    If your archive node ran with PGCLIENTENCODING=LATIN1 as a workaround for this problem, run the script with the option below, and remove PGCLIENTENCODING from the archive node's environment after the upgrade:

    PGOPTIONS='-c archive.value_encoding=LATIN1' psql "postgres://USER:PASSWORD@HOST:5432/archive" -f /etc/mina/archive/upgrade.sql

  3. Install and start the 4.0.3 archive node.

  4. Fill the skipped blocks. The missing-blocks guardian does this, or add them with mina-archive-blocks --precomputed --archive-uri <URI> <block files>. The precomputed blocks are in the mina_network_block_data bucket, for example https://storage.googleapis.com/mina_network_block_data/mainnet-565083-3NLSAECx1Z9wsk7EWNjtf8LZp4fyHq2ruffw1f8SBU9MTwYAHV6T.json. A missing block is the parent of a block that has no parent link:

    SELECT height - 1, parent_hash FROM blocks
WHERE parent_id IS NULL AND height > (SELECT MIN(height) FROM blocks)
ORDER BY height;

    Repeat until the query returns no rows.

The script can be run again with no effect. It also adds three indexes on user_commands (fee payer, source and receiver). While they are built, block inserts wait. To avoid that wait, create them on the running archive node before step 1:

CREATE INDEX CONCURRENTLY idx_user_commands_fee_payer_id ON user_commands(fee_payer_id);
CREATE INDEX CONCURRENTLY idx_user_commands_source_id    ON user_commands(source_id);
CREATE INDEX CONCURRENTLY idx_user_commands_receiver_id  ON user_commands(receiver_id);

To go back to 4.0.2, stop the archive node and run the 4.0.3 downgrade.sql. It stops without a change if the database holds a token symbol or zkApp URI that the 4.0.2 schema cannot store.

If you ran the upgrade script without archive.value_encoding=LATIN1

This section is only for operators whose archive node ran with PGCLIENTENCODING=LATIN1 before the upgrade, and who then ran upgrade.sql without archive.value_encoding=LATIN1.

In this case the upgrade script stored the Mainnet token symbol 0xff (block 565083) as the two bytes 0xc3 0xbf. The archive node does not report an error, but the data is wrong: the replayer stops at global slot 1005347 with ledger hash differs from expected ledger hash, and the token symbol is not the on-chain value.

The upgrade script cannot be run again to correct this, because the columns are already bytea.

  1. Find the token symbols and zkApp URIs that are not ASCII:

    SELECT 'token_symbols' AS tbl, id, encode(value, 'hex') AS value FROM token_symbols
WHERE encode(value, 'escape') ~ '\\\d{3}'
UNION ALL
SELECT 'zkapp_uris', id, encode(value, 'hex') FROM zkapp_uris
WHERE encode(value, 'escape') ~ '\\\d{3}';

    On Mainnet the only correct result is one token_symbols row with the value ff. If the result is ff, or if there are no rows, your database is correct and you do not have to do more.

  2. If a row has the value c3bf, repair it. The archive node can continue to run:

    UPDATE token_symbols SET value = '\xff'::bytea WHERE value = '\xc3bf'::bytea;

    The statement matches by value, not by id, because the id is different in each database. If you run it again, it changes no rows.

  3. Run the query in step 1 again. The result must be one row with the value ff.

  4. Remove PGCLIENTENCODING from the environment of the archive node and of all other tools that use the archive database (for example the replayer and the missing-blocks guardian). The 4.0.3 archive node stores these values as bytes and does not need it.

If the query in step 1 shows a different value, do not change it. Open an issue with Release: 4.0.3 in the name and include the query result.

Supported environments include Linux (Debian 12, Ubuntu 22.04 LTS & Ubuntu 24.04 LTS), and any host machine with Docker.

🚨NOTE - Ubuntu 20.04 (focal) and Debian 11 (bullseye) have reached EOL and are no longer supported. This release ships no packages or images for them.🚨

Upgrading & Connecting to Mainnet

Debian Packages:

Debian Repository: First, set up and update the signed stable Debian Repository for your platform with the following commands:

sudo rm -f /etc/apt/sources.list.d/mina*.list
curl -fsSL https://stable.apt.packages.minaprotocol.com/repo-signing-key.asc | sudo gpg --dearmor -o /usr/share/keyrings/mina.gpg
echo "deb [signed-by=/usr/share/keyrings/mina.gpg] https://stable.apt.packages.minaprotocol.com $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/mina.list
sudo apt-get update

The unsigned packages.o1test.net repository is still supported and also carries 4.0.3. The signed repository is preferred, because apt verifies the package signatures. If you keep using packages.o1test.net, set it up with:

sudo rm -f /etc/apt/sources.list.d/mina*.list
echo "deb [trusted=yes] http://packages.o1test.net $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/mina.list
sudo apt-get update

Then, install the package(s) that you need:

Mina Archive: sudo apt-get install --allow-downgrades -y mina-archive-mainnet=4.0.3-73134bf
Mina Daemon: sudo apt-get install --allow-downgrades -y mina-mainnet=4.0.3-73134bf
Mina Rosetta: sudo apt-get install --allow-downgrades -y mina-rosetta-mainnet=4.0.3-73134bf
Mina Archive Schema: create_schema.sql
Config File: /var/lib/coda/config_73134bfd.json

Docker Images:

Every docker image is built for each of the supported platforms, replace CODENAME in the images below with your choice of jammy, noble, or bookworm to select the base docker image.

Mina Archive: minaprotocol/mina-archive:4.0.3-73134bf-CODENAME-mainnet
Mina Daemon: minaprotocol/mina-daemon:4.0.3-73134bf-CODENAME-mainnet
Mina Rosetta: minaprotocol/mina-rosetta:4.0.3-73134bf-CODENAME-mainnet

Step by Step Guide:

Docker only Make sure to run with:

--peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt or add PEER_LIST_URL=https://bootnodes.minaprotocol.com/networks/mainnet.txt to your .mina-env.

Run mina daemon with:

mina daemon --peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt

Check out our documentation for complete instructions on using this version to connect to Mainnet.

If you are running the correct version on the correct network MAINNET, mina client status will show:

Chain id:     0718f61ab88f9d0fa643ff4dc3a3d5998dd6d51a6008b2b0339b0dbb26133886
Git SHA-1:    73134bfd3a43c3f4d8ee564b3d04a0c6ed2e0b3c

Upgrading & Connecting to Devnet

Use the same Debian repository setup as above, then install:

Mina Archive: sudo apt-get install --allow-downgrades -y mina-archive-devnet=4.0.3-73134bf
Mina Daemon: sudo apt-get install --allow-downgrades -y mina-devnet=4.0.3-73134bf
Mina Rosetta: sudo apt-get install --allow-downgrades -y mina-rosetta-devnet=4.0.3-73134bf
Config File: /var/lib/coda/config_73134bfd.json

Docker Images:

Mina Archive: minaprotocol/mina-archive:4.0.3-73134bf-CODENAME-devnet
Mina Daemon: minaprotocol/mina-daemon:4.0.3-73134bf-CODENAME-devnet
Mina Rosetta: minaprotocol/mina-rosetta:4.0.3-73134bf-CODENAME-devnet

Run mina daemon with:

mina daemon --peer-list-url https://bootnodes.minaprotocol.com/networks/devnet.txt

If you are running the correct version on the correct network DEVNET, mina client status will show:

Chain id:     ebfce0d570bc22eb041e1a7b0a46bbc3ed5d0a1030ef2ab5e36eef908b93eba8
Git SHA-1:    73134bfd3a43c3f4d8ee564b3d04a0c6ed2e0b3c
Assets 2
Loading