bug: nonces in the archive db are wrong or missing

[michal0mina]

Problem

nonces in some archive tables are wrong or missing.

Acceptance Criteria

we need to provide a way for community fix data on their archive nodes.

Tasks

• figure out a user friendly deliverable
• communicate to our users

[michal0mina]

Notes from a call:

1. Investigate how to utilize logs to identify erratic tables and row

2. If ^^ doesn't work then figure out db diff. 3 party tools, bash diff between tables, or SQL query between two database in one postgres instance

3. How to decide what is good final database to do replayer against it. Sync with O1 -- to see when they deploy fix and the dumps start to be correct. This will be good timestamp/slot to correct as last one.

4. Replayer is not viable option for 3rd party users (large memory footprint, 12-30+h execution, automation needed)

[simisimis]

I have documented a way to run a replayer and get those nonces fixed. As well as how users should upgrade their current running databases. Which was simplified down basically to two steps:
restarting archive node with a newest mina-archive version and running an sql patch against their postgres database.
I informed our colleagues responsible to communicate those details to our community users.

[jrwashburn]

Where should one look for the communication and details on this?
Would this issue exist on a new database built from scratch by reloading precomputed blocks?

[simisimis]

the bug fix has been merged in this pr #13406
however it's still in alpha release #14943 and hasn't made it's way to a public mainnet release.
Once that fix will be tested and rolled out in mainnet release, documentation and communication will follow.
At this point date of the release is still not known, but it's supposed to happen before HF when users will need to run mina replayer for archive node trustless migration path.