Stabilize FlatKV crash recovery CI test

[masih]

The verify_flatkv_crash_recovery.sh was treating survivor progress while sei-node-3 remained down as a hard FlatKV recovery invariant. The CI flake showed pre_kill=360 during_kill=360: the SIGKILL landed after the victim had participated in the next Tendermint height, and the devnet did not commit another block until the validator was restarted. That temporary consensus stall is orthogonal to FlatKV crash recovery.

Make the no-progress-while-down case diagnostic instead of fatal, then restart the victim and require every validator to advance past pre_kill + COMPARE_BUFFER before selecting the FlatKV comparison height. This keeps the storage check just as strong: the digest comparison is still over a height committed after the SIGKILL/restart cycle by all validators, so corruption or divergence introduced by FlatKV crash recovery will still fail the test.

Flaked in unrelated changes here.

[github-actions]

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
✅ passed	✅ passed	✅ passed	✅ passed	May 18, 2026, 3:59 PM

[cursor]

PR Summary

Low Risk
Low risk: changes are confined to the verify_flatkv_crash_recovery.sh CI integration test logic, loosening a flaky liveness assertion while keeping the post-crash state divergence check.

Overview
Stabilizes the FlatKV crash-recovery CI contract test by no longer treating “no blocks produced while the killed validator is down” as a hard failure, since Tendermint may temporarily stall depending on when the SIGKILL lands.

After restarting the killed node and confirming catch-up, the test now waits for all validators to advance past pre_kill + COMPARE_BUFFER + 1 before selecting the comparison height, ensuring the FlatKV digest comparison is taken from a height committed after the crash/restart cycle.

^{Reviewed by Cursor Bugbot for commit 2a14b78. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.27%. Comparing base (adf03c1) to head (2a14b78).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3457      +/-   ##
==========================================
+ Coverage   59.24%   59.27%   +0.02%     
==========================================
  Files        2126     2126              
  Lines      175715   175699      -16     
==========================================
+ Hits       104097   104137      +40     
+ Misses      62538    62480      -58     
- Partials     9080     9082       +2     

Flag	Coverage Δ
sei-db	70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 38 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.