F3 Test with chain exchange progresses too fast

[masih]

The top level F3 tests us mock clock and advance the EC clock at a high rate. This surfaces a computational bottleneck on CI where the computationally heavy tasks would struggle to keep up on CI runner hardware, resulting in extended time to test completion.

This issue is specifically visible when tests are run with race detector.
To overcome this, the maximum chain length in tests was reduced to 30 (from 100). It seems that additional computation optimisations in Merkle Tree root calculation are still not sufficient to get the tests to pass consistently on CI.

Actionables:

• Try various EC progress time dilution and timeouts in tests to continue to use long chains but also have the tests pass consistently.
• Time-box the effort.

Relates to: #935 (comment)

[Kubuxu]

The instances are progressing with base decision. Both nodes propose consistent small chunks of the chain but by PREPARE they both propose base, which suggests that chain didn't propagate in time.

[BigLep]

What are the next steps here? Is this a priority to handle before f3 activation?

[masih]

Is this a priority to handle before f3 activation?

No.

What are the next steps here?

Defer this to after F3 activation and approach again at the time of post activation code cleanup.

[BigLep]

2025-05-02 coversation: this is a timeboxed task. It should be easier now because there are no dynamic manifests anymore and that's where the racy tests were.