fix(miner): Avoid some duplicate block errors

[teor2345]

Motivation

Sometimes one of Zebra's miners will:

• submit a valid block
• restart mining on the same template
• submit a duplicate of that block

This happens when a valid solution is mined quickly, before the block has time to validate, and the template has time to refresh.

PR Author Checklist

Check before marking the PR as ready for review:

• Will the PR name make sense to users?
• Does the PR have a priority label?
• Have you added or updated tests?
• Is the documentation up to date?

If a checkbox isn't relevant to the PR, mark it as done.

Complex Code or Requirements

This is a pretty standard concurrency bug.

Solution

• Decrease the time that the template generator waits between RPC calls to 2 seconds
• When a miner is successful, make it wait for 3 seconds before restarting mining, for block verification and RPC template generation (this time can be increased if the bug still happens)
• When a block fails, start re-mining after the standard 2 second rate-limit
• Define a constant for the standard RPC and miner rate-limit, and another constant for the wait time after mining a block

Testing

I've tested this manually and it's no worse that the current code.

Review

This is a routine fix.

Reviewer Checklist

Check before approving the PR:

• Does the PR scope match the ticket?
• Are there enough tests to make sure it works? Do the tests cover the PR motivation?
• Are all the PR blockers dealt with?
  PR blockers can be dealt with in new tickets or PRs.

And check the PR Author checklist is complete.

Follow Up Work

Adjust the constants as needed.

[teor2345]

Waiting for SSH key to propagate.
ssh: connect to host 34.139.23.104 port 22: Connection refused
ERROR: (gcloud.compute.ssh) Could not SSH into the instance. It is possible that your SSH key has not propagated to the instance yet. Try running this command again. If you still cannot connect, verify that the firewall and instance are set to accept ssh traffic.

https://github.com/ZcashFoundation/zebra/actions/runs/7495982068/job/20407505648?pr=8150#step:13:117

Let's restart this when the other jobs finish.

[teor2345]

I manually tested this and there weren't any duplicate blocks after 2 hours of mining on 2 threads.

There is the occasional duplicate indexes error from equihash, but that might be a known issue. Or maybe the latest C code fixes will fix it.