fix: poll in reconnect test

[bradleystachurski]

reconnect_test is flakey, which can likely be resolved by polling.

https://github.com/fedimint/fedimint/actions/runs/7747755393/job/21128923550#step:5:1433

2024-02-01T21:34:35.3311143Z 00:00:30 Error: command: /nix/store/fdbrlr3n4n94gs5a1lgnzskhgmv8pimq-fedimint-cli/bin/fedimint-cli --data-dir=/tmp/nix-shell.qkvd7G/devimint-73588/cl
ients/default dev api module_2_block_count
2024-02-01T21:34:35.3312020Z 00:00:30
2024-02-01T21:34:35.3312197Z 00:00:30 Caused by:
2024-02-01T21:34:35.3312415Z 00:00:30     exit status: 1
2024-02-01T21:34:35.3312648Z 00:00:30     stdout:
2024-02-01T21:34:35.3312845Z 00:00:30
2024-02-01T21:34:35.3313047Z 00:00:30     stderr:
2024-02-01T21:34:35.3313661Z 00:00:30     2024-02-01T21:34:34.877575Z  INFO fedimint_cli::db_locked: Acquiring database lock
2024-02-01T21:34:35.3314777Z 00:00:30     2024-02-01T21:34:35.074923Z  INFO fedimint_client: Last client reference dropped, shutting down client task group
2024-02-01T21:34:35.3315862Z 00:00:30     2024-02-01T21:34:35.077403Z  INFO fedimint_client::sm::executor: Starting state machine executor task
2024-02-01T21:34:35.3317015Z 00:00:30     2024-02-01T21:34:35.081059Z  INFO fedimint_client::sm::executor: Shutting down state machine executor runner due to shutdown signal
2024-02-01T21:34:35.3317705Z 00:00:30     {
2024-02-01T21:34:35.3317921Z 00:00:30       "error": "CliError",
2024-02-01T21:34:35.3318208Z 00:00:30       "kind": "general_failure",
2024-02-01T21:34:35.3319757Z 00:00:30       "message": "Federation rpc error {general => Received errors from 1 peers: peer-0: Rpc error: Networking or low-level protocol error:
Error when opening the TCP socket: Connection refused (os error 111)), 0 => Rpc error: Networking or low-level protocol error: Error when opening the TCP socket: Connection refus
ed (os error 111)), }",
2024-02-01T21:34:35.3322451Z 00:00:30       "raw_error": "Federation rpc error {general => Received errors from 1 peers: peer-0: Rpc error: Networking or low-level protocol error
: Error when opening the TCP socket: Connection refused (os error 111)), 0 => Rpc error: Networking or low-level protocol error: Error when opening the TCP socket: Connection ref
used (os error 111)), }"
2024-02-01T21:34:35.3323774Z 00:00:30     }
2024-02-01T21:34:35.3323966Z 00:00:30
2024-02-01T21:34:35.3324168Z ## FAILED: reconnect_test

[codecov]

Codecov Report

Attention: 14 lines in your changes are missing coverage. Please review.

Comparison is base (df9b41a) 57.94% compared to head (a3d82e9) 57.99%.

Files	Patch %	Lines
devimint/src/federation.rs	0.00%	13 Missing ⚠️
devimint/src/tests.rs	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4220      +/-   ##
==========================================
+ Coverage   57.94%   57.99%   +0.05%     
==========================================
  Files         197      197              
  Lines       43413    43414       +1     
==========================================
+ Hits        25154    25177      +23     
+ Misses      18259    18237      -22     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dpc]

Thinking aloud: should this be just part of await_all_peers? Otherwise won't we hit it soon someplace else using it?

I guess if someone is using await_all_peers ... they can kind of expect some peers to be down, and don't want this function to fail because of it?

[bradleystachurski]

That's an excellent observation 🙂