Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[base node] server stops sending pings after being banned for a while #6356

Open
hansieodendaal opened this issue May 27, 2024 · 0 comments
Open

[base node] server stops sending pings after being banned for a while #6356

hansieodendaal opened this issue May 27, 2024 · 0 comments

Comments

@hansieodendaal
Copy link
Contributor

  • only 2 archival nodes on the network, one with a full blockchain (server 5b4ec333590e56ab), and the other brand new (client 12cef37a6734972c)
  • header sync completed successfully
  • block sync fails, at some point, sync peer is banned
  • ban lifted form syn peer
  • pings only received at the server from the client; server stops sending pings

client: 12cef37a6734972c

2024-05-18 06:59:27.957387500 [p2p::services::liveness] DEBUG Initializing Liveness Service
2024-05-18 06:59:27.957405900 [p2p::services::liveness] DEBUG Liveness service initialized
2024-05-18 06:59:27.957723600 [p2p::services::liveness] DEBUG Liveness service started
2024-05-18 06:59:27.957810600 [p2p::services::liveness] DEBUG Config = LivenessConfig { auto_ping_interval: Some(30s), num_peers_per_round: 8, monitored_peers: [], max_allowed_ping_failures: 2 }
2024-05-18 06:59:57.967141100 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available
2024-05-18 07:00:27.981142400 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available
2024-05-18 07:00:57.991157500 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available
2024-05-18 07:01:28.007171300 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available
2024-05-18 07:01:43.515904300 [p2p::services::liveness] DEBUG Received ping from peer '5b4ec333590e56ab' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:01:59.060008700 [p2p::services::liveness] DEBUG Received pong from peer '5b4ec333590e56ab' with useragent 'tari/basenode/1.0.0-pre.13'. Latency: 1.05s (Trace: Tag#0)
2024-05-18 07:02:17.115073600 [p2p::services::liveness] DEBUG Received ping from peer '5b4ec333590e56ab' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:02:29.264599000 [p2p::services::liveness] DEBUG Received pong from peer '5b4ec333590e56ab' with useragent 'tari/basenode/1.0.0-pre.13'. Latency: 1.25s (Trace: Tag#0)
2024-05-18 07:02:42.563130500 [p2p::services::liveness] DEBUG Received ping from peer '5b4ec333590e56ab' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:03:00.446998100 [p2p::services::liveness] DEBUG Received pong from peer '5b4ec333590e56ab' with useragent 'tari/basenode/1.0.0-pre.13'. Latency: 2.43s (Trace: Tag#0)
2024-05-18 07:03:12.540626300 [p2p::services::liveness] DEBUG Received ping from peer '5b4ec333590e56ab' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:03:28.020882200 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available
2024-05-18 07:03:58.035208100 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available

server: 5b4ec333590e56ab

2024-05-18 07:01:09.038529600 [p2p::services::liveness] DEBUG Initializing Liveness Service
2024-05-18 07:01:09.038584100 [p2p::services::liveness] DEBUG Liveness service initialized
2024-05-18 07:01:09.038864700 [p2p::services::liveness] DEBUG Liveness service started
2024-05-18 07:01:09.038906500 [p2p::services::liveness] DEBUG Config = LivenessConfig { auto_ping_interval: Some(30s), num_peers_per_round: 8, monitored_peers: [], max_allowed_ping_failures: 2 }
2024-05-18 07:01:41.081928600 [p2p::services::liveness] DEBUG Received pong from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13'. Latency: 2.04s (Trace: Tag#0)
2024-05-18 07:01:55.530637300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:02:14.895885500 [p2p::services::liveness] DEBUG Received pong from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13'. Latency: 5.84s (Trace: Tag#0)
2024-05-18 07:02:25.625854900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:02:40.410281300 [p2p::services::liveness] DEBUG Received pong from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13'. Latency: 1.35s (Trace: Tag#0)
2024-05-18 07:02:56.294614300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:03:10.308023900 [p2p::services::liveness] DEBUG Received pong from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13'. Latency: 1.25s (Trace: Tag#0)
2024-05-18 07:03:39.072122700 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available
2024-05-18 07:04:09.085967000 [p2p::services::liveness] INFO  Cannot broadcast pings because there are no broadcast peers available
2024-05-18 07:04:25.700153500 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:04:56.428225200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:05:26.952606900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:05:55.677214500 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:06:25.843844800 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:06:55.720519400 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:07:25.679809200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:07:55.870699300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:08:25.739865600 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:08:55.569213600 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:09:26.037617800 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:09:55.766148200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:10:26.247720900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:10:55.951007900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:11:26.206322200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:11:56.038496000 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:12:25.944233900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:12:56.251600200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:13:26.077170500 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:13:56.569614800 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:14:27.669905200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:14:59.454036700 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:15:25.659257300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:15:56.059131800 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:16:26.151083600 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:16:55.906240600 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:17:25.970814200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:17:55.829043000 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:18:25.854758300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:18:55.860832300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:19:27.296488400 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:19:56.633237100 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:20:25.900764900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:20:55.964073100 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:21:26.683671800 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:21:55.864751900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:22:25.861881700 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:22:56.389088300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:23:25.788112600 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:23:55.947056900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:24:25.820582200 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:24:55.937895000 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:25:26.066462300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:25:56.101932300 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:26:25.990672600 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:26:55.965607400 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:27:26.008223600 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:27:56.205051100 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:28:26.143176900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:28:57.904321700 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:29:30.386172900 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:29:57.363768800 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
2024-05-18 07:30:29.046293400 [p2p::services::liveness] DEBUG Received ping from peer '12cef37a6734972c' with useragent 'tari/basenode/1.0.0-pre.13' (Trace: Tag#0)
@hansieodendaal hansieodendaal mentioned this issue May 27, 2024
Merged
4 tasks
SWvheerden pushed a commit that referenced this issue May 27, 2024
@hansieodendaal
feat: add default message tag (#6355)
ef387d7 
Description
---
Add a default message tag to `FinalSendMessageParams`. This enables the
tracing of ping-pong and other messages across nodes.

Motivation and Context
---
- We have a related issue with ping-pong, see #6356. This tracing will
assist in analysing the issue.
- With transaction testing, we noticed that some messages have a zero
tag; this will also help that tracing.

How Has This Been Tested?
---
System-level tests.

What process can a PR reviewer use to test or verify this change?
---
Review code.

<!-- Checklist -->
<!-- 1. Is the title of your PR in the form that would make nice release
notes? The title, excluding the conventional commit
tag, will be included exactly as is in the CHANGELOG, so please think
about it carefully. -->


Breaking Changes
---

- [x] None
- [ ] Requires data directory on base node to be deleted
- [ ] Requires hard fork
- [ ] Other - Please specify

<!-- Does this include a breaking change? If so, include this line as a
footer -->
<!-- BREAKING CHANGE: Description what the user should do, e.g. delete a
database, resync the chain -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@hansieodendaal