Skip to content

Conversation

@Hugal31
Copy link
Contributor

@Hugal31 Hugal31 commented Apr 9, 2025

When loosing the connection to a peer, if the send buffer happens to fill up before the keep alive timeout, the tx_task can be stuck in the send_batch operation. On my computer, it can be stuck for 15 minutes before returning with EHOSTUNREACH ("No route to host").

Now this is not that bat itself, it's just a dangling task, TransportLinkUnicastUniversal and TransportUnicastUniversal. But this make the race condition mentionned in #1886 very likely: if the peer reconnect within those 15 minutes, the reconnection is bogus.

This "fix" make TransportLinkUnicastUniversal::close() not wait for the rx and tx task before returning. It doesn't seem to cause any issue. I didn't do the same in TransportUnicastLowlatency because I am not sure it is necessary.

Now I am not really happy with this hack, but I would like to have a solution for #1886 and it's a bit too involved for me alone.

An alternative would be to watch for the cancellation token while calling send_batch.

@github-actions
Copy link

github-actions bot commented Apr 9, 2025

PR missing one of the required labels: {'dependencies', 'internal', 'new feature', 'enhancement', 'documentation', 'bug', 'breaking-change'}

@codecov
Copy link

codecov bot commented Apr 9, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.20%. Comparing base (1a10597) to head (2a2f45e).
Report is 57 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1896      +/-   ##
==========================================
+ Coverage   71.18%   71.20%   +0.01%     
==========================================
  Files         364      364              
  Lines       65647    65647              
==========================================
+ Hits        46733    46741       +8     
+ Misses      18914    18906       -8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@josephduchesne
Copy link

josephduchesne commented Jun 4, 2025

I'm running into this on systems using Wifi Halow I believe (out at the edge of connectivity ~1.1km in my case). Is there any way to detect it happening and force a reconnect? I'd rather not have to add watchdogs to all systems using zenoh if I can avoid it.

@Hugal31
Copy link
Contributor Author

Hugal31 commented Jun 5, 2025

I'm running into this on systems using Wifi Halow I believe (out at the edge of connectivity ~1.1km in my case). Is there any way to detect it happening and force a reconnect? I'd rather not have to add watchdogs to all systems using zenoh if I can avoid it.

Not that I am aware of. And nothing would prevent the bug from happening again right after...

I'm glad (and sorry!) to hear we're not the only ones with this bug. Could you add some logs, details (or just a thumbsup) on #1886?

@Hugal31
Copy link
Contributor Author

Hugal31 commented Jun 5, 2025

I noticed this hack caused issues elsewhere (i.e. it made an unrelated keep_alive TX loop miss deadline for some reasons). So that's not an option.

@Hugal31 Hugal31 closed this Jun 5, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants