Skip to content

fix(updates): prevent deadlock when relaying channel difference - #1716

Merged
ernado merged 1 commit into
mainfrom
fix/updates-channel-deadlock
Jun 2, 2026
Merged

fix(updates): prevent deadlock when relaying channel difference#1716
ernado merged 1 commit into
mainfrom
fix/updates-channel-deadlock

Conversation

@ernado

@ernado ernado commented Jun 2, 2026

Copy link
Copy Markdown
Member

Problem

A circular wait can hang update processing between the two worker goroutines:

  • The internalState worker (main Run loop) is the only reader of internalQueue. While processing a batch in applyCombinedhandleChannelchannelState.Push, it blocks writing into a channel's bounded updates queue (cap 10) when full.
  • The channelState worker, inside getDifference, blocks writing diff.OtherUpdates into s.out (= internalQueue, cap 10) when full — and it's full precisely because the main worker is stuck above and not draining it.

Both selects only release on ctx.Done(), so updates stop flowing until shutdown.

The same hazard is already mitigated in the diffTimeout wait branch of getDifference ("Ignoring updates to prevent *internalState worker from blocking"), but the actual s.out <- hand-off had no such protection.

Fix

Add channelState.sendOut, which keeps draining the channel's own input queue while waiting for the worker to accept the hand-off. This guarantees the main worker can always complete its Push and return to its select to drain internalQueue, so the cycle can no longer form.

This reuses the existing drain-and-ignore pattern; dropped updates are recovered by a future getChannelDifference call (or the idle timeout), so there is no permanent loss.

Safety

  • getDifference/sendOut run only on the single channelState.Run goroutine, so there is no concurrent read of s.updates and no new data race (verified with -race).
  • s.out now has exactly one send site.
  • The only new behavior — possibly dropping concurrent channel updates during the hand-off — is identical in kind to the pre-existing diffTimeout drain and is recoverable via the gap mechanism.

🤖 Generated with Claude Code

The internalState worker is the only reader of internalQueue, yet it can
block in applyCombined -> handleChannel -> channelState.Push while writing
into a channel's bounded updates queue. Meanwhile the channelState worker
blocks in getDifference writing diff.OtherUpdates into internalQueue, which
is full precisely because the internalState worker is stuck. Both selects
only release on ctx.Done(), so update processing hangs until shutdown.

Add channelState.sendOut, which keeps draining the channel's own input queue
while waiting for the worker to accept the hand-off, guaranteeing the worker
can always complete its Push and return to drain internalQueue. This reuses
the same drain-and-ignore pattern already used in the diffTimeout wait;
dropped updates are recovered by a future getChannelDifference call.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.66%. Comparing base (5214d45) to head (daca24f).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
telegram/updates/state_channel.go 0.00% 16 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1716      +/-   ##
==========================================
- Coverage   69.74%   69.66%   -0.08%     
==========================================
  Files         464      464              
  Lines       20216    20234      +18     
==========================================
- Hits        14100    14097       -3     
- Misses       5064     5084      +20     
- Partials     1052     1053       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ernado
ernado merged commit 8387942 into main Jun 2, 2026
14 checks passed
@ernado
ernado deleted the fix/updates-channel-deadlock branch June 2, 2026 21:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant