Skip to content

fix(pusher): prevent unbounded audiobuffer growth#4826

Merged
beastoin merged 4 commits intomainfrom
fix/pusher-audiobuffer-leak
Feb 15, 2026
Merged

fix(pusher): prevent unbounded audiobuffer growth#4826
beastoin merged 4 commits intomainfrom
fix/pusher-audiobuffer-leak

Conversation

@beastoin
Copy link
Copy Markdown
Collaborator

Summary

  • Guard audiobuffer.extend() and trigger_audiobuffer.extend() — only accumulate when there's a consumer
  • trigger_audiobuffer only extends when has_audio_apps_enabled
  • audiobuffer only extends when audio_bytes_webhook_delay_seconds is set
  • Without this guard, both bytearrays grow ~16KB/s indefinitely for users with no audio apps (~57MB/hour)

Part of #4825 (Fix 1/3). Follow-up to PR #4784.

Test plan

  • Verify users WITH audio apps still receive audio bytes triggers
  • Verify users WITHOUT audio apps don't accumulate audiobuffer memory
  • Verify webhook audio bytes still work when audio_bytes_webhook_delay_seconds is set
  • Monitor pusher pod memory after deploy

🤖 Generated with Claude Code

…udio apps

Only accumulate audio into trigger_audiobuffer when has_audio_apps_enabled,
and into audiobuffer when audio_bytes_webhook_delay_seconds is set. Without
this guard, both bytearrays extend() on every audio chunk but never get
cleared, growing ~16KB/s indefinitely (~57MB/hour per connection).

Found during deep memory leak audit (follow-up to PR #4784).
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request aims to fix an unbounded memory growth issue in the audio buffers by adding guards to only accumulate audio data when there is an active consumer. The approach is generally correct, but a critical issue was found with the implementation for webhook audio buffers, where a 0-second delay configuration is incorrectly handled, effectively disabling the feature for that case. The review includes a suggestion to fix this logic.

Comment thread backend/routers/pusher.py Outdated
Comment on lines +388 to +389
if audio_bytes_webhook_delay_seconds:
audiobuffer.extend(audio_data)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This check is not quite right. If audio_bytes_webhook_delay_seconds is 0, this condition will evaluate to False, and audiobuffer will never be extended. This would break the feature for users who want immediate webhook triggers with a 0-second delay.

To fix this, you should check for None explicitly.

Additionally, the corresponding condition for sending the buffer around line 430 also uses a truthiness check (if audio_bytes_webhook_delay_seconds and ...) and will fail for a 0-second delay. That should also be updated to if audio_bytes_webhook_delay_seconds is not None and ... to ensure correct behavior.

Suggested change
if audio_bytes_webhook_delay_seconds:
audiobuffer.extend(audio_data)
if audio_bytes_webhook_delay_seconds is not None:
audiobuffer.extend(audio_data)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 81c10df. Changed both the audiobuffer.extend() guard (line 280) and the downstream send condition (line 430) from truthiness check to is not None:

# Guard: accumulate if webhook is configured (including delay=0)
if audio_bytes_webhook_delay_seconds is not None:
    audiobuffer.extend(audio_data)

# Send: trigger when buffer exceeds threshold (works for delay=0 → immediate)
if (
    audio_bytes_webhook_delay_seconds is not None
    and len(audiobuffer) > sample_rate * audio_bytes_webhook_delay_seconds * 2
):

@beastoin
Copy link
Copy Markdown
Collaborator Author

Chaos Engineering Test Results — Audiobuffer Leak

Test: 30s of continuous header-101 audio chunks with has_audio_apps_enabled=False and audio_bytes_webhook_delay_seconds=0 (user with no audio apps).

Metric Vulnerable (main) Fixed (this PR)
Max audiobuffer 9,536,000 bytes (9.5MB) 0 bytes
Growth rate 317,758 bytes/sec (~18.6MB/min) 0 bytes/sec
Max trigger_audiobuffer 9,536,000 bytes 0 bytes

Verdict: PASS — Vulnerable grows linearly, fixed stays at zero.

Reproducer:

cd backend/testing/chaos-audiobuffer/
pip install fastapi uvicorn websockets
./run_chaos_test.sh

Test harness at backend/testing/chaos-audiobuffer/ — standalone, no Docker needed.

Kelvin (AI Agent) and others added 3 commits February 15, 2026 04:31
Per Gemini review: a delay of 0 seconds means "immediate trigger", not
"disabled". Truthiness check would incorrectly treat 0 as falsy, breaking
the feature for users with 0-delay webhook config.

Also fix the downstream send condition at line 430 for consistency.
13 tests covering all guard condition combinations:
- has_audio_apps_enabled True/False x delay None/0/positive
- Webhook send threshold: None never sends, 0 sends immediately
- Repeated accumulation and zero-growth verification

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- test_delay_zero_immediate_flush_cycle: 10 chunks arrive, each flushes
  immediately (threshold=0), buffer stays at 0 after each cycle
- test_positive_delay_accumulates_before_flush: contrast showing delay=5
  accumulates to >80KB before flushing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@beastoin beastoin merged commit 23a57c5 into main Feb 15, 2026
1 check passed
@beastoin beastoin deleted the fix/pusher-audiobuffer-leak branch February 15, 2026 12:26
@beastoin
Copy link
Copy Markdown
Collaborator Author

lgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant