Skip to content

v1.3.8 - Fix parallel chunk sending for large log files

Choose a tag to compare

@rindhuja-johnson rindhuja-johnson released this 23 Oct 10:31
· 31 commits to main since this release

What's Changed

  • Fix Issue #28: Enable parallel chunk sending for large log files
    • All chunks now reach the backend (not just chunk 0)
    • Parallel sending with 0.1s delays to prevent WebSocket overload
    • Backend aggregation support with thread_id grouping
    • Improved reliability for chunked log transmission

Technical Details

When sending large log files (>2.5MB) that require chunking, the previous implementation only sent the first chunk. This release implements parallel chunk sending:

Changes Made

File: scripts/agent_cli.py

  1. Added _wait_for_event_type() method for backend event acknowledgment
  2. Modified _send_chunked_logs() to send all chunks in parallel
  3. Small 0.1s delay between sends to avoid overwhelming WebSocket
  4. Waits for agent_aggregation_complete event after all chunks sent

Benefits

✅ Fast: Parallel sending instead of sequential waiting
✅ Reliable: All chunks reach the backend
✅ Safe: Delays prevent WebSocket overload
✅ Minimal backend changes required

Installation

pip install --upgrade netra-zen

Verification

Test the parallel chunk sending with large log files:

# Create a test file >2.5MB
zen --apex --send-logs --message 'test chunking'

Full Changelog: v1.3.7...v1.3.8