Bound CI-V reassembly buffer to prevent OOM on a terminator-less stream#649
Merged
Merged
Conversation
…am (#649) CivFrameSplitter carried every byte after the last 0xFD terminator into the next read as the remainder, with no upper bound. A CI-V stream that never delivers a terminator -- a rig on the wrong baud, serial line noise, or a misbehaving network peer streaming garbage -- grew each rig's dataBuffer by every received byte until the app ran out of memory. This hit all CI-V rigs (IcomRig, XieGuRig, XieGu6100Rig), which share this reassembler and store result.remainder back into dataBuffer. Bound the remainder in the one shared place it is produced. When it exceeds a hard cap (1024 bytes, far above any real CI-V command), resynchronise to the most recent FE FE preamble -- the earliest point a valid command could still begin -- dropping the unparseable bytes before it (they can never complete a frame, since a valid one would have ended in 0xFD and already been split off). With no preamble present, keep at most a trailing 0xFE in case one is split across the read boundary; a final trailing-window clamp guarantees boundedness in the pathological lone-preamble case. Behaviour is byte-for-byte identical for all well-formed and normally-fragmented traffic. Added pure-JVM CivFrameSplitterTest cases: a terminator-less stream stays bounded across 100 callbacks (fails before the fix), overflow resyncs to the last preamble and the frame still completes on the next chunk, no-preamble overflow keeps only a trailing 0xFE, pure noise is dropped, a lone preamble with a huge tail is hard-capped, and a large-but-under-cap partial is unchanged. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## dev #649 +/- ##
============================================
+ Coverage 36.73% 36.81% +0.08%
- Complexity 197 207 +10
============================================
Files 216 218 +2
Lines 26888 27012 +124
Branches 3287 3315 +28
============================================
+ Hits 9877 9945 +68
- Misses 16785 16832 +47
- Partials 226 235 +9
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR hardens the CI-V stream reassembly logic (CivFrameSplitter) by bounding the carried-over remainder buffer so a terminator-less stream can’t grow memory usage unbounded and eventually OOM the app.
Changes:
- Add a hard cap (
MAX_BUFFERED_BYTES = 1024) to the carried-over remainder and resynchronization logic to the most recentFE FEpreamble on overflow. - Add unit tests covering terminator-less streams, overflow resync behavior, and edge cases (no preamble, trailing
0xFE, lone preamble with huge tail).
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| ft8af/app/src/main/java/com/k1af/ft8af/rigs/CivFrameSplitter.java | Bounds oversized reassembly remainder and resyncs to the most recent CI‑V preamble to prevent unbounded memory growth. |
| ft8af/app/src/test/java/com/k1af/ft8af/rigs/CivFrameSplitterTest.java | Adds coverage for malformed/terminator-less streams and overflow resynchronization edge cases. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Compute the resync copy start first and copy at most MAX_BUFFERED_BYTES bytes in one shot, rather than allocating a potentially huge intermediate array from the early FE FE preamble only to trim it back down. Behaviour is byte-identical to the previous two-branch form; the guard fires precisely in the malformed-stream case this bound exists to handle, where the extra allocation would spike peak memory the most. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The shared CI-V (Icom / Xiegu) stream reassembler,
CivFrameSplitter, bufferedthe trailing bytes after the last frame terminator (
0xFD) and carried them intothe next read via each rig's
dataBuffer. Nothing bounded that buffer: a CI-Vstream that never delivers a terminator — a rig connected at the wrong baud rate,
line noise on a serial link, or a misbehaving network peer streaming garbage —
made
dataBuffergrow by every received byte, without limit, until the appeventually ran out of memory.
This affects every CI-V rig, since
IcomRig,XieGuRig, andXieGu6100Rigallfeed their bytes through
CivFrameSplitter.split(...)and storeresult.remainderback into
dataBuffer.Root cause
CivFrameSplitter.splitreturned everything after the last0xFDas theremainder with no upper bound. Under well-formed traffic the remainder is a
command still in progress and stays tiny (a CI-V command this app exchanges is a
few to a few dozen bytes), so the growth only manifests when the terminator never
arrives — i.e. exactly the malformed-stream case a robust reassembler must
tolerate without falling over.
Fix
Bound the carried-over remainder in the one shared place it is produced
(
CivFrameSplitter.boundRemainder). When the remainder exceeds a hard cap(
MAX_BUFFERED_BYTES = 1024, orders of magnitude above any real CI-V command):FE FEpreamble — the earliest point avalid command could still begin — and drop the unparseable bytes before it.
Those bytes can never complete a frame (a valid one would have ended in
0xFDand already been split off), so this is lossless for well-formed traffic and
only ever trims garbage.
trailing
0xFE, in case a preamble is split across the read boundary.case of a lone preamble followed by a huge terminator-less run.
Behaviour is unchanged for all well-formed and normally-fragmented traffic (the
remainder never approaches the cap there), so decode/CAT interoperability is
untouched.
Testing
./gradlew :app:testDebugUnitTest --tests com.k1af.ft8af.rigs.CivFrameSplitterTestAdded pure-JVM cases to the existing
CivFrameSplitterTest:<= 1024bytes (fails before the fix — the remainder grows unbounded);
FE FEpreamble and the resyncedcommand still completes on the next chunk;
0xFE;All existing reassembly tests continue to pass (single/multi-frame,
split-across-callbacks, no stray bytes).
Risk
Low. The change is confined to the shared pure-logic helper and only alters
behaviour once the buffer exceeds a cap far above any legitimate CI-V frame;
normal and normally-fragmented traffic is byte-for-byte identical. No native, DSP,
encoder/decoder, or protocol changes.
🤖 Generated with Claude Code