Read bundled data assets fully so a short read can't corrupt the callsign/DXCC/zone databases

[patrickrb]

Root cause

The bundled reference-data assets are read with:

byte[] bytes = new byte[inputStream.available()];
inputStream.read(bytes);            // return value ignored

Both halves are unsafe:

• InputStream.available() is only a hint, not the length.
• A single read(byte[]) is explicitly permitted to return fewer bytes than requested.

Android's AssetInputStream decompresses assets on the fly, so a large compressed asset is handed back in chunks. The lone read keeps only the first chunk; the tail of the buffer stays as NUL bytes, silently truncating the file.

Impact

• cty.dat (~280 KB) — the callsign→country / CQ-zone / ITU-zone map (CallsignFileOperation). Truncation drops whole countries, so callsigns past the first chunk resolve to the wrong (or no) country/zone in the decode list, on the map, and in ADIF export.
• ituzone.json / cqzone.json / dxcc_list.json (450–740 KB) — loaded in DatabaseOpr and parsed as JSON immediately after the read. A truncated buffer makes new JSONObject(new String(bytes)) throw JSONException, which is caught and logged — wiping the entire zone/DXCC table.
• bands.txt / rigaddress.txt / help texts — smaller, but the same latent defect (OperationBand, RigNameList, HelpDialog, ClearCacheDataDialog).

This is the same class of bug the team already fixed for log import in LogFileImport.readFully (which carries a detailed comment on exactly this hazard). These eight sites were the remaining copies of the pattern.

Fix

Added com.k1af.ft8af.util.Streams.readAllBytes(InputStream), which drains the stream to EOF in a loop (8 KB working buffer, available() used only as an initial sizing hint). Routed all eight call sites through it. Decoding is otherwise unchanged — still new String(bytes) with the platform default charset — so this is strictly a "read the whole file" fix with no behavioral change on inputs that already fit in one read.

Files: CallsignFileOperation, RigNameList, OperationBand, DatabaseOpr (×3), HelpDialog, ClearCacheDataDialog, + new Streams.

Testing

• StreamsTest (new, pure JVM): readAllBytes fully drains a stream that yields one byte per read, and one that under-reports available(); plus exact-content and empty-stream cases.
• CallsignFileOperationTest (new case): feeds getLinesFromInputStream a chunked (one-byte-per-read) stream of 2000 ;-separated records and asserts every record survives. Verified this fails against the old single-read code and passes with the fix.
• ./gradlew :app:testDebugUnitTest — full suite green.
• ./gradlew :app:assembleDebug — full APK (all 4 ABIs, native + hamlib) builds successfully.

Risk

Low. No DSP/native/protocol changes. The only behavioral change is that these bundled files are now read in full instead of a truncated prefix; on any input small enough that the old single read already succeeded, output is byte-identical.

[Copilot]

Pull request overview

Fixes a long-standing asset-loading bug where bundled reference-data files could be silently truncated due to unsafe available() sizing and single-call read(byte[]) usage, leading to corrupted callsign/DXCC/zone data at runtime.

Changes:

• Added com.k1af.ft8af.util.Streams.readAllBytes(InputStream) to drain streams to EOF reliably.
• Routed multiple asset-loading call sites to use Streams.readAllBytes.
• Added/extended pure-JVM unit tests to regress short-read / under-reported-available scenarios.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
ft8af/app/src/main/java/com/k1af/ft8af/util/Streams.java	New helper to fully read InputStream contents to EOF.
ft8af/app/src/main/java/com/k1af/ft8af/callsign/CallsignFileOperation.java	Uses Streams.readAllBytes when splitting cty.dat content.
ft8af/app/src/main/java/com/k1af/ft8af/database/DatabaseOpr.java	Uses Streams.readAllBytes for DXCC/CQ/ITU JSON asset reads.
ft8af/app/src/main/java/com/k1af/ft8af/database/RigNameList.java	Uses Streams.readAllBytes for rigaddress.txt parsing helper.
ft8af/app/src/main/java/com/k1af/ft8af/database/OperationBand.java	Uses Streams.readAllBytes for bands.txt parsing helper.
ft8af/app/src/main/java/com/k1af/ft8af/ui/HelpDialog.java	Uses Streams.readAllBytes when reading help text assets.
ft8af/app/src/main/java/com/k1af/ft8af/ui/ClearCacheDataDialog.java	Uses Streams.readAllBytes when reading dialog/help assets.
ft8af/app/src/test/java/com/k1af/ft8af/util/StreamsTest.java	New unit tests covering short reads and lying available().
ft8af/app/src/test/java/com/k1af/ft8af/callsign/CallsignFileOperationTest.java	New regression test ensuring chunked reads don’t truncate semicolon records.

Comments suppressed due to low confidence (3)

ft8af/app/src/main/java/com/k1af/ft8af/database/RigNameList.java:106

• getLinesFromInputStream returns null on IOException, but callers (e.g., getRigNamesFromFile) assume a non-null array and will crash with NullPointerException. Prefer returning an empty array (and logging) or throwing a checked/unchecked exception with the cause so failures don’t turn into NPEs.

            byte[] bytes = Streams.readAllBytes(inputStream);
            return (new String(bytes)).split(deLimited);
        }catch (IOException e){
            return null;
        }

ft8af/app/src/main/java/com/k1af/ft8af/database/OperationBand.java:373

• getLinesFromInputStream returns null on IOException, but getBandsFromFile passes the result to parseBandLines without a null check, which will crash with NullPointerException. Prefer returning an empty array (and logging) or throwing an exception with the cause.

            byte[] bytes = Streams.readAllBytes(inputStream);
            return (new String(bytes)).split(deLimited);
        }catch (IOException e){
            return null;
        }

ft8af/app/src/main/java/com/k1af/ft8af/callsign/CallsignFileOperation.java:62

• getLinesFromInputStream returns null on IOException, but getCallSingInfoFromFile assumes a non-null array (uses st.length) and will crash with NullPointerException if an asset read fails. Prefer returning an empty array (and logging) or throwing an exception with the original cause.

            byte[] bytes = Streams.readAllBytes(inputStream);
            return (new String(bytes)).split(deLimited);
        }catch (IOException e){
            return null;
        }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.