NSRL Whitelisting

NSRL-Backed Whitelisting

HydraDragonAV Mobile uses the National Software Reference Library (NSRL) to maintain a comprehensive whitelist of known-good software. This ensures legitimate apps are never flagged as malicious.

Dual-Layer Architecture

The whitelist operates across two independent layers:

Layer 1: Binary-Fuse XOR Filter (In-Memory)

• Contents: Whole-file MD5 hashes from NSRL
• Storage: whitelist.xf — a Binary-Fuse16 (Bf16) XOR filter loaded as a memory-mapped asset (AAsset_getBuffer) at init
• Implementation: Native Rust (hydradragonxorfilter), loaded once into Engine.whitelist
• Speed: Nanosecond-level hash checks — O(1), orders of magnitude faster than any lookup table
• Limitation: Hash membership only — no metadata

Layer 2: CSV Package Whitelist

• Contents: key → md5 pairs parsed from whitelist_packages.csv (format: one "key",md5 per line, RFC-4180)
• Storage: whitelist_packages.csv — bundled with the APK in assets/scan/, read once at init into Engine.package_whitelist: HashMap<String, String>
• Lookup: By package name key + whole-APK MD5 — both fields must match to whitelist a file (prevents spoofed package names)
• Purpose: Fast benign-skip for known-good APKs whose package + MD5 are in the NSRL set

┌──────────────────────┐
│   File MD5 Hash      │
└──────────┬───────────┘
           │
    ┌──────▼───────┐
    │ Layer 1:     │
    │ XOR Filter   │───── NOT FOUND ───► Proceed to detection engines
    │ (.xf asset)  │
    └──────┬───────┘
           │ FOUND
    ┌──────▼───────────┐
    │ Layer 2:         │
    │ CSV Package List │───── NOT FOUND ───► Proceed to detection engines
    │ (HashMap)        │
    └──────┬───────────┘
           │ FOUND (key + md5 match)
    ┌──────▼──────────┐
    │   WHITELISTED   │
    │   → Skip all    │
    │   detectors     │
    └─────────────────┘

Why Two Layers?

Layer	Speed	Metadata	Storage
XOR Filter (.xf)	~nanoseconds	None (hash only)	~10–20 MB in-memory
CSV Package List	~microseconds	Package key + MD5	In-memory HashMap

The XOR filter handles the common case (95%+ of scanned files are known-good) at maximum speed. Only when a hash matches the XOR filter does the system check the CSV package list for a key+MD5 confirmation.

Security Properties

• No false negatives for known-good: A legitimate NSRL-indexed app is never flagged
• No hash collision attacks: A malicious file cannot borrow a whitelist entry — the MD5 must match exactly
• Package spoofing resistant: Layer 2 requires BOTH the package key AND the MD5 to match — a renamed APK is not whitelisted
• Tamper-proof: .xf filter and CSV are read-only assets, never modified at runtime

Generation

Whitelist assets are generated offline from the NSRL RDS (Reference Data Set) SQLite database using two scripts:

# Layer 1: XOR filter from NSRL MD5 hashes (whole-APK)
# Input: NSRL RDS SQLite → Output: whitelist.xf
python gen_whitelist_apk.py

# Layer 2: CSV package whitelist (key,md5 pairs)
# Input: NSRL RDS SQLite → Output: whitelist_packages.csv
python gen_whitelist_packages.py

The SQLite database is only used during offline generation — it is not bundled in the APK. Only the compiled outputs (whitelist.xf and whitelist_packages.csv) are shipped as assets. See Data-Pipeline for details.

See Also

• Detection-Engines — How scanning works with whitelisting
• Data-Pipeline — Whitelist generation workflow