Skip to content

AI ML Models

Jump to bottom
Emirhan Uçan edited this page Aug 2, 2026 · 13 revisions

AI / ML Models

HydraDragonAV Mobile employs a Burn (wgpu) binary classifier for APK malware detection, plus a Java-side logistic-regression heuristic (AIEngine) used only as a corroboration signal. There is also a separate MinHash/Jaccard benign-content whitelist (benign_db.rs) that is sometimes confused with the ML model — it is not the classifier; it is a fast benign-skip path.

Earlier docs described the model as "MinHash/LSH + Isolation Forest" and later as "a tract-onnx feature-hash classifier". Both are incorrect — there is no LSH or Isolation Forest in the codebase, and the classifier is now a Burn neural network (training and inference in pure Rust, no ONNX). MinHash exists, but in benign_db.rs (the benign-content skip), not in the ML crate.

For the function-level reference, see Rust-API-Reference#hydradragonml--burn-classifier and Java-API-Reference#aienginejava-78-lines.

1. Burn Binary Classifier (hydradragonml)

Technique

APK string extraction → vocabulary tokenization → Embedding + mean-pool fused with 26 engine features → MLP → sigmoid confidence:

  1. Tokenizer (features.rs): opens the APK as a zip::ZipArchive, harvests printable ASCII strings from entry names, AndroidManifest.xml, resources.arsc, all classes*.dex, and META-INF/*, splits each fragment on delimiters (. / ; : - \ _), and maps it to a vocabulary index via vocab.json (20K subword tokens, 0 = UNK, capped at 120,000 tokens). The same vocab.json is used for training and inference so both tokenize identically.
  2. Model (model.rs, ApkClassifier): a Burn (wgpu) neural network trained offline on malware + benign APK corpora. Text branch: Embedding(20K → 64) → mean-pool → Linear(64 → 32) → ReLU. Engine branch: Linear(26 → 32) → ReLU over the 26 normalized EngineFeatures. The two 32-d vectors are concatenated and passed through Linear(64 → 32) → ReLU → Linear(32 → 1) → Sigmoid, producing a single f32 confidence in [0.0, 1.0].
  3. Engine features (EngineFeatures): the same 26 normalized signals the Android engine computes during a real scan (DEX/ELF/manifest/URL/IP/certificate/benign-DB/media/HIPS, each scaled to [0, 1]) are fused with the text embeddings. Fields the trainer cannot compute (emulation, IP/cert lookups, benign-DB) stay at neutral defaults at training time.
  4. Thresholds (in lib.rs): DEFAULT_CONFIDENCE_THRESHOLD = 0.95 (malicious), SUSPICIOUS_THRESHOLD = 0.90 (suspicious if not malicious). Configurable via Model::set_threshold (clamped 0.0–1.0). Note the actual default is 0.95, not 0.5 as one doc comment says.

Why Not More Features?

The feature set is deliberately limited to string-based tokens. Adding opcode n-grams, ELF header analysis, control-flow graphs, or API call graphs introduces real risks:

  • Curse of dimensionality: when features exceed training samples, the model memorises noise rather than generalising.
  • Overfitting: every new feature is a new axis the model can exploit to separate training samples by accident; on unseen malware these collapse, degrading recall.
  • Feature decay: opcode distributions, ELF structures, and API graphs change rapidly across Android API levels and compiler versions. String features (permissions, URLs, class names, intent actions) are far more stable.
  • Device constraints: more features → larger model → more RAM and slower inference on-device. A 20K-token embedding plus 26 engine features is a proven trade-off.

Implementation

hydradragonml/
├── src/
│   ├── lib.rs           # Model loading + inference (Burn/wgpu)
│   ├── features.rs      # Tokenizer + EngineFeatures (26-d) + MinHash
│   ├── model.rs         # ApkClassifier (Burn module)
│   └── main.rs          # Dataset scanner CLI (hydradragonml-scan)
└── src/bin/
    ├── train.rs         # Training binary (hydradragonml-train)
    ├── debug_features.rs
    └── dump_features.rs

Dataset Scanner CLI

A standalone binary that batch-scans the entire dataset/ folder:

cargo run --release --bin hydradragonml-scan -- `
  --dataset ..\dataset\ `
  --model ..\app\src\main\assets\scan\model.mpk `
  --vocab ..\vocab.json `
  --threshold 0.95

--model and --vocab are required; without them no APK can be scored. ground_truth(path) infers the expected label from path substrings (benign/clean/f-droid → false; malware/malicious/malwarebazaar → true). Output: per-file verdict + confidence + elapsed, then accuracy/precision/recall/F1 vs folder labels.

Training

Training is a Rust binary (hydradragonml-train, src/bin/train.rs) using Burn autodiff on the wgpu backend. It walks the benign/malware corpus, tokenizes each APK and extracts its engine features, does an 80/20 train/validation split, and trains ApkClassifier with Adam + binary cross-entropy (LR halves each epoch), then saves the weights as a Burn .mpk file:

cargo run --release --bin hydradragonml-train -- `
  --benign ..\dataset\benign --malware ..\dataset\malware `
  --vocab ..\vocab.json --output model.mpk `
  [--epochs 6] [--lr 0.001] [--batch-size 8]

The --vocab file is the same vocab.json shipped in the Android assets; it must be built over the same corpus so training and inference tokenize identically. The trainer uses the real FossRust dex-core/dex-analysis DEX parser (the same one the Android engine uses at serving time) for the DEX feature signals, and a minimal AXML manifest parser for the manifest signals.

Dataset hygiene is critical: a malware APK in a benign/ folder trains the model to call malware "benign", and vice versa. Verify the split first. The dump_features binary writes the extracted features for inspection.

On-device consumption

hydradragonandroid/src/lib.rs loads the model once at init into Engine.model: Option<Model> (from model.mpk weights + vocab.json, both shipped as assets) and scores each non-whitelisted APK buffer via Model::scan_with_features during run_scan Phase 3a. The Java side reads ml.malicious/ml.jaccard/ml.anomaly/ml.nearest from the verdict JSON; an ML flag counts as malicious only if DetectionCategories.ML is enabled AND jaccard >= 0.55 AND anomaly >= 0.33 (see Java-API-Reference).

Strengths

  • Detects zero-day malware that shares no signatures with known threats.
  • No training on the device — model is pre-trained offline.
  • Low false positive rate with clean dataset separation.

2. AIEngine (Logistic Regression) — corroboration only

A lightweight Java-side logistic-regression classifier that operates on 8 boolean code-feature flags produced by CodeAnalyzer (obfuscation, dynamic loading, crypto/socket/shell APIs, adware SDKs, dangerous permissions). BIAS = -2.0; sigmoid → 0–100. Returns an anomaly result only if aiScore > 50 && activeFeatures >= 3, else clean. Also classifies a threatType (RANSOMWARE/SPYWARE/ADWARE/TROJAN) from which feature is set.

Its verdict is applied by ScanEngine only when the real (native) engine corroborates it — it is not an independent detection path. See Java-API-Reference#aienginejava-78-lines.

3. MinHash/Jaccard Benign Whitelist (benign_db.rs) — not the ML model

hydradragonandroid/src/benign_db.rs uses MinHash (64 permutations, FNV-1a) + Jaccard similarity (threshold 0.85) to recognize a known-benign APK by its content and skip the heavy ClamAV/ML pass. It loads benign_signatures.bin (built offline by gen_benign_signatures.py) and declares KNOWN_BENIGN if any stored signature for the package has estimated Jaccard similarity ≥ 0.85. This is a whitelist optimization, not an anomaly detector. See Rust-API-Reference#benign_dbrs--minhashjaccard-benign-content-whitelist.

See Also

Clone this wiki locally