Skip to content

AI ML Models

Jump to bottom
Emirhan Uçan edited this page Aug 5, 2026 · 13 revisions

AI / ML Models

HydraDragonAV Mobile employs a Burn (ndarray) binary classifier for APK malware detection, plus a Java-side logistic-regression heuristic (AIEngine) used only as a corroboration signal. There is also a separate MinHash/Jaccard benign-content whitelist (benign_db.rs) that is sometimes confused with the ML model — it is not the classifier; it is a fast benign-skip path.

Earlier docs described the model as "MinHash/LSH + Isolation Forest" and later as "a tract-onnx feature-hash classifier". Both are incorrect — there is no LSH or Isolation Forest in the codebase, and the classifier is a Burn neural network (training and inference in pure Rust, no ONNX). MinHash exists, but in benign_db.rs (the benign-content skip), not in the ML classifier.

For the function-level reference, see Rust-API-Reference#hydradragonml--burn-classifier and Java-API-Reference#aienginejava-78-lines.

1. Burn Binary Classifier (hydradragonml)

Technique

APK string extraction → vocab.json subword tokenization → Embedding + mean-pool fused with 11 engine features → MLP → sigmoid confidence:

  1. Tokenizer (features.rs, Tokenizer): loads a subword token vocabulary from vocab.json (Tokenizer::load_json, a JSON map token → id, with 0 reserved for <UNK>), parses the APK via ripzip, harvests printable ASCII/UTF-16LE strings from entry names, AndroidManifest.xml, resources.arsc, all classes*.dex, and META-INF/*, splits each fragment on delimiters (. / ; : - \ _), lowercases it, and looks it up in the vocabulary — unknown fragments fall back to id 0 (<UNK>). Token streams are capped at MAX_TOKENS (8192). Training, hydradragonml-scan, and on-device inference must all use the same vocab.json so they tokenize identically.
  2. Model (model.rs, ApkClassifier): a Burn (ndarray / autodiff) neural network trained offline on malware + benign APK corpora. Text branch: Embedding(20K → 64) → mean-pool over tokens → Linear(64 → 32) → ReLU. Engine branch: Linear(11 → 32) → ReLU over the 11 corpus-percentile-normalized EngineFeatures. Fused head: cat([text 32-d, engine 32-d]) = 64-d → Linear(64 → 32) → ReLU → Linear(32 → 1) → Sigmoid, producing a single f32 confidence in [0.0, 1.0].
  3. Engine features (EngineFeatures): 11 content-derived signals computed directly from the APK's DEX files (3 fields), ELF shared objects (1 field), binary AndroidManifest (6 fields), and a Shannon-entropy feature over the decompressed DEX/ELF/manifest bytes (1 field). Every feature is derived strictly from file contents; no placeholder or external-reputation fields remain.
  4. Normalization (FeaturePercentiles): raw feature values are not normalized by hardcoded caps — at train time each feature's raw values across the whole corpus are sorted per-column and persisted to features.json next to the model; at inference each raw value is mapped to its linear-interpolated rank percentile in that training distribution (0.0–1.0). Scan and train both load the same features.json so they normalize identically.
  5. Thresholds (in lib.rs): DEFAULT_CONFIDENCE_THRESHOLD = 0.95 (malicious), SUSPICIOUS_THRESHOLD = 0.90 (suspicious if not malicious). Configurable via Model::set_threshold (clamped 0.0–1.0).

Implementation

hydradragonml/
├── src/
│   ├── lib.rs           # Model loading + inference API
│   ├── main.rs          # hydradragonml-scan CLI binary
│   ├── model.rs         # ApkClassifier (Burn module)
│   ├── training.rs      # TrainStep/InferenceStep + ApkBatcher glue for burn-train
│   ├── features.rs      # EngineFeatures, FeaturePercentiles, Tokenizer & for_each_entry
│   ├── axml.rs          # Binary AndroidManifest.xml parser
│   ├── dex.rs           # DEX parser
│   ├── elf.rs           # ELF parser
│   └── bin/
│       ├── train.rs     # Training binary (hydradragonml-train)
│       └── build_vocab.rs  # Vocabulary builder (builds vocab.json over a corpus)

Dataset Scanner CLI (hydradragonml-scan)

A standalone binary that walks a dataset folder and scores every .apk:

cargo run --release --bin hydradragonml-scan -- `
  --dataset ..\dataset\ --model ..\app\src\main\assets\scan\model.mpk `
  --vocab vocab.json --features features.json [--threshold 0.5] [--dump-features]

Prints a per-file verdict (MALICIOUS / SUSPICIOUS / BENIGN) + confidence, then a metrics summary (TP/FP/TN/FN, accuracy, precision, recall, F1) computed against the benign/malware labels inferred from the dataset folder structure. --features features.json is required (it carries the corpus percentile normalization loaded at inference), and --dump-features prints each file's 11 normalized feature values for diagnostics.

Training (hydradragonml-train)

Training is a Rust binary (hydradragonml-train, src/bin/train.rs) using Burn autodiff with the burn-train Learner (SupervisedTraining). It walks the benign/malware corpora, tokenizes each APK with the --vocab vocabulary and extracts its engine features, derives the per-feature corpus percentiles (FeaturePercentiles::from_samples), normalizes every sample against them, performs an 80/20 train/validation split, and trains ApkClassifier with Adam + binary cross-entropy using a step LR scheduler that halves the learning rate each epoch. It saves the weights as a Burn .mpk file and writes two companion files next to it: vocab.json (copied from --vocab) and features.json (the percentiles).

The vocabulary must be built over the same corpus used for training so training and inference tokenize identically — pass the same vocab.json that ships in the Android assets:

cargo run --release --bin hydradragonml-train -- `
  --benign ..\dataset\benign --malware ..\dataset\malware `
  --vocab vocab.json `
  --output model.mpk [--epochs 6] [--lr 0.001] [--batch-size 8]

The trainer uses the in-crate DEX parser (features/dex.rs), AXML parser (features/axml.rs), and ELF parser (features/elf.rs).

On-device consumption

hydradragonandroid/src/lib.rs loads the model once at init into Engine.model: Option<Model> from the model.mpk weights plus the vocab.json vocabulary and features.json percentile stats shipped as assets (Model::load(model_bytes, vocab_bytes, feature_stats, device)), and scores each non-whitelisted APK buffer via Model::scan_with_features(data, engine_features) during run_scan Phase 3a. All three files must be bundled — the tokenizer requires vocab.json and inference requires features.json at load time.

2. AIEngine (Logistic Regression) — corroboration only

A lightweight Java-side logistic-regression classifier that operates on 8 boolean code-feature flags produced by CodeAnalyzer (obfuscation, dynamic loading, crypto/socket/shell APIs, adware SDKs, dangerous permissions). BIAS = -2.0; sigmoid → 0–100. Returns an anomaly result only if aiScore > 50 && activeFeatures >= 3, else clean.

3. MinHash/Jaccard Benign Whitelist (benign_db.rs) — not the ML model

hydradragonandroid/src/benign_db.rs uses MinHash (64 permutations, FNV-1a) + Jaccard similarity (threshold 0.85) to recognize a known-benign APK by its content and skip the heavy ClamAV/ML pass. It loads benign_signatures.bin and declares KNOWN_BENIGN if any stored signature for the package has estimated Jaccard similarity ≥ 0.85. This is a whitelist optimization, not an anomaly detector.

See Also

Clone this wiki locally