MuCamp2 is published in IEEE Access.
- IEEE Xplore: https://ieeexplore.ieee.org/document/11552376
- DOI: https://doi.org/10.1109/ACCESS.2026.3700808
MuCamp2 is a generation-validation architecture for campaign-level TTP augmentation that places LLMs outside the trust boundary. Using deterministic candidate-pool constraints and multi-level semantic validation based on the MITRE ATT&CK Adversary Emulation Planner (AEP), MuCamp2 produces structurally consistent cyber campaign variants from authentic APT group TTP sequences and reports their semantic validity under explicitly stated AEP coverage limits.
Key Innovation: The architecture does not treat the LLM as a free generator. Instead, deterministic constraints enforce structural integrity, and the LLM provides selective refinement within a code-restricted candidate space; an independent AEP-based validator gates the output.
Improvements over the SR (Synonym Replacement) baseline are modest in absolute terms. We report absolute differences (percentage points) first; relative percentages are provided as secondary context.
| Metric | LLM Few-shot | SR Baseline | Absolute (relative) |
|---|---|---|---|
| L3 pass rate (Lazarus) | 15.7% | 10.2% | +5.5 pp (+54%) |
| L3 pass rate (MenuPass) | 10.5% | 7.5% | +3.0 pp (+40%) |
| Tactic violation rate | 0% | 26–32% | structurally precluded |
| Downstream RF Macro-F1 (matched n=1,402) | 0.988–0.989 | 0.980 | +0.008–0.009 |
| Downstream kNN Macro-F1 (matched n) | 0.976–0.982 | 0.936 | +0.040–0.046 |
| Cross-group class balance | 1.7:1 | 5.1:1 | more balanced |
The most robust gain is structural (tactic violations 26–32% → 0%). The strict AEP-based (L3) gain is modest in absolute terms but statistically significant (non-overlapping 95% bootstrap CIs for Lazarus). Downstream attribution gains at matched sample sizes are bounded (~0.008–0.009 RF Macro-F1 at n=1,402); the practical contribution is best understood as a balanced supply of usable variants across groups rather than a large per-sample LLM superiority.
AEP coverage of the validator is partial: AEP currently maps 197 of 356 ATT&CK v18 techniques (55.3%). Unmapped techniques have empty requires/provides sets, so they avoid failure in the capability walk by construction. As a result:
- L3 should be read as an AEP-based operational proxy, not as complete semantic ground truth.
- For sequences dominated by AEP-unmapped techniques (the low-coverage bin, <50%), the L3 pass rate reaches 98.5% — this reflects reduced validator discrimination, not higher intrinsic validity.
- All "validated" wording in this repository and the paper should be read as "validated under current AEP coverage."
Expanding AEP mappings and reporting coverage-qualified subsets remain important directions for future work.
MuCamp2 consists of two phases:
Phase 1 (Generation under constraint): A candidate pool is built from MITRE ATT&CK at the same-tactic level, and the LLM selects replacements within that code-restricted space using few-shot prompts.
Phase 2 (Validation outside the trust boundary): Code-level verification (V1–V4) and AEP-based multi-level semantic validation (L1–L3) are applied sequentially to gate variants for inclusion in the augmented dataset.
- L1 (Structural): Tactic order non-decreasing + goal tactics preserved
- L2 (Execution Feasibility): AEP requires-satisfaction rate ≥ 0.7
- L3 (Strict): L2 + fail budget ≤ 1 (at most one unsatisfied dependency)
L1–L3 are interpreted under the AEP coverage limitation noted above.
- V1: Length match (
|S'| = |S|) - V2: Index consistency (mutated indices match the pre-specified set)
- V3: Non-mutated position preservation
- V4: Candidate pool membership (technique IDs belong to same-tactic pool)
93 seed TTP sequences from two state-sponsored APT groups, derived from a corpus of 858 public security reports collected by the precursor work, MuCamp.
| Item | Lazarus | MenuPass | Total |
|---|---|---|---|
| Selected source reports | 106 | 47 | 153 |
| Seed sequences (min_ttp ≥ 5, has goal tactic) | 63 | 30 | 93 |
| Mean techniques per sequence | 12.43 | 14.55 | 13.08 |
| Unique techniques used | 106 | 77 | 121 |
| Reference data for distributional metrics (n_real) | 49 | 15 | 64 |
The 47 MenuPass source reports were exhaustively verified through a triple-pipeline alias matching process (PDF parse, OCR parse, processed) against 18 known group aliases (APT10, Stone Panda, Red Apollo, Cicada, etc.). For replication transparency, the repository includes (i) data/sources/source_report_list_lazarus_menupass.csv with the 153 selected source-report records and (ii) data/seeds/seed_sequences_attack_ids.csv with the 93 ATT&CK-ID seed sequences. The legacy ttp_sequences_with_meta_*.csv files are also included for script compatibility.
Three LLM models served via Ollama on a local GPU:
| Model | Parameters | Inference |
|---|---|---|
| gpt-oss:20b | 20B | Ollama, Local GPU |
| qwen3-vl:30b | 30B | Ollama, Local GPU |
| glm-4.7-flash | 30B | Ollama, Local GPU |
MuCamp2/
├── README.md
├── LICENSE
├── requirements.txt
├── .gitignore
│
├── scripts/ # Core pipeline scripts
│ ├── llm_mutation_fewshot.py # LLM few-shot variant generation (main)
│ ├── mucamp_mutation.py # SR baseline generation
│ ├── eval_variants_v18.py # Evaluation (PR, C2ST, L1/L2/L3)
│ ├── aggregate_eval_variants_ext.py # Result aggregation
│ ├── process_ttp.py # TTP sequence preprocessing
│ ├── util_env.py # Shared env loading
│ │
│ ├── ablation_constrained_sr.py # Ablation: SR vs Constrained-SR vs LLM
│ ├── bootstrap_ci_fast.py # Bootstrap confidence intervals
│ ├── downstream_attribution.py # Group attribution (RF/kNN, 5-fold CV)
│ ├── downstream_learning_curve.py # Quality vs quantity learning curves
│ ├── l1_failure_mechanism_analysis.py # L1 failure root cause analysis
│ ├── l3_failure_analysis.py # L3 failure root cause analysis
│ ├── aep_coverage_bias_analysis.py # AEP coverage bias analysis
│ ├── llm_selection_analysis.py # LLM selection distribution analysis
│ │
│ ├── plot_main_figure.py # Paper figures (architecture, example)
│ ├── plot_paper_results.py # Paper figures (quantitative results)
│ └── visualize_metrics_png.py # Metric visualization
│
├── data/
│ ├── sources/
│ │ └── source_report_list_lazarus_menupass.csv
│ ├── aep/
│ │ └── agent_promises_v18_MERGED.json # AEP Requires/Provides (197 techniques)
│ ├── attack_v18/
│ │ └── technique_to_tactic_v18.csv # MITRE ATT&CK v18 mapping
│ ├── filters/
│ │ └── exclude_titles.txt # Reports excluded from evaluation
│ ├── seeds/
│ │ ├── seed_sequences_attack_ids.csv
│ │ ├── ttp_sequences_with_meta_Lazarus.csv
│ │ ├── ttp_sequences_with_meta_MenuPass.csv
│ │ ├── ttp_preprocessed_Lazarus.csv
│ │ └── ttp_preprocessed_MenuPass.csv
│ └── sr_baseline/
│ ├── MUCAMP_mutations_Lazarus.csv
│ └── MUCAMP_mutations_MenuPass.csv
│
├── dataset/ # Compatibility mirror used by scripts
│ ├── attack_v18/technique_to_tactic_v18.csv
│ └── ttp/
│ ├── ttp_sequences_with_meta_{Lazarus,MenuPass}.csv
│ ├── filters/exclude_titles.txt
│ └── augmented/MUCAMP_mutations_{Lazarus,MenuPass}.csv
│
├── aep-data/data/agent_promises_v18_MERGED.json # Compatibility path used by validators
│
└── figures/ # Paper figures (PDF + PNG)
├── fig_architecture.{pdf,png} # Fig. 1: Pipeline overview
├── fig_ttp_example.{pdf,png} # Fig. 2: Mutation & validation example
├── fig_downstream.{pdf,png} # Fig. 3: Downstream attribution results
├── fig_learning_curve.{pdf,png} # Fig. 4: Learning curve (RF Macro-F1)
└── fig_aep_coverage.{pdf,png} # Fig. 5: AEP coverage bias analysis
The repository includes the MuCamp2 Attack Variant Workbench GUI in gui/.
# Docker deployment
docker compose up --build -d
# open http://localhost:8501 or http://SERVER_IP:8501
# Live generation defaults to OpenRouter; paste a key in the UI or set OPENROUTER_API_KEY.Local Streamlit run:
python3 -m venv .venv-gui
source .venv-gui/bin/activate
pip install -r gui/requirements.txt
streamlit run gui/app.pyGUI features:
- Select existing seed TTP sequences or paste a custom sequence.
- Replay SR, LLM zero-shot, and LLM few-shot variants.
- Generate one-shot live mutations through Ollama, OpenAI, OpenRouter, or an OpenAI-compatible API.
- Compare original vs mutated ATT&CK timelines.
- Validate L1/L2/L3 semantic constraints and AEP requires/provides traces.
- Export an incident-response scenario as Markdown.
See gui/README.md for deployment and provider configuration details.
git clone https://github.com/cyailab/MuCamp2.git
cd MuCamp2
python3.12 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txtCreate a .env file in the repository root:
# For local Ollama inference
OLLAMA_HOST=http://localhost:11434
# For remote providers (optional)
OPENAI_API_KEY=your_key_here
OPENROUTER_API_KEY=your_key_herepython scripts/process_ttp.py --groups Lazarus MenuPassLLM Few-shot (main method):
python scripts/llm_mutation_fewshot.py \
--provider ollama --model gpt-oss:20b \
--single-call --groups Lazarus MenuPassSR Baseline:
python scripts/mucamp_mutation.py --groups Lazarus MenuPass --Nm 10# Evaluate LLM variants
python scripts/eval_variants_v18.py \
--root TID_TNAME_FEWSHOT --model gpt-oss_20b_newfewshot \
--groups Lazarus MenuPass --embed svd --semantic-level 3 \
--out-dir runs/eval
# Aggregate results
python scripts/aggregate_eval_variants_ext.py# Ablation study (SR vs Constrained-SR vs LLM)
python scripts/ablation_constrained_sr.py --groups Lazarus MenuPass
# Downstream attribution (RF/kNN 5-fold CV)
python scripts/downstream_attribution.py --groups Lazarus MenuPass
# Bootstrap confidence intervals
python scripts/bootstrap_ci_fast.py --groups Lazarus MenuPass
# Learning curve analysis
python scripts/downstream_learning_curve.py --groups Lazarus MenuPass
# Failure analysis
python scripts/l1_failure_mechanism_analysis.py --groups Lazarus MenuPass
python scripts/l3_failure_analysis.py --groups Lazarus MenuPass
# AEP coverage bias
python scripts/aep_coverage_bias_analysis.py --groups Lazarus MenuPass
# LLM selection distribution (entropy, KL-divergence, chi-square)
python scripts/llm_selection_analysis.py --groups Lazarus MenuPasspython scripts/plot_main_figure.py
python scripts/plot_paper_results.py
python scripts/visualize_metrics_png.py --input <summary.csv> --output figures/- kNN Precision/Recall: Coverage and fidelity on 50-dim TF-IDF+SVD embeddings
- C2ST (Classifier Two-Sample Test): Binary classifier distinguishability (0.5 = ideal)
- L1/L2/L3 pass rates: Three-level semantic validation (see definitions and limitations above)
- Tactic violation rate: Frequency of invalid tactic orderings (structurally precluded by candidate-pool constraints)
- Macro-F1 (RF/kNN): Group attribution accuracy with 5-fold stratified cross-validation
The call_llm() function in llm_mutation_fewshot.py supports three providers:
| Provider | Method | Notes |
|---|---|---|
ollama |
HTTP POST (urllib) | Local GPU inference |
openrouter |
curl subprocess + exponential backoff | Remote API |
openai |
OpenAI API (urllib) | Remote API |
If you use MuCamp2 in your research, please cite:
@article{kim2026mucamp2,
title={MuCamp2: Generating Validated Cyber Campaign Variants via Constrained LLMs for Group Attribution},
author={Kim, Keunho and Lee, Jimin and Lee, Insup and Han, Insung and Lee, Jungsik and Shim, Shinwoo and Choi, Changhee},
journal={IEEE Access},
year={2026},
doi={10.1109/ACCESS.2026.3700808},
url={https://ieeexplore.ieee.org/document/11552376}
}This project is licensed under the Apache License 2.0. See LICENSE for details.
- Corresponding Author: Changhee Choi (choich@sejong.ac.kr)
- Lab: CYAI Lab, Sejong University
The data/ directory contains the curated supplementary data files. The dataset/ and aep-data/ directories mirror the same inputs at the legacy paths expected by the released scripts, so the commands above can be run immediately after cloning the repository. LLM generation requires a local Ollama server or remote API credentials; deterministic SR-baseline generation and metadata checks can be run without API credentials.