EVIL-Detect

Code for the paper EVIL-Detect for NLPCC 2026 Shared Task 6: LLM-Generated Text Detection. EVIL-Detect ranked first in NLPCC 2026 Shared Task 6.

EVIL-Detect is a multi-signal Chinese text detector for three labels:

• 0 - human-written text (HWT)
• 1 - LLM-generated text (LGT)
• 2 - LLM-refined text (HLT)

The system combines EditLens-style editing-extent regression, Soft-EditLens semantic signals, EchoPrompt likelihood contrasts, lexical statistics, conflict-aware fusion, and conservative text rules.

Results

Phase	Samples	Macro-F1	Accuracy	HWT-F1	LGT-F1	HLT-F1
testp1	3,600	0.8913	0.8911	0.9083	0.9267	0.8391
testp2	1,152	0.8888	0.8880	0.9039	0.9219	0.8407

Released scope

This repository contains only the methods used by the paper's final system:

1. EditLens - character n-gram editing-extent targets and Qwen3.5-4B-Base regression.
2. Soft-EditLens - phrase-level semantic targets with regression and ordinal bucket heads.
3. EchoPrompt - likelihood-contrast votes from base/instruction model pairs.
4. Lexical statistics - label-wise character n-gram lexicons and log-odds features.
5. Conflict-aware fusion - calibrated base decisions, nine LGT-support votes, pair-specific conflict handling, and high-precision rules.

Development-only alternatives reported in the paper, such as direct generative SFT and Binoculars, are intentionally not included because they are not part of EVIL-Detect.

Repository layout

configs/                 model and fusion configuration examples
docs/                    data and reproduction notes
scripts/data/            official-data cleaning
scripts/editlens/        EditLens target, training, calibration, and scoring
scripts/soft_editlens/   Soft-EditLens target, training, and scoring
scripts/echoprompt/      zero-shot likelihood-contrast votes
scripts/lexical/         lexical lexicon construction and feature scoring
scripts/fusion/          conflict-aware integration and final ZIP generation
tests/                   CPU-only tests for deterministic components

Installation

Python 3.11, PyTorch 2.5+, and CUDA 12.1 are recommended. The supervised models were trained on NVIDIA V100 GPUs.

conda env create -f environment.yml
conda activate evildetect

Alternatively:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

The model IDs used in the paper are listed in configs/models.example.yaml. Model paths can be Hugging Face IDs or local directories.

Data

Competition data is not redistributed by this repository. Obtain it from the official NLPCC 2026 Task 6 repository and follow its license and access conditions.

Expected input format:

[
  {"id": "sample-1", "text": "待检测的中文文本"}
]

Training records may use either individual text/label rows or grouped HWT/LGT/HLT fields, depending on the script. See docs/reproduction.md.

Quick validation

The deterministic fusion logic can be tested without downloading model weights:

python -m unittest discover -s tests -v
python scripts/fusion/run_pipeline.py --config configs/fusion.example.json --dry-run

Reproduction

Generate each component's score or vote files by following docs/reproduction.md. Then edit configs/fusion.example.json so that it points to those artifacts and run:

python scripts/fusion/run_pipeline.py --config configs/fusion.example.json

The final submission files are written to:

outputs/evildetect/final/prediction.json
outputs/evildetect/final/prediction.zip

The ZIP archive contains exactly one file named prediction.json.

Model weights

Base models are downloaded from Hugging Face. LoRA adapters are not stored in Git; the included training scripts reproduce them from the official task data. Keep local adapters under checkpoints/ or override the corresponding command-line paths.

License

Code is released under the MIT License. Model and dataset licenses remain with their respective owners.